For the strapdown inertial navigation system’s system level calibration installation error matrix contains three groups of coupling relationship problems, a decoupling method based on matrix decomposition is proposed. The installation error matrix is decomposed into symmetric matrices and skew symmetric matrices at first. Then the influence is deduced to the strapdown inertial navigation system’s attitude equation and the rate equation. Under this deduced conclusions, the reason why there are coupling installation error matrices in the calibration of the system level is revealed, that is, influence of skew symmetric error and gyro instrument group skew symmetric error of the accelerometer instrument cluster on the system level calibration is coupled. And then a decoupling method that appropriately selects the reference frame to set the gyro instrument skew symmetric matrix to zero is proposed. The decoupling method is a kind of average allocation method. Compared with the traditional decoupling method, this method can reduce the second order truncation error of system operation. Simulation example shows that the decoupling method is effective.