We present a novel approach to estimate the relative sensor orientation from inertial sensors placed on connected body segments. Drift in the relative orientation estimates obtained by integrating the gyroscope measurements is corrected solely by incorporating common information in the inertial sensor measurements due to the connection of the body segments. We solve the estimation problem using a complementary filtering implementation to reduce the computational complexity. We study its robustness under common real-world model violations, e.g., soft tissue artifacts and spikes in the acceleration signals due to impacts. The efficacy of the method is illustrated with numerical simulations and is compared to a multiplicative extended Kalman filter implementation, both with and without outlier rejection. In addition, a human experiment strengthened the simulation results under realistic sensor errors.