The ability to capture joint kinematics in outside-laboratory environments is clinically relevant. In order to estimate kinematics, inertial measurement units can be attached to body segments and their absolute orientations can be estimated. However, the heading part of such orientation estimates is known to drift over time, resulting in drifting joint kinematics. This study proposes a novel joint kinematic estimation method that tightly incorporates the connection between adjacent segments within a sensor fusion algorithm, to obtain drift-free joint kinematics. Drift in the joint kinematics is eliminated solely by utilizing common information in the accelerometer and gyroscope measurements of sensors placed on connecting segments. Both an optimization-based smoothing and a filtering approach were implemented. Validity was assessed on a robotic manipulator under varying measurement durations and movement excitations. Standard deviations of the estimated relative sensor orientations were below 0.89° in an optimization-based smoothing implementation for all robot trials. The filtering implementation yielded similar results after convergence. The method is proven to be applicable in biomechanics, with a prolonged gait trial of 7 minutes on 11 healthy subjects. Three-dimensional knee joint angles were estimated, with mean RMS errors of 2.14°, 1.85°, 3.66° in an optimization-based smoothing implementation and mean RMS errors of 3.08°, 2.42°, 4.47° in a filtering implementation, with respect to a golden standard optical motion capture reference system.