3D Kinematics Estimation with Biomechanics Model

Abstract

Human 3D kinematics estimation involves measuring joint angles and body segment scales to quantify and analyze the mechanics of human movements. It has applications in areas such as injury prevention, disease identification, and sports science. Conventional marker-based motion capture methods are expensive both in terms of financial investment and the expertise required. On the other hand, due to the scarcity of large-scale annotated datasets, existing markerless motion capture methods suffer from challenges including unreliable 2D keypoint detection, limited anatomic accuracy, and low generalization capability.

In this work, we are the first to propose a pipeline to create synthetic data with accurate kinematics annotations by aligning the body mesh from the SMPL-X model and the biomechanics skeleton from OpenSim. The generated dataset, named ODAH, exhibits diverse variations in body shapes, clothing, lighting, and camera views. For kinematics estimation, we develop a novel biomechanics-aware model that is exclusively trained on ODAH, and directly tested on real-world data. Our extensive experiments demonstrate that the proposed approach outperforms previous state-of-the-art methods when evaluated across multiple datasets, revealing the potential for advancing the resolution of human 3D kinematics estimation.