3D Kinematics Estimation with Biomechanics Model

3D Kinematics Estimation with Biomechanics Model

Lin, Zhi-Yi (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Pattern Recognition and Bioinformatics)

van Gemert, J.C. (mentor)
Zhang, X. (mentor)
Kellnhofer, P. (graduation committee)

Delft University of Technology

Computer Science

2023-07-21

Human 3D kinematics estimation involves measuring joint angles and body segment scales to quantify and analyze the mechanics of human movements. It has applications in areas such as injury prevention, disease identification, and sports science. Conventional marker-based motion capture methods are expensive both in terms of financial investment and the expertise required. On the other hand, due to the scarcity of large-scale annotated datasets, existing markerless motion capture methods suffer from challenges including unreliable 2D keypoint detection, limited anatomic accuracy, and low generalization capability.

In this work, we are the first to propose a pipeline to create synthetic data with accurate kinematics annotations by aligning the body mesh from the SMPL-X model and the biomechanics skeleton from OpenSim. The generated dataset, named ODAH, exhibits diverse variations in body shapes, clothing, lighting, and camera views. For kinematics estimation, we develop a novel biomechanics-aware model that is exclusively trained on ODAH, and directly tested on real-world data. Our extensive experiments demonstrate that the proposed approach outperforms previous state-of-the-art methods when evaluated across multiple datasets, revealing the potential for advancing the resolution of human 3D kinematics estimation.

Computer Vision
3D Human Pose Estimation
Biomechanical Model
Synthetic Data
OpenSim
Deep Learning

http://resolver.tudelft.nl/uuid:4a118d7a-b714-4210-9ea0-8d77f83bec01

2024-06-01

Student theses

master thesis

© 2023 Zhi-Yi Lin