Human Interaction Pose Prior via Conditional Normalizing Flows
D. Maksymchuk (TU Delft - Electrical Engineering, Mathematics and Computer Science)
P. Kellnhofer – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
L. Cavalcante Siebert – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Close physical interactions such as hugs, handshakes, and collaborative tasks are commonplace in everyday life, yet 3D reconstruction methods still struggle to capture them accurately. Most approaches treat each person independently, producing reconstructions where bodies unrealistically intersect, drift apart, or collapse to averaged poses, particularly under occlusion. Furthermore, without an explicit probability model over interactions, there is no principled way to assess whether a reconstruction is plausible in the first place.
We address this by learning an explicit probabilistic prior over close human-human interactions. Using a conditional normalizing flow, we model how one person's pose and position relate to the other's, yielding a tractable likelihood that can be evaluated for any pair of poses at low computational cost, unlike diffusion models, which require iterative sampling chains, or VAEs, whose likelihoods are lower bounds rather than exact densities. Our model serves three purposes: it generates diverse and realistic reaction poses given an observed person, scores the plausibility of arbitrary interaction pairs, and acts as a differentiable regularizer in image-to-mesh optimization to improve two-person 3D reconstruction from a single RGB image. Experiments demonstrate strong performance in generating diverse reaction poses and in discriminating plausible from implausible interaction pairs. Used as an optimization prior, our method improves two-person image-to-mesh reconstruction, with the largest gains in the relative placement of the two bodies.