An Empirical Investigation on Variational Autoencoder-Based Dynamic Modeling of Deformable Objects from RGB Data
Tomás Coleman (TU Delft - Learning & Autonomous Control)
R Babuška (Czech Technical University, TU Delft - Learning & Autonomous Control)
J. Kober (TU Delft - Learning & Autonomous Control)
Cosimo Della Santina (TU Delft - Learning & Autonomous Control, Deutsches Zentrum für Luft- und Raumfahrt (DLR))
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Formulating the dynamics of continuously deformable objects and other mechanical systems analytically from first principles is an exceedingly challenging task, often impractical in real-world scenarios. What makes this challenge even harder to solve is that, usually, the object has not been observed previously, and the only information that we can get from it is a stream of RGB camera data. In this study, we explore the use of deep learning techniques to solve this nonlinear identification problem. We specifically focus on extracting dynamic models of simple deformable objects from the high-dimensional sensor input coming from an RGB camera. We investigate a two-stage approach to achieve this goal. First, we train a variational autoencoder to extract an extremely low-dimensional representation of the object configuration. Then, we learn a dynamic model that predicts the evolution of these latent space variables. The proposed architecture can accurately predict the object's state up to one second into the future.