Print Email Facebook Twitter Gaussians as Supervision for Joint Physical Parameter Estimation and Appearance Reconstruction of Elastic Objects Title Gaussians as Supervision for Joint Physical Parameter Estimation and Appearance Reconstruction of Elastic Objects Author Makridis, Pavlos (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Kellnhofer, P. (mentor) Uzolas, L. (mentor) Eisemann, E. (graduation committee) Raman, C.A. (graduation committee) Degree granting institution Delft University of Technology Programme Computer Science Date 2024-06-24 Abstract Recovering the appearance and physical parameters of elastic objects from multi-view video is essential for many applications that require simulation of the real world. Past methods for this task have provided accurate results in recovering physical properties; however, their re- liance on Neural Radiance Fields (NeRFs) for novel view synthesis means they trade off visual quality for rendering speed. To address this issue, we present a novel framework for the joint appearance reconstruction and physical parameter estimation of elastic objects relying on 3D Gaussian splatting. Our key insight is that dynamic 3D Gaussian kernels extracted from multi-view video can be used to reconstruct the object’s geometry and supervise elastic parame- ter fitting through a differentiable physics engine. Novel views and object behaviours can then be constructed by forward simulating the extracted mesh and using it to drive the Gaussian kernels. We demon- strate that our method is competitive with the state of the art on phys- ical parameter estimation while being better at reconstructing object appearance. Additionally, our method can simulate novel views and object interactions at near real-time rates that outperform past ap- proaches. Subject 3D Gaussian splattingNovel view synthesisphysical simulation To reference this document use: http://resolver.tudelft.nl/uuid:2e4df797-34ee-4c7c-aad7-694303ce5745 Embargo date 2024-09-20 Part of collection Student theses Document type master thesis Rights © 2024 Pavlos Makridis Files file embargo until 2024-09-20