3D Scene Compression for Autonomous Driving using Neural Radiance Fields

3D Scene Compression for Autonomous Driving using Neural Radiance Fields

Enting, Marnix (TU Delft Mechanical Engineering)

Caesar, Holger (mentor)
Weinmann, M. (graduation committee)

Delft University of Technology

Mechanical Engineering | Vehicle Engineering | Cognitive Robotics

2024-05-03

Neural Radiance Fields (NeRFs) have showcased remarkable effectiveness in capturing complex 3D scenes and synthesizing novel viewpoints. By inherently capturing the entire scene in a compact representation, they offer a promising avenue for applications such as simulators, where efficient storage of real-world data, fast rendering and dynamic generation of new content are crucial. However, the potential for compression in NeRFs has been largely neglected in the existing literature. Moreover, the practical deployment of NeRFs in real-world scenarios, including simulators, faces significant obstacles such as constraints in training time, rendering speed, and scalability to large scenes. While recent advancements have tackled some of these hurdles individually, none have offered a comprehensive solution. In this paper, we introduce a new NeRF architecture based on a textured polygon-based method and augment this architecture by integrating encodings to expedite training. Additionally, we introduce learned pose refinement and an appearance embedding to enhance scalability to larger scenes. Through experimentation on the nuScenes dataset, we demonstrate that our method achieves competitive reconstruction performance with existing techniques while surpassing them in rendering speed. Furthermore, in terms of compression, our findings indicate that our method achieves competitive compression rates comparable to image-based compression techniques, while also enabling novel-view synthesis. This underscores its potential utility in applications like simulators.

Neural Radiance Fields
3D Scene Compression
Autonomous Driving
Deep Learning
Driving Simulator

http://resolver.tudelft.nl/uuid:c2044642-4521-4381-8b91-3cfb50cd27b0

Student theses

master thesis

© 2024 Marnix Enting