Semantic 3D segmentation of 3D Gaussian Splats
Assessing existing point cloud segmentation techniques on semantic segmentation of synthetic 3D Gaussian Splats scenes
K.K. Jurski (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Xucong Zhang – Mentor (TU Delft - Pattern Recognition and Bioinformatics)
M. Weinmann – Graduation committee member (TU Delft - Computer Graphics and Visualisation)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
3D Gaussian Splatting (3DGS) is a promising 3D reconstruction and novel-view synthesis technique. However, the field of semantic 3D segmentation of 3D Gaussian Splats scenes remains largely unexplored. This paper discusses the challenges of performing 3D segmentation directly on 3D Gaussian Splats, introduces a new dataset facilitating evaluation of 3DGS semantic segmentation and proposes use of PointNet++, initially developed for point cloud segmentation, as a 3DGS segmentation model. As the results show, PointNet++ is also capable of performing 3DGS segmentation with performance close to the performance achieved in point cloud segmentation tasks. When taking into account only the positions, 3D Gaussian splats appear to be more difficult for PointNet++ to process than point clouds sampled from mesh faces, possibly due to their irregularity. However, as shown in the paper, inclusion of size, rotation and opacity of each splat allows PointNet++ to achieve nearly 87% of accuracy, outperforming PointNet++ on point clouds sampled from meshes.