Push-the-Boundary

Push-the-Boundary: Boundary-Aware Feature Propagation for Semantic Segmentation of 3D Point Clouds

Du, S. (TU Delft Urban Data Science)
Ibrahimli, N. (TU Delft Urban Data Science)
Stoter, J.E. (TU Delft Urban Data Science)
Kooij, J.F.P. (TU Delft Intelligent Vehicles)
Nan, L. (TU Delft Urban Data Science)

Ceballos, Cristina (editor)

2022

Feedforward fully convolutional neural networks currently dominate in semantic segmentation of 3D point clouds. Despite their great success, they suffer from the loss of local information at low-level layers, posing significant challenges to accurate scene segmentation and precise object boundary delineation. Prior works either address this issue by post-processing or jointly learn object boundaries to implicitly improve feature encoding of the networks. These approaches often require additional modules which are difficult to integrate into the original architecture. To improve the segmentation near object boundaries, we propose a boundary-aware feature propagation mechanism. This mechanism is achieved by exploiting a multitask learning framework that aims to explicitly guide the boundaries to their original locations. With one shared encoder, our network outputs (i) boundary localization, (ii) prediction of directions pointing to the object's interior, and (iii) semantic segmentation, in three parallel streams. The predicted boundaries and directions are fused to propagate the learned features to refine the segmentation. We conduct extensive experiments on the S3DIS and SensatUrban datasets against various baseline methods, demonstrating that our proposed approach yields consistent improvements by reducing boundary errors. Our code is available at https://github.com/shenglandu/PushBoundary.

point cloud compression
location awareness
three-dimensional displays
semantic segmentation
semantics
self-supervised learning
encoding

http://resolver.tudelft.nl/uuid:18f3be6d-76da-4bc2-9740-a1f3a0105274

https://doi.org/10.1109/3DV57658.2022.00025

IEEE, Prague

2023-09-22

978-1-6654-5670-8

Proceedings - 2022 International Conference on 3D Vision, 3DV 2022

2022 International Conference on 3D Vision (3DV), 2022-09-12 → 2022-09-15, Czech Technical University, Prague, Czech Republic

This work was supported by the 3D Urban Understanding Lab funded by the TU Delft AI Initiative. Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2022 S. Du, N. Ibrahimli, J.E. Stoter, J.F.P. Kooij, L. Nan