CSDN

Cross-Modal Shape-Transfer Dual-Refinement Network for Point Cloud Completion

Journal Article (2023)
Author(s)

Zhe Zhu (Nanjing University of Aeronautics and Astronautics)

L. Nan (TU Delft - Urban Data Science)

Haoran Xie (Lingnan University, Hong Kong)

Honghua Chen (Nanjing University of Aeronautics and Astronautics)

Jun Wang (Nanjing University of Aeronautics and Astronautics)

Mingqiang Wei (Nanjing University of Aeronautics and Astronautics)

Jing Qin (The Hong Kong Polytechnic University)

Research Group
Urban Data Science
DOI related publication
https://doi.org/10.1109/TVCG.2023.3236061
More Info
expand_more
Publication Year
2023
Language
English
Research Group
Urban Data Science
Bibliographical Note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.@en
Issue number
7
Volume number
30 (2024)
Pages (from-to)
3545-3563
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

How will you repair a physical object with some missings? You may imagine its original shape from previously captured images, recover its overall (global) but coarse shape first, and then refine its local details. We are motivated to imitate the physical repair procedure to address point cloud completion. To this end, we propose a cross-modal shape-transfer dual-refinement network (termed CSDN), a coarse-to-fine paradigm with images of full-cycle participation, for quality point cloud completion. CSDN mainly consists of “shape fusion” and “dual-refinement” modules to tackle the cross-modal challenge. The first module transfers the intrinsic shape characteristics from single images to guide the geometry generation of the missing regions of point clouds, in which we propose IPAdaIN to embed the global features of both the image and the partial point cloud into completion. The second module refines the coarse output by adjusting the positions of the generated points, where the local refinement unit exploits the geometric relation between the novel and the input points by graph convolution, and the global constraint unit utilizes the input image to fine-tune the generated offset. Different from most existing approaches, CSDN not only explores the complementary information from images but also effectively exploits cross-modal data in the whole coarse-to-fine completion procedure. Experimental results indicate that CSDN performs favorably against twelve competitors on the cross-modal benchmark.

Files

CSDN_Cross-Modal_Shape-Transfe... (pdf)
(pdf | 14.8 Mb)
- Embargo expired in 11-07-2023
License info not available