Top-I2P
Explore Open-Domain Image-to-Point Cloud Registration Using Topology Relationship
Pei An (Huazhong University of Science and Technology)
Jiaqi Yang (Northwestern Polytechnical University)
Muyao Peng (Huazhong University of Science and Technology)
You Yang (Huazhong University of Science and Technology)
Qiong Liu (Huazhong University of Science and Technology)
Jie Ma (Huazhong University of Science and Technology)
Liangliang Nan (TU Delft - Architecture and the Built Environment)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Image-to-point cloud (I2P) registration is a fundamental task in computer vision, which aims to align pixels in 2D images with corresponding points in 3D point clouds. While deep learning based methods dominate this field, they often fail to generalize to the open domain. In this paper, we address open-domain I2P registration from the topology relationship perspective. Firstly, we find that topology relationship reflect sparse connections between pixels and points, which shows the significant potential in enhancing cross-modality feature interaction in the open domain. Building on this insight, we develop an I2P registration framework using topology relationship. After that, to construct and leverage the topology relationship between the heterogeneous 2D and 3D spaces, we design a registration network, Top-I2P, with correction-based topology reasoning and fast topology feature interaction modules. Extensive experiments on 7-Scenes, RGBD-V2, ScanNet, and self-collected I2P datasets demonstrate that Top-I2P achieves superior registration performance in open-domain scenarios.