DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks

DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks

Ibrahimli, N. (TU Delft Urban Data Science)
Ledoux, H. (TU Delft Urban Data Science)
Kooij, J.F.P. (TU Delft Intelligent Vehicles)
Nan, L. (TU Delft Urban Data Science)

2023

We propose an enhancement module called depth discontinuity learning (DDL) for learning-based multi-view stereo (MVS) methods. Traditional methods are known for their accuracy but struggle with completeness. While recent learning-based methods have improved completeness at the cost of accuracy, our DDL approach aims to improve accuracy while retaining completeness in the reconstruction process. To achieve this, we introduce the joint estimation of depth and boundary maps, where the boundary maps are explicitly utilized for further refinement of the depth maps. We validate our idea by integrating it into an existing learning-based MVS pipeline where the reconstruction depends on high-quality depth map estimation. Extensive experiments on various datasets, namely DTU, ETH3D, “Tanks and Temples”, and BlendedMVS, show that our method improves reconstruction quality compared to our baseline, Patchmatchnet. Our ablation study demonstrates that incorporating the proposed DDL significantly reduces the depth map error, for instance, by more than 30% on the DTU dataset, and leads to improved depth map quality in both smooth and boundary regions. Additionally, our qualitative analysis has shown that the reconstructed point cloud exhibits enhanced quality without any significant compromise on completeness. Finally, the experiments reveal that our proposed model and strategies exhibit strong generalization capabilities across the various datasets.

multi-view stereo
3D reconstruction
depth map refinement
depth boundary estimation

http://resolver.tudelft.nl/uuid:e1700152-fb07-455c-bbd5-17ea02a37cf1

https://doi.org/10.3390/rs15122970

2072-4292

Remote Sensing, 15 (12)

Institutional Repository

journal article

© 2023 N. Ibrahimli, H. Ledoux, J.F.P. Kooij, L. Nan