Change detection is essential to keep 3D city models up-to-date. LiDAR data with high accuracy are used to create 3D city models. However, updating LiDAR data at state or nation level often takes around a decade. Very high resolution (VHR) stereo images, with often yearly updating rate and dense 3D information, provide an option for validating and updating LiDAR data. However, the 3D information in both data sources has quality problems. LiDAR point clouds are sparse and irregularly spaced, and have mixed returns near building edges, while 3D information extracted from stereo images are affected by shadow and low texture. This research proposes LiDAR-guided dense matching to address these problems explicitly for detecting accurate building changes. Data sparsity and irregular spacing is addressed by densifying LiDAR points in a form of a digital surface model (DSM). Instead of applying interpolation with associated edge problems due to mixed returns, three candidate DSMs are created by linking each DSM pixel to up to three planes as identified in segmented and triangulated LiDAR data. The candidate DSMs limit the disparity search space for dense matching, addressing low texture and shadow problems in images. Through edge-aware dense matching, the detailed building edge information in stereo pairs determine the optimal heights to address LiDAR edge problem. Changes are detected where corresponding pixels from dense matching have large color differences. Due to homogeneous surroundings and shadows, only partial changes are initially detected. A second hierarchical dense matching step is employed to complete changes and update 3D information by propagating initial partial changes iteratively. The proposed method is applied on data from two cities, Amersfoort and Assen, the Netherlands, with around 1200 existing buildings. In both areas, the method successfully verifies unchanged buildings while detecting minimum changes of 2×2×2m³. New and removed building detection in Amersfoort both have a F1 score of over 0.8, both in pixel and object evaluation, while F1 scores in Assen are over 0.9 for both categories. The experiments also show that the proposed method outperforms two well-known change detection methods in terms of verifying unchanged buildings and detecting small changes simultaneously.

Up-to-date 3D building models are important for many applications. Airborne very high resolution (VHR) images often acquired annually give an opportunity to create an up-to-date 3D model. Building segmentation is often the first and utmost step. Convolutional neural networks (CNNs) draw lots of attention in interpreting VHR images as they can learn very effective features for very complex scenes. This paper employs Mask R-CNN to address two problems in building segmentation: detecting different scales of building and segmenting buildings to have accurately segmented edges. Mask R-CNN starts from feature pyramid network (FPN) to create different scales of semantically rich features. FPN is integrated with region proposal network (RPN) to generate objects with various scales with the corresponding optimal scale of features. The features with high and low levels of information are further used for better object classification of small objects and for mask prediction of edges. The method is tested on ISPRS benchmark dataset by comparing results with the fully convolutional networks (FCN), which merge high and low level features by a skip-layer to create a single feature for semantic segmentation. The results show that Mask R-CNN outperforms FCN with around 15% in detecting objects, especially in detecting small objects. Moreover, Mask R-CNN has much better results in edge region than FCN. The results also show that choosing the range of anchor scales in Mask R-CNN is a critical factor in segmenting different scale of objects. This paper provides an insight into how a good anchor scale for different dataset should be chosen.

Representation of scenes on the Earth surface by using voxels is gaining attention because of its suitability for integrating heterogeneous data sources in simulations and quantitative models. Computation of shadows in such models is needed, for example, to obtain crop suitability of agricultural fields in the presence of trees and buildings, or to analyze urban heat island causes and effects. We present an efficient algorithm to compute which of the voxels in a dataset receive direct sunlight, given the solar azimuth and elevation angles. The algorithm can work with multiple (sparse and dense) voxel storage strategies.

Various kinds of urban applications require true orthophotos. True orthophoto generation requires a DSM (Digital Surface Model) to project the photo orthogonally and minimize geometric distortion due to topographic variance. DSMs are often generated from airborne laser scan data. In urban scenes, DSM data may fail to deliver sharp and straight building roof edges. This will affect the quality of the resulting orthophotos. Therefore, it is necessary to incorporate good quality building outlines as breaklines during DSM interpolation. This study proposes a data-driven approach to construct building roof outlines from LiDAR point clouds by a workflow consisting of the following steps: given roof segments, roof boundary points are extracted using a concave hull algorithm. Straight edges may be difficult to find in complex roof configurations. Therefore, two ingredients are combined. First, RanSAC corner point preselection, and second, DBSCAN-based clustering of edge points. The method is demonstrated on an area of ±1.2 km² containing 42 buildings of different characteristics. A quality assessment shows that the proposed method is able to deliver 92% of building lines with acceptable geometric accuracy in comparison to a building line in the base map.

Change detection is an essential step to locate the area where an old model should be updated. With high density and accuracy, LiDAR data is often used to create a 3D city model. However, updating LiDAR data at state or nation level often takes years. Very high resolution (VHR) images with high updating rate is therefore an option for change detection. This paper provides a novel and efficient approach to derive pixel-based building change detection between past LiDAR and new VHR images. The proposed approach aims notably at reducing false alarms of changes near edges. For this purpose, LiDAR data is used to supervise the process of finding stereo pairs and derive the changes directly. This paper proposes to derive three possible heights (so three DSMs) by exploiting planar segments from LiDAR data. Near edges, the up to three possible heights are transformed into discrete disparities. A optimal disparity is selected from a reasonable and computational efficient range centered on them. If the optimal disparity is selected, but still the stereo pair found is wrong, a change has been found. A Markov random field (MRF) with built-in edge awareness from images is designed to find optimal disparity. By segmenting the pixels into plane and edge segments, the global optimization problem is split into many local ones which makes the optimization very efficient. Using an optimization and a consecutive occlusion consistency check, the changes are derived from stereo pairs having high color difference. The algorithm is tested to find changes in an urban areas in the city of Amersfoort, the Netherlands. The two different test cases show that the algorithm is indeed efficient. The optimized disparity images have sharp edges along those of images and false alarms of changes near or on edges and occlusions are largely reduced.

Semantic segmentation, especially for buildings, from the very high resolution (VHR) airborne images is an important task in urban mapping applications. Nowadays, the deep learning has significantly improved and applied in computer vision applications. Fully Convolutional Networks (FCN) is one of the tops voted method due to their good performance and high computational efficiency. However, the state-of-art results of deep nets depend on the training on large-scale benchmark datasets. Unfortunately, the benchmarks of VHR images are limited and have less generalization capability to another area of interest. As existing high precision base maps are easily available and objects are not changed dramatically in an urban area, the map information can be used to label images for training samples. Apart from object changes between maps and images due to time differences, the maps often cannot perfectly match with images. In this study, the main mislabeling sources are considered and addressed by utilizing stereo images, such as relief displacement, different representation between the base map and the image, and occlusion areas in the image. These free training samples are then fed to a pre-trained FCN. To find the better result, we applied fine-tuning with different learning rates and freezing different layers. We further improved the results by introducing atrous convolution. By using free training samples, we achieve a promising building classification with 85.6% overall accuracy and 83.77% F1 score, while the result from ISPRS benchmark by using manual labels has 92.02% overall accuracy and 84.06% F1 score, due to the building complexities in our study area.