Semantic segmentation of roof superstructures

Master thesis (2022)

Authors

I.M.M. Apra Architecture and the Built Environment

Contributors

G.A.K. Arroyo Ohori Urban Data Science - Architecture and the Built Environment (supervisor 1)
G. Agugiaro Urban Data Science - Architecture and the Built Environment (supervisor 2)
Sebastian Krapf Technische Universität München (coach)
Bruno Willenborg Technische Universität München (coach)
S. Du Urban Data Science - Architecture and the Built Environment (coach)
Faculty
Architecture and the Built Environment
Copyright: © 2022 Irène Apra
Deep Learning CNN LiDAR 3D city models Semantic segmentation Roof superstructures Fusion network
More Info
expand_more

Published Date

28-06-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Architecture and the Built Environment

Abstract

Automated reconstruction of detailed semantic 3D city models is challenging due to the need for high-resolution (HR) and large-scale input datasets, the ambiguous definition of the ensuing model, the intricacy of the processing pipeline, and its costs. Furthermore, existing methods mainly focus on geometry rather than semantics. Detailed semantic models may include roof installations whose size and function vary: dormers, windows, chimneys, etc. All elements visible on the roof from an aerial view are called ‘‘superstructures”. Deep Learning techniques can facilitate their modelization. This work inscribes itself in a project developed at the Technical University of Munich. The existing pipeline employs a convolutional neural network (CNN) on aerial images segmenting roof superstructures. These results can then be vectorized, extruded in 3D with their semantic description, and added to a simple 3D model.

This thesis demonstrates that building height data fused to a CNN on RGB aerial images improves the semantic segmentation of roof superstructures for classes with relief. Fusion of absolute and relative height data with different interpolation methods applied to LiDAR point cloud data is achieved through a fusion network from the state-of-the-art (FuseNet). First, experiments prove that detection accuracies increase by 11% on average for dormers and 12% for chimneys compared to U-Net output on the same dataset. Best performance is reached with the fusion of absolute height (rather than normalized) and IDW or NN interpolation technique (rather than none). However, although superstructure types are better recognized, their boundaries are fuzzier due to data input mismatches, and more background pixels are classified. Secondly, the predictions and modelization of both a Bavarian and Dutch test set prove the technique scalability. However, a training set annotated for Bavaria and applied to a test set in the Netherlands yields inaccurate results due to local architectural typologies and different input data characteristics.