Automated reconstruction of detailed semantic 3D city models is challenging due to the need for high-resolution (HR) and large-scale input datasets, the ambiguous definition of the ensuing model, the intricacy of the processing pipeline, and its costs. Furthermore, existing methods mainly focus on geometry rather than semantics. Detailed semantic models may include roof installations whose size and function vary: dormers, windows, chimneys, etc. All elements visible on the roof from an aerial view are called ‘‘superstructures”. Deep Learning techniques can facilitate their modelization. This work inscribes itself in a project developed at the Technical University of Munich. The existing pipeline employs a convolutional neural network (CNN) on aerial images segmenting roof superstructures. These results can then be vectorized, extruded in 3D with their semantic description, and added to a simple 3D model. This thesis demonstrates that building height data fused to a CNN on RGB aerial images improves the semantic segmentation of roof superstructures for classes with relief. Fusion of absolute and relative height data with different interpolation methods applied to LiDAR point cloud data is achieved through a fusion network from the state-of-the-art (FuseNet). First, experiments prove that detection accuracies increase by 11% on average for dormers and 12% for chimneys compared to U-Net output on the same dataset. Best performance is reached with the fusion of absolute height (rather than normalized) and IDW or NN interpolation technique (rather than none). However, although superstructure types are better recognized, their boundaries are fuzzier due to data input mismatches, and more background pixels are classified. Secondly, the predictions and modelization of both a Bavarian and Dutch test set prove the technique scalability. However, a training set annotated for Bavaria and applied to a test set in the Netherlands yields inaccurate results due to local architectural typologies and different input data characteristics.

Led in cooperation with the company Brink, who provides management and consultation services for construction and real estate sectors, this Synthesis Project aims at automatically deriving meaningful information about buildings. More precisely, the focus is to automatically detect roof obstacles - such as dormers, chimneys, and solar panels - to be able to determine the available roof surface for new solar panel installation, and therefore to perform more accurate solar potential analysis. For this purpose, three different methods are developed and implemented to increase the results’ accuracy, which are geometry-based, unsupervised, and supervised classification. While AHN3 point cloud and 3D BAG Level of Detail (LoD) 2.2 building models are used for the geometry-based classification, the input data of the unsupervised image classification consists of aerial images and BAG footprints. Finally, supervised image classification method makes use of the aerial images as well as the BAG footprints and a dataset of manually labelled solar panel polygons. The results show that the accuracy of individual methods is not sufficient; therefore, the outputs of all three methods are merged together into one pipeline, with the aim of obtaining one final end product. The latter is the 3D BAG LoD2.2 building model in CityJSON format, enhanced with three new attributes per building: the obstacle area on the roof, the available area for installing solar panels, and a Boolean value showing whether the building has existing solar panels or not. Additionally, an enhanced point cloud for future use is generated, with a new attribute per point indicating its distance to the 3D model and therefore its potential for being an obstacle or not. The assessment of the results with the ground truth illustrates that the algorithm gives promising results; however, the scope of the project can be broadened, and improvements can be made to increase the accuracy as well as the efficiency.