PHYSHADE-Net: Leveraging Geometric-Priors in Physics-Guided Neural Networks for Building Shadow Segmentation and Height Estimation
L.C. Huizer (TU Delft - Architecture and the Built Environment)
A. Rafiee – Mentor (TU Delft - Digital Technologies)
E Verbree – Mentor (TU Delft - Digital Technologies)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Building heights are important information for a variety of subjects, such as wind analysis, energy demand simulations and solar potential assessment, yet large-scale LiDAR scanning is costly. This thesis introduces PHYSHADE: a set of physics-guided U-Net-based models. It employs shadow projections derived from building footprints and solar geometry into an aerial image shadow segmentation pipeline, for the purposes of building shadow extraction and consequently the large-scale estimation of building heights. Thirty-five aerial images in the Netherlands were manually annotated for buildings and their associated buildings. By employing transfer learning, based on a general purpose shadow-segmentation model, a total of 130 models were trained, which can be categorized into three different implementations of PHYSHADE. Through these different configurations, the performance impact of the various methods of addition of pseudo-shadows to the models was ablated. Afterwards, the best-performing PHYSHADE configurations were used with a raycasting algorithm to convert shadow lengths and solar altitudes back into building heights. The inclusion of pseudo-shadows lifted the mean Dice scores from 0.53 to 0.85, with an average gain of 0.32 and statistical significance across different folds. Physics-guided loss, based on the pseudo-shadows, was not found to be significantly different in most cases, whilst hurting model performance in some cases compared to the pseudo-shadow enabled models. On six out-of-fold test tiles the best PHYSHADE variants retained Dice scores of 0.72 – 0.95, although recall declined in one winter scene. Finally, height estimation on these tiles using the inference from the best PHYSHADE variants resulted in mean RMSE of ≅ 1.9m and MAE of ≅ 1.5m. While its application needs to be tested in broader contexts, PHYSHADE offers a viable low-cost complement to LiDAR for building height estimation.