Detecting building element/material through ground-based thermal imagery using Deep Neural Networks approach

Abstract

Building material identification plays an important role in applications such as sustainable building assessment, facade inspection, energy efficiency analysis, and urban environmental studies. Traditional material identification approaches mainly rely on optical imagery or geometric information, which are often limited in capturing the thermophysical characteristics of facade materials. Thermal imagery provides additional information related to heat transfer behavior, emissivity, and thermal inertia, offering an alternative perspective for building material analysis.

This thesis investigates the use of spatio-temporal thermal imagery combined with a Convolutional Long Short-Term Memory (Convolutional Long Short-Term Memory (ConvLSTM))- based deep learning framework for pixel-level building material classification. Groundbased thermal image sequences of building facades were collected under different environmental and temporal conditions using a thermal camera. Building materials including glass, concrete, and brick were manually annotated through Red Green Blue (RGB) imagery and Segment Anything Model (SAM)-assisted segmentation. Image registration techniques were subsequently applied to align the RGB and thermal image sequences, enabling the transfer of material labels into the thermal domain.

A ConvLSTM-based semantic segmentation model was developed to capture both spatial thermal distributions and temporal thermal dynamics from sequential thermal imagery. Different dataset splitting strategies and temporal datasets were evaluated to investigate the influence of spatial generalization and temporal variation on classification performance. The experimental results demonstrate that spatio-temporal thermal sequences provide meaningful discriminative information for building material classification. Under the random sequence-level split strategy, the proposed framework achieved a mean F1-score of approximately 0.686 and a mean Intersection over Union (IoU) of approximately 0.524 across the evaluated material classes.

The results indicate that temporal thermal behavior significantly contributes to distinguishing facade materials. Brick surfaces exhibited relatively stable and spatially uniform thermal characteristics, while glass surfaces demonstrated stronger temporal variation and environmental sensitivity. Concrete exhibited intermediate thermal behavior, making it comparatively more difficult to classify. The findings suggest that incorporating temporal thermal information improves segmentation performance compared with single-frame thermal analysis.