Towards onboard thermal hotspots segmentation with raw multispectral satellite imagery

Abstract

The rapid spread and destructive nature of wildfires and volcanic activity have intensified the need for low latency detection systems. The growing intensity and frequency of globally distributed thermal hotspots have driven the development of satellite-based detection solutions. Conventional approaches rely on ground-based processing, which limits low latency capabilities due to revisit times over ground stations and data handling requirements. This work proposes the first onboard payload processing pipeline for segmentation of thermal hotspots in raw multispectral satellite imagery. The pipeline leverages the Near InfraRed (NIR) and Short-Wave InfraRed (SWIR) spectral bands, and the combination of onboard Artificial Intelligence (AI) and raw imagery significantly reduces the delay between image acquisition and event detection. Furthermore, we present Segmentation of Thermal Hotspots in Raw Sentinel-2 data (SegTHRawS), the first publicly available dataset for thermal hotspot segmentation in raw multispectral satellite imagery. The segmentation model employed is a Fully Convolutional Network (FCN) derived from U-Net, named ResUnet-S2, designed for fast on-device inference. This model achieved an Intersection over Union (IoU) of 0.988 and an F-1 score of 0.986 on SegTHRawS, with its detection and generalization capabilities validated using an external thermal hotspot segmentation dataset. The proposed pipeline was verified on CubeSat-compatible hardware, achieving an end-to-end execution, from image acquisition to event detection, in 1.45 s, faster than the image acquisition process, and consuming a peak power of 4.05 W. These results demonstrate the potential of onboard processing solutions for minimizing the detection latency of current approaches, particularly for thermal hotspot segmentation, using edge computing satellite hardware.