Autolabeling & Semantic Segmentation with 4D Radar Tensors
B. Sun (TU Delft - Electrical Engineering, Mathematics and Computer Science)
F. Fioranelli – Mentor (TU Delft - Microwave Sensing, Signals & Systems)
J. F. P. Kooij – Graduation committee member (TU Delft - Intelligent Vehicles)
Ignacio Roldan Montero – Mentor (TU Delft - Microwave Sensing, Signals & Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
4D millimeter-wave radar is increasingly important in advanced driver-assistance systems due to its ability to capture Doppler/velocity information and robustness in low-light or adverse weather conditions. Unlike traditional 3D radar, 4D radar provides elevation information, enhancing 3D spatial perception. Most of the perception tasks using 4D radar tensors focus on target classification using bounding boxes. In contrast, although semantic segmentation is typically used to process images and LiDAR point clouds, it has not been well explored for 4D radar tensors.
This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.