F. Fioranelli
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
45 records found
1
Automotive radar is robust to adverse weather and illumination conditions, but its sparse, noisy, and not always spatially accurate measurements make dense occupancy estimation challenging. This thesis investigates light detection and ranging (LiDAR)-supervised radar-based first-return occupancy grid mapping on a polar range–azimuth grid. Radar is used as the sole input modality during inference, while synchronized LiDAR point clouds are processed into radar-aligned supervision representing visible free space, the nearest occupied boundary, and the unknown region behind it.
The proposed framework formulates occupancy estimation as an azimuth-wise first-return prediction problem rather than independent cell-wise classification. The complete range–Doppler–azimuth (RDA) radar cube is processed using a Range–Azimuth-Preserving Doppler Spectrum Encoder with parallel multi-scale Doppler-only branches, followed by a U-Net 3+ prediction backbone. The predicted first-return distribution is rendered into a visibility-consistent three-state occupancy grid. Training combines a Lovasz-based occupancy objective with a smoothed cumulative distribution function (CDF)-L1 boundary loss. Radar-aware azimuth weighting and causal temporal feature fusion are further investigated as controlled extensions.
Experiments are conducted on the RaDelft dataset using a scene-level train, validation, and test split. The proposed single-frame baseline achieves a free-space intersection over union (IoU) of 0.79, an unfree-space IoU of 0.82, a mean IoU (mIoU) of 0.80, and a first-return mean absolute error (MAE) of 5.38 m. It outperforms an odometry-compensated multi-frame inverse sensor model and a RaDelft-adapted PolarNet reference taken from the literature. Additional experiments show that retaining the complete Doppler spectrum is more effective than fixed mean or maximum aggregation. The proposed Doppler encoder provides accuracy comparable to a reference encoder from the literature, but at lower computational cost.
Overall, the results demonstrate that structured first-return prediction from full Doppler-resolved radar measurements provides an effective approach to LiDAR-supervised polar occupancy mapping, with the single-frame full-RDA model offering the most favourable accuracy–efficiency balance among the evaluated full-RDA configurations. ...
The proposed framework formulates occupancy estimation as an azimuth-wise first-return prediction problem rather than independent cell-wise classification. The complete range–Doppler–azimuth (RDA) radar cube is processed using a Range–Azimuth-Preserving Doppler Spectrum Encoder with parallel multi-scale Doppler-only branches, followed by a U-Net 3+ prediction backbone. The predicted first-return distribution is rendered into a visibility-consistent three-state occupancy grid. Training combines a Lovasz-based occupancy objective with a smoothed cumulative distribution function (CDF)-L1 boundary loss. Radar-aware azimuth weighting and causal temporal feature fusion are further investigated as controlled extensions.
Experiments are conducted on the RaDelft dataset using a scene-level train, validation, and test split. The proposed single-frame baseline achieves a free-space intersection over union (IoU) of 0.79, an unfree-space IoU of 0.82, a mean IoU (mIoU) of 0.80, and a first-return mean absolute error (MAE) of 5.38 m. It outperforms an odometry-compensated multi-frame inverse sensor model and a RaDelft-adapted PolarNet reference taken from the literature. Additional experiments show that retaining the complete Doppler spectrum is more effective than fixed mean or maximum aggregation. The proposed Doppler encoder provides accuracy comparable to a reference encoder from the literature, but at lower computational cost.
Overall, the results demonstrate that structured first-return prediction from full Doppler-resolved radar measurements provides an effective approach to LiDAR-supervised polar occupancy mapping, with the single-frame full-RDA model offering the most favourable accuracy–efficiency balance among the evaluated full-RDA configurations. ...
Automotive radar is robust to adverse weather and illumination conditions, but its sparse, noisy, and not always spatially accurate measurements make dense occupancy estimation challenging. This thesis investigates light detection and ranging (LiDAR)-supervised radar-based first-return occupancy grid mapping on a polar range–azimuth grid. Radar is used as the sole input modality during inference, while synchronized LiDAR point clouds are processed into radar-aligned supervision representing visible free space, the nearest occupied boundary, and the unknown region behind it.
The proposed framework formulates occupancy estimation as an azimuth-wise first-return prediction problem rather than independent cell-wise classification. The complete range–Doppler–azimuth (RDA) radar cube is processed using a Range–Azimuth-Preserving Doppler Spectrum Encoder with parallel multi-scale Doppler-only branches, followed by a U-Net 3+ prediction backbone. The predicted first-return distribution is rendered into a visibility-consistent three-state occupancy grid. Training combines a Lovasz-based occupancy objective with a smoothed cumulative distribution function (CDF)-L1 boundary loss. Radar-aware azimuth weighting and causal temporal feature fusion are further investigated as controlled extensions.
Experiments are conducted on the RaDelft dataset using a scene-level train, validation, and test split. The proposed single-frame baseline achieves a free-space intersection over union (IoU) of 0.79, an unfree-space IoU of 0.82, a mean IoU (mIoU) of 0.80, and a first-return mean absolute error (MAE) of 5.38 m. It outperforms an odometry-compensated multi-frame inverse sensor model and a RaDelft-adapted PolarNet reference taken from the literature. Additional experiments show that retaining the complete Doppler spectrum is more effective than fixed mean or maximum aggregation. The proposed Doppler encoder provides accuracy comparable to a reference encoder from the literature, but at lower computational cost.
Overall, the results demonstrate that structured first-return prediction from full Doppler-resolved radar measurements provides an effective approach to LiDAR-supervised polar occupancy mapping, with the single-frame full-RDA model offering the most favourable accuracy–efficiency balance among the evaluated full-RDA configurations.
The proposed framework formulates occupancy estimation as an azimuth-wise first-return prediction problem rather than independent cell-wise classification. The complete range–Doppler–azimuth (RDA) radar cube is processed using a Range–Azimuth-Preserving Doppler Spectrum Encoder with parallel multi-scale Doppler-only branches, followed by a U-Net 3+ prediction backbone. The predicted first-return distribution is rendered into a visibility-consistent three-state occupancy grid. Training combines a Lovasz-based occupancy objective with a smoothed cumulative distribution function (CDF)-L1 boundary loss. Radar-aware azimuth weighting and causal temporal feature fusion are further investigated as controlled extensions.
Experiments are conducted on the RaDelft dataset using a scene-level train, validation, and test split. The proposed single-frame baseline achieves a free-space intersection over union (IoU) of 0.79, an unfree-space IoU of 0.82, a mean IoU (mIoU) of 0.80, and a first-return mean absolute error (MAE) of 5.38 m. It outperforms an odometry-compensated multi-frame inverse sensor model and a RaDelft-adapted PolarNet reference taken from the literature. Additional experiments show that retaining the complete Doppler spectrum is more effective than fixed mean or maximum aggregation. The proposed Doppler encoder provides accuracy comparable to a reference encoder from the literature, but at lower computational cost.
Overall, the results demonstrate that structured first-return prediction from full Doppler-resolved radar measurements provides an effective approach to LiDAR-supervised polar occupancy mapping, with the single-frame full-RDA model offering the most favourable accuracy–efficiency balance among the evaluated full-RDA configurations.
Automotive radar is a key all-weather sensor for autonomous driving, providing range and radial-velocity measurements under adverse illumination and weather conditions. Target detection and subsequent point-cloud generation are therefore critical stages in automotive radar signal processing. However, conventional constant false alarm rate (CFAR) detectors rely on fixed local reference rules and statistical assumptions that are often violated by extended targets and non-homogeneous backgrounds in road scenes, resulting in sparse and spatially inaccurate point clouds. End-to-end deep-learning detectors can exploit richer radar context, but may require substantial computational resources and provide limited interpretability and post-training control of the detection threshold.
In this work, we propose a hybrid neural CFAR-like detector in which a three-level U-Net estimates a background-reliability map from radar data, and an explicit local thresholding stage uses this map to form cell-wise detection decisions. Evaluated experimentally on the \textit{RaDelft} dataset, the proposed detector achieved the lowest Chamfer Distance (CD) among the evaluated methods, reaching 1.36 m compared with 3.19 m for CA-CFAR, 3.29 m for OS-CFAR, and 1.60 m for a deep-learning baseline taken from the literature. Moreover, the proposed method used 0.87 million parameters and achieved 10.58 GFLOPs, which is 13 times less than the evaluated deep-learning baseline.
These results indicate that learning the reliability of local background samples can improve automotive radar point-cloud quality while retaining an explicit and adjustable detection procedure.
...
In this work, we propose a hybrid neural CFAR-like detector in which a three-level U-Net estimates a background-reliability map from radar data, and an explicit local thresholding stage uses this map to form cell-wise detection decisions. Evaluated experimentally on the \textit{RaDelft} dataset, the proposed detector achieved the lowest Chamfer Distance (CD) among the evaluated methods, reaching 1.36 m compared with 3.19 m for CA-CFAR, 3.29 m for OS-CFAR, and 1.60 m for a deep-learning baseline taken from the literature. Moreover, the proposed method used 0.87 million parameters and achieved 10.58 GFLOPs, which is 13 times less than the evaluated deep-learning baseline.
These results indicate that learning the reliability of local background samples can improve automotive radar point-cloud quality while retaining an explicit and adjustable detection procedure.
...
Automotive radar is a key all-weather sensor for autonomous driving, providing range and radial-velocity measurements under adverse illumination and weather conditions. Target detection and subsequent point-cloud generation are therefore critical stages in automotive radar signal processing. However, conventional constant false alarm rate (CFAR) detectors rely on fixed local reference rules and statistical assumptions that are often violated by extended targets and non-homogeneous backgrounds in road scenes, resulting in sparse and spatially inaccurate point clouds. End-to-end deep-learning detectors can exploit richer radar context, but may require substantial computational resources and provide limited interpretability and post-training control of the detection threshold.
In this work, we propose a hybrid neural CFAR-like detector in which a three-level U-Net estimates a background-reliability map from radar data, and an explicit local thresholding stage uses this map to form cell-wise detection decisions. Evaluated experimentally on the \textit{RaDelft} dataset, the proposed detector achieved the lowest Chamfer Distance (CD) among the evaluated methods, reaching 1.36 m compared with 3.19 m for CA-CFAR, 3.29 m for OS-CFAR, and 1.60 m for a deep-learning baseline taken from the literature. Moreover, the proposed method used 0.87 million parameters and achieved 10.58 GFLOPs, which is 13 times less than the evaluated deep-learning baseline.
These results indicate that learning the reliability of local background samples can improve automotive radar point-cloud quality while retaining an explicit and adjustable detection procedure.
In this work, we propose a hybrid neural CFAR-like detector in which a three-level U-Net estimates a background-reliability map from radar data, and an explicit local thresholding stage uses this map to form cell-wise detection decisions. Evaluated experimentally on the \textit{RaDelft} dataset, the proposed detector achieved the lowest Chamfer Distance (CD) among the evaluated methods, reaching 1.36 m compared with 3.19 m for CA-CFAR, 3.29 m for OS-CFAR, and 1.60 m for a deep-learning baseline taken from the literature. Moreover, the proposed method used 0.87 million parameters and achieved 10.58 GFLOPs, which is 13 times less than the evaluated deep-learning baseline.
These results indicate that learning the reliability of local background samples can improve automotive radar point-cloud quality while retaining an explicit and adjustable detection procedure.
Maritime radars are widely used for surveillance, search and rescue and detection of vessels. However, target detection, especially in maritime environments, is often limited by sea clutter, which is produced by the radar returns of the continuously changing sea surface. Sea clutter changes with environmental conditions and radar configurations and may occupy similar regions in the Range-Doppler (RD) map as a moving target. As a result, the target can be masked by the clutter, making detection very challenging and generating a large number of false alarms. Although several classical and learning-based clutter suppression techniques have been proposed, they are often evaluated using different datasets and performance criteria, making
direct comparison difficult. In this thesis, classical and machine learning approaches for sea clutter suppression are evaluated under a common framework. A classical detector is first applied directly to RD maps to provide a detection baseline. Then a Selective Singular Value Decomposition (SVD) is used as a clutter suppression stage before Cell-Averaging Constant False Alarm Rate (CA-CFAR) detection. Then, the benefit of temporal information is also investigated by integrating consecutive RD maps. In parallel, a U-Net neural network is trained to reconstruct target only
representations from the original RD maps, while measurements with clutter only are also included to limit false target generation. Multi frame integration was also examined in the U-Net by stacking consecutive RD maps as separate input channels. All approaches are evaluated on a TNO experimental dataset using the same detection and clutter suppression metrics including target detection fraction, frame-level clutter breakthrough, signal-to-clutter ratio improvement and clutter suppression factor. The results show that Selective Singular Value Decomposition and temporal integration can improve target detection compared with CFAR detection alone, but the clutter breakthrough remained relatively high. On the contrary, the U-Net managed to have a much better balance between target detection and clutter breakthrough, especially when incorporating multiple frames. The three frame U-Net achieved the highest target detection fraction of approximately 94.2% and the four frame U-Net achieved the best balance with a target detection fraction over 90% and average clutter breakthrough 0.04 clusters per RD map. Overall, the results demonstrate that learning based sea clutter suppression is a promising alternative to
conventional maritime radar processing. ...
direct comparison difficult. In this thesis, classical and machine learning approaches for sea clutter suppression are evaluated under a common framework. A classical detector is first applied directly to RD maps to provide a detection baseline. Then a Selective Singular Value Decomposition (SVD) is used as a clutter suppression stage before Cell-Averaging Constant False Alarm Rate (CA-CFAR) detection. Then, the benefit of temporal information is also investigated by integrating consecutive RD maps. In parallel, a U-Net neural network is trained to reconstruct target only
representations from the original RD maps, while measurements with clutter only are also included to limit false target generation. Multi frame integration was also examined in the U-Net by stacking consecutive RD maps as separate input channels. All approaches are evaluated on a TNO experimental dataset using the same detection and clutter suppression metrics including target detection fraction, frame-level clutter breakthrough, signal-to-clutter ratio improvement and clutter suppression factor. The results show that Selective Singular Value Decomposition and temporal integration can improve target detection compared with CFAR detection alone, but the clutter breakthrough remained relatively high. On the contrary, the U-Net managed to have a much better balance between target detection and clutter breakthrough, especially when incorporating multiple frames. The three frame U-Net achieved the highest target detection fraction of approximately 94.2% and the four frame U-Net achieved the best balance with a target detection fraction over 90% and average clutter breakthrough 0.04 clusters per RD map. Overall, the results demonstrate that learning based sea clutter suppression is a promising alternative to
conventional maritime radar processing. ...
Maritime radars are widely used for surveillance, search and rescue and detection of vessels. However, target detection, especially in maritime environments, is often limited by sea clutter, which is produced by the radar returns of the continuously changing sea surface. Sea clutter changes with environmental conditions and radar configurations and may occupy similar regions in the Range-Doppler (RD) map as a moving target. As a result, the target can be masked by the clutter, making detection very challenging and generating a large number of false alarms. Although several classical and learning-based clutter suppression techniques have been proposed, they are often evaluated using different datasets and performance criteria, making
direct comparison difficult. In this thesis, classical and machine learning approaches for sea clutter suppression are evaluated under a common framework. A classical detector is first applied directly to RD maps to provide a detection baseline. Then a Selective Singular Value Decomposition (SVD) is used as a clutter suppression stage before Cell-Averaging Constant False Alarm Rate (CA-CFAR) detection. Then, the benefit of temporal information is also investigated by integrating consecutive RD maps. In parallel, a U-Net neural network is trained to reconstruct target only
representations from the original RD maps, while measurements with clutter only are also included to limit false target generation. Multi frame integration was also examined in the U-Net by stacking consecutive RD maps as separate input channels. All approaches are evaluated on a TNO experimental dataset using the same detection and clutter suppression metrics including target detection fraction, frame-level clutter breakthrough, signal-to-clutter ratio improvement and clutter suppression factor. The results show that Selective Singular Value Decomposition and temporal integration can improve target detection compared with CFAR detection alone, but the clutter breakthrough remained relatively high. On the contrary, the U-Net managed to have a much better balance between target detection and clutter breakthrough, especially when incorporating multiple frames. The three frame U-Net achieved the highest target detection fraction of approximately 94.2% and the four frame U-Net achieved the best balance with a target detection fraction over 90% and average clutter breakthrough 0.04 clusters per RD map. Overall, the results demonstrate that learning based sea clutter suppression is a promising alternative to
conventional maritime radar processing.
direct comparison difficult. In this thesis, classical and machine learning approaches for sea clutter suppression are evaluated under a common framework. A classical detector is first applied directly to RD maps to provide a detection baseline. Then a Selective Singular Value Decomposition (SVD) is used as a clutter suppression stage before Cell-Averaging Constant False Alarm Rate (CA-CFAR) detection. Then, the benefit of temporal information is also investigated by integrating consecutive RD maps. In parallel, a U-Net neural network is trained to reconstruct target only
representations from the original RD maps, while measurements with clutter only are also included to limit false target generation. Multi frame integration was also examined in the U-Net by stacking consecutive RD maps as separate input channels. All approaches are evaluated on a TNO experimental dataset using the same detection and clutter suppression metrics including target detection fraction, frame-level clutter breakthrough, signal-to-clutter ratio improvement and clutter suppression factor. The results show that Selective Singular Value Decomposition and temporal integration can improve target detection compared with CFAR detection alone, but the clutter breakthrough remained relatively high. On the contrary, the U-Net managed to have a much better balance between target detection and clutter breakthrough, especially when incorporating multiple frames. The three frame U-Net achieved the highest target detection fraction of approximately 94.2% and the four frame U-Net achieved the best balance with a target detection fraction over 90% and average clutter breakthrough 0.04 clusters per RD map. Overall, the results demonstrate that learning based sea clutter suppression is a promising alternative to
conventional maritime radar processing.
Bachelor thesis
(2026)
-
M.T. Hoekstra, C. Sparreboom, F. Fioranelli, Y. Aslan, G.J.T. Leus, M. Cvetkovic
For this thesis, a patch antenna was designed. This was done as part of a larger system, designed to detect hostile intruding probes attempting to reach a protected server. The server is housed within a metallic enclosure. This was achieved by both combining theory regarding antennas, resonant cavities and in particular reverberation chambers, and simulating different intrusion scenarios using a designed automated testing setup. The performance of the antenna system was quantified using the deviation in scattering parameters. The final design consisted of a patch antenna with insets to provide matching and a defect in the ground to increase bandwidth. It was designed for a centre frequency of 2.41 GHz and an impedance bandwidth of 240 MHz. Four antennas were then fabricated and tested. The fabricated antennas showed an increase in impedance bandwidth, such that the fabricated antennas demonstrated a bandwidth of 487 MHz. Two of the antennas were then deployed in a physical metallic enclosure and connected via coaxial cables to either a software-defined radio or vector network analyser, which was in turn connected to a Raspberry Pi, responsible for the signal processing pipeline. The proposed antennas were tested and consistently detect the intruding needles. Furthermore, they outperformed a pair of reference antennas, that were not optimised for this application.
...
For this thesis, a patch antenna was designed. This was done as part of a larger system, designed to detect hostile intruding probes attempting to reach a protected server. The server is housed within a metallic enclosure. This was achieved by both combining theory regarding antennas, resonant cavities and in particular reverberation chambers, and simulating different intrusion scenarios using a designed automated testing setup. The performance of the antenna system was quantified using the deviation in scattering parameters. The final design consisted of a patch antenna with insets to provide matching and a defect in the ground to increase bandwidth. It was designed for a centre frequency of 2.41 GHz and an impedance bandwidth of 240 MHz. Four antennas were then fabricated and tested. The fabricated antennas showed an increase in impedance bandwidth, such that the fabricated antennas demonstrated a bandwidth of 487 MHz. Two of the antennas were then deployed in a physical metallic enclosure and connected via coaxial cables to either a software-defined radio or vector network analyser, which was in turn connected to a Raspberry Pi, responsible for the signal processing pipeline. The proposed antennas were tested and consistently detect the intruding needles. Furthermore, they outperformed a pair of reference antennas, that were not optimised for this application.
Protection against hardware attacks is becoming increasingly important, especially when dealing with sensitive data. Well-resourced, highly skilled actors will go to great lengths to compromise a system and extract what they are after. No matter how well you harden the software side, those defenses can be rendered moot the moment an attacker gains physical access — whether by tapping data bus lines or launching other hardware-level attacks.
For these reasons, this thesis is one of the three theses that develop and discuss a proof of concept for an Anti-Tamper Radio system. This is a device that can detect tampering by hardware rather than software means, by measuring the Radio-Frequency (RF) response of the total enclosed system. Physical modifications to such system, for example by inserting probes by a potential attacker, should be detected via variations in the RF response.
This thesis will specifically focus on the RF circuits and system that are at work in the Anti-Tamper Radio. This includes an analysis on how much power is transmitted and received inside the system, what are different approaches for generating and sampling the RF signals, and how reliable is the system. ...
For these reasons, this thesis is one of the three theses that develop and discuss a proof of concept for an Anti-Tamper Radio system. This is a device that can detect tampering by hardware rather than software means, by measuring the Radio-Frequency (RF) response of the total enclosed system. Physical modifications to such system, for example by inserting probes by a potential attacker, should be detected via variations in the RF response.
This thesis will specifically focus on the RF circuits and system that are at work in the Anti-Tamper Radio. This includes an analysis on how much power is transmitted and received inside the system, what are different approaches for generating and sampling the RF signals, and how reliable is the system. ...
Protection against hardware attacks is becoming increasingly important, especially when dealing with sensitive data. Well-resourced, highly skilled actors will go to great lengths to compromise a system and extract what they are after. No matter how well you harden the software side, those defenses can be rendered moot the moment an attacker gains physical access — whether by tapping data bus lines or launching other hardware-level attacks.
For these reasons, this thesis is one of the three theses that develop and discuss a proof of concept for an Anti-Tamper Radio system. This is a device that can detect tampering by hardware rather than software means, by measuring the Radio-Frequency (RF) response of the total enclosed system. Physical modifications to such system, for example by inserting probes by a potential attacker, should be detected via variations in the RF response.
This thesis will specifically focus on the RF circuits and system that are at work in the Anti-Tamper Radio. This includes an analysis on how much power is transmitted and received inside the system, what are different approaches for generating and sampling the RF signals, and how reliable is the system.
For these reasons, this thesis is one of the three theses that develop and discuss a proof of concept for an Anti-Tamper Radio system. This is a device that can detect tampering by hardware rather than software means, by measuring the Radio-Frequency (RF) response of the total enclosed system. Physical modifications to such system, for example by inserting probes by a potential attacker, should be detected via variations in the RF response.
This thesis will specifically focus on the RF circuits and system that are at work in the Anti-Tamper Radio. This includes an analysis on how much power is transmitted and received inside the system, what are different approaches for generating and sampling the RF signals, and how reliable is the system.
Developing ways to protect hardware against physical attacks has become a significant concern for organizations having sensitive data, as physical tampering attacks are becoming more common in addition to the well-known software or cyber attacks.
This BAP project proposes an experimental proof of concept for an Anti-Tamper Radio system, whose goal is to detect when a device is subject to a physical intrusion attempt. The system relies on having a device under protection (e.g., a server) inside a metallic enclosure (e.g., a server rack). The enclosure has an antenna pair mounted inside, which is responsible for gathering the necessary information to make an RF fingerprint of the enclosure.
As the specific focus of this thesis, detection algorithms using four different error metrics have been developed and validated, which all return a detection score. This detection score is then given a meaning, indicating the tamper state of the system. Furthermore, the Isolation Forest machine learning model was implemented and tested for the same tamper detection purpose, and compared to the threshold-based detector. Tampering events are simulated via the insertion of a needle with a 0.15 mm diameter and at a penetration depth of 4 cm.
The developed system, using the Mean Squared Error of the magnitudes, 50 references, no temporal windowing and custom manufactured microstrip patch antennas, managed to detect this intrusion with a 1.30% False Positive Rate, 3.64% False Negative Rate and detection time of 914.0 ms. ...
This BAP project proposes an experimental proof of concept for an Anti-Tamper Radio system, whose goal is to detect when a device is subject to a physical intrusion attempt. The system relies on having a device under protection (e.g., a server) inside a metallic enclosure (e.g., a server rack). The enclosure has an antenna pair mounted inside, which is responsible for gathering the necessary information to make an RF fingerprint of the enclosure.
As the specific focus of this thesis, detection algorithms using four different error metrics have been developed and validated, which all return a detection score. This detection score is then given a meaning, indicating the tamper state of the system. Furthermore, the Isolation Forest machine learning model was implemented and tested for the same tamper detection purpose, and compared to the threshold-based detector. Tampering events are simulated via the insertion of a needle with a 0.15 mm diameter and at a penetration depth of 4 cm.
The developed system, using the Mean Squared Error of the magnitudes, 50 references, no temporal windowing and custom manufactured microstrip patch antennas, managed to detect this intrusion with a 1.30% False Positive Rate, 3.64% False Negative Rate and detection time of 914.0 ms. ...
Developing ways to protect hardware against physical attacks has become a significant concern for organizations having sensitive data, as physical tampering attacks are becoming more common in addition to the well-known software or cyber attacks.
This BAP project proposes an experimental proof of concept for an Anti-Tamper Radio system, whose goal is to detect when a device is subject to a physical intrusion attempt. The system relies on having a device under protection (e.g., a server) inside a metallic enclosure (e.g., a server rack). The enclosure has an antenna pair mounted inside, which is responsible for gathering the necessary information to make an RF fingerprint of the enclosure.
As the specific focus of this thesis, detection algorithms using four different error metrics have been developed and validated, which all return a detection score. This detection score is then given a meaning, indicating the tamper state of the system. Furthermore, the Isolation Forest machine learning model was implemented and tested for the same tamper detection purpose, and compared to the threshold-based detector. Tampering events are simulated via the insertion of a needle with a 0.15 mm diameter and at a penetration depth of 4 cm.
The developed system, using the Mean Squared Error of the magnitudes, 50 references, no temporal windowing and custom manufactured microstrip patch antennas, managed to detect this intrusion with a 1.30% False Positive Rate, 3.64% False Negative Rate and detection time of 914.0 ms.
This BAP project proposes an experimental proof of concept for an Anti-Tamper Radio system, whose goal is to detect when a device is subject to a physical intrusion attempt. The system relies on having a device under protection (e.g., a server) inside a metallic enclosure (e.g., a server rack). The enclosure has an antenna pair mounted inside, which is responsible for gathering the necessary information to make an RF fingerprint of the enclosure.
As the specific focus of this thesis, detection algorithms using four different error metrics have been developed and validated, which all return a detection score. This detection score is then given a meaning, indicating the tamper state of the system. Furthermore, the Isolation Forest machine learning model was implemented and tested for the same tamper detection purpose, and compared to the threshold-based detector. Tampering events are simulated via the insertion of a needle with a 0.15 mm diameter and at a penetration depth of 4 cm.
The developed system, using the Mean Squared Error of the magnitudes, 50 references, no temporal windowing and custom manufactured microstrip patch antennas, managed to detect this intrusion with a 1.30% False Positive Rate, 3.64% False Negative Rate and detection time of 914.0 ms.
Towards Robust Radar Perception in Autonomous Vehicles
Deep Learning Methods for Motion Estimation, Radar Calibration, and Scene Segmentation
Doctoral thesis
(2026)
-
S. Zhu, Alexander Yarovoy , F. Fioranelli
Autonomous driving requires reliable perception under diverse and adverse conditions. Among modern sensing modalities, automotive radar plays a unique role due to its robustness in poor weather and low visual visibility, its direct measurement of radial velocity via the Doppler effect, and its ability to detect objects beyond the line of sight. Despite these advantages, radar data are sparse, noisy, and affected by artifacts such as multipath reflections and sidelobes, resulting in weak geometric representations and high false alarm rates. Consequently, perception methods developed for cameras and lidar cannot be directly transferred to radar. This dissertation advances robust radar perception for autonomous vehicles by developing deep learning methods tailored to radar characteristics for motion estimation, multi-radar fusion, extrinsic calibration, and scene segmentation. The central research question is whether radar can be elevated from a supporting sensor to a primary perception modality capable of delivering robust and accurate information for automotive applications. To address this, the thesis investigates which tasks are best suited to radar, how deep learning can be effectively integrated with radar data, how multiple radars can be fused without strict synchronization, how extrinsic misalignment can be corrected during normal driving, and whether common perception requirements can be addressed within a unified framework.
...
Autonomous driving requires reliable perception under diverse and adverse conditions. Among modern sensing modalities, automotive radar plays a unique role due to its robustness in poor weather and low visual visibility, its direct measurement of radial velocity via the Doppler effect, and its ability to detect objects beyond the line of sight. Despite these advantages, radar data are sparse, noisy, and affected by artifacts such as multipath reflections and sidelobes, resulting in weak geometric representations and high false alarm rates. Consequently, perception methods developed for cameras and lidar cannot be directly transferred to radar. This dissertation advances robust radar perception for autonomous vehicles by developing deep learning methods tailored to radar characteristics for motion estimation, multi-radar fusion, extrinsic calibration, and scene segmentation. The central research question is whether radar can be elevated from a supporting sensor to a primary perception modality capable of delivering robust and accurate information for automotive applications. To address this, the thesis investigates which tasks are best suited to radar, how deep learning can be effectively integrated with radar data, how multiple radars can be fused without strict synchronization, how extrinsic misalignment can be corrected during normal driving, and whether common perception requirements can be addressed within a unified framework.
Towards robust radar perception in autonomous vehicles
Deep learning methods for object detection, velocity estimation, and data association
Doctoral thesis
(2026)
-
S.M. Hassan, Alexander Yarovoy , F. Fioranelli
Autonomous driving relies on the ability of vehicles to perceive and interpret their surroundings reliably under a wide range of operating conditions. Among the sensing modalities used in modern perception systems, automotive radar holds a distinctive position. It operates robustly in adverse weather, directly measures radial velocity, and provides long-range sensing capabilities. These properties make radar a key component for reliable perception in real-world environments. However, radar-based perception remains fundamentally challenging. Radar measurements are sparse and noisy, have limited spatial resolution, and are affected by clutter, multipath reflections, and other artifacts. These characteristics result in weak object representations, unreliable motion estimates, and limited cues for robust data association.
This dissertation addresses these challenges in the context of radar-based multi object tracking consisting of object detection, state estimation, and data association. Within this framework, the limitations of radar perception stem from key weaknesses across the pipeline: object detection is unreliable due to thresholding-induced information loss; velocity estimation is inaccurate under sparse and noisy measurements; and data association is challenged by limited geometric observability and unstable classification. In addition, the effective use of intermediate representations across stages remains underexplored, limiting system-level performance. To address these limitations, this thesis adopts a hybrid perspective in which learning-based components are selectively integrated into a classical tracking pipeline. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components through representations designed specifically for radar data. These components include object detection, state estimation, data association, and downstream integration.
The first contribution, presented in Chapter3 ,addresses radar object detection. Conventional CFAR-based pipelines convert radar spectra into sparse point clouds, thereby discarding localized spectral information that is important for representing extended objects. To overcome this limitation, a spectrum-augmented object detection frame work is proposed in which localized regions of interest are extracted from the radar spectrum and processed using convolutional neural networks to enrich point-level representations. These features are then integrated into a point-based detection pipeline, resulting in stronger object representations and improved detection accuracy. Experimental results show substantial improvements, including a 21.07% increase in mAP on a simulated dataset and an 8.18% increase in mAP on the K-Radar dataset. These results demonstrate that selectively reintroducing radar-specific spectral information can significantly enhance object detection performance.
Building on improved object-level observations, Chapter 4 focuses on state estimation through learning-based instantaneous velocity estimation. Conventional approaches rely on heuristic outlier rejection and least-squares fitting, which are highly sensitive to noise and sparsity. To address these limitations, deep learning-based methods are proposed to improve robustness under sparse and noisy measurements. These methods enable more reliable use of radar motion information in downstream tracking. The proposed approach achieves up to 59% reduction in MAE for velocity estimation and up to 41% reduction in velocity RMSE when integrated into a tracking framework.
Chapter 5 addresses data association and semantic consistency in radar-based multi-object tracking. Because radar measurements provide limited geometric information and single-frame semantic predictions are often unstable, conventional tracking approaches can suffer from unreliable data association and track fragmentation. To address these challenges, a hybrid tracking framework is developed that combines class-agnostic tracking, feature-based data association, and multi-frame track-level classification. Class-agnostic tracking reduces dependence on noisy single-frame class predictions, learned feature representations improve the discriminative power of data association, and temporal aggregation stabilizes semantic estimates over time. Experimental results demonstrate consistent gains in tracking performance, including MOTA improvements of 5.67%, 5.55%, and 12.77% for cars, pedestrians, and cyclists, respectively.
Finally, Chapter6 investigates system-level integration by examining how intermediate motion representations, obtained through learning-based motion prior estimation, can be incorporated into downstream object detection. Motion priors are integrated using two strategies: filtering-based point removal and feature-based augmentation. The results show that filtering-based integration degrades performance because it removes useful structural information from the input representation. In contrast, feature-based integration preserves the full point cloud while enriching it with motion cues, resulting in a 3.63% improvement in mAP over the baseline detector. An oracle analysis further reveals the gap between learned and ideal motion representations, highlighting the potential of motion priors when represented and integrated effectively within the perception pipeline.
Together, the contributions of this dissertation show that the limitations of radar based perception can be effectively addressed through a hybrid design that combines classical tracking with targeted learning-based enhancements. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components — object detection, state estimation, data association, and downstream integration — through representations designed specifically for radar data. More importantly, the results demonstrate that the value of learning-based methods is not limited to improving individual tasks. These methods also produce informative intermediate representations. In turn, these representations enable more reliable and consistent interaction across the different stages of the perception pipeline. By systematically bridging perception components through these representations, this work moves radar perception beyond isolated module-level improvements toward a more integrated and robust system design. Overall, this dissertation establishes automotive radar as a robust and scalable sensing modality capable of delivering strong perception performance while complementing other sensors in future multi-modal perception systems. Furthermore, the presented research provides a principled foundation for the design of next-generation radar-based perception and tracking systems for autonomous driving. ...
This dissertation addresses these challenges in the context of radar-based multi object tracking consisting of object detection, state estimation, and data association. Within this framework, the limitations of radar perception stem from key weaknesses across the pipeline: object detection is unreliable due to thresholding-induced information loss; velocity estimation is inaccurate under sparse and noisy measurements; and data association is challenged by limited geometric observability and unstable classification. In addition, the effective use of intermediate representations across stages remains underexplored, limiting system-level performance. To address these limitations, this thesis adopts a hybrid perspective in which learning-based components are selectively integrated into a classical tracking pipeline. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components through representations designed specifically for radar data. These components include object detection, state estimation, data association, and downstream integration.
The first contribution, presented in Chapter3 ,addresses radar object detection. Conventional CFAR-based pipelines convert radar spectra into sparse point clouds, thereby discarding localized spectral information that is important for representing extended objects. To overcome this limitation, a spectrum-augmented object detection frame work is proposed in which localized regions of interest are extracted from the radar spectrum and processed using convolutional neural networks to enrich point-level representations. These features are then integrated into a point-based detection pipeline, resulting in stronger object representations and improved detection accuracy. Experimental results show substantial improvements, including a 21.07% increase in mAP on a simulated dataset and an 8.18% increase in mAP on the K-Radar dataset. These results demonstrate that selectively reintroducing radar-specific spectral information can significantly enhance object detection performance.
Building on improved object-level observations, Chapter 4 focuses on state estimation through learning-based instantaneous velocity estimation. Conventional approaches rely on heuristic outlier rejection and least-squares fitting, which are highly sensitive to noise and sparsity. To address these limitations, deep learning-based methods are proposed to improve robustness under sparse and noisy measurements. These methods enable more reliable use of radar motion information in downstream tracking. The proposed approach achieves up to 59% reduction in MAE for velocity estimation and up to 41% reduction in velocity RMSE when integrated into a tracking framework.
Chapter 5 addresses data association and semantic consistency in radar-based multi-object tracking. Because radar measurements provide limited geometric information and single-frame semantic predictions are often unstable, conventional tracking approaches can suffer from unreliable data association and track fragmentation. To address these challenges, a hybrid tracking framework is developed that combines class-agnostic tracking, feature-based data association, and multi-frame track-level classification. Class-agnostic tracking reduces dependence on noisy single-frame class predictions, learned feature representations improve the discriminative power of data association, and temporal aggregation stabilizes semantic estimates over time. Experimental results demonstrate consistent gains in tracking performance, including MOTA improvements of 5.67%, 5.55%, and 12.77% for cars, pedestrians, and cyclists, respectively.
Finally, Chapter6 investigates system-level integration by examining how intermediate motion representations, obtained through learning-based motion prior estimation, can be incorporated into downstream object detection. Motion priors are integrated using two strategies: filtering-based point removal and feature-based augmentation. The results show that filtering-based integration degrades performance because it removes useful structural information from the input representation. In contrast, feature-based integration preserves the full point cloud while enriching it with motion cues, resulting in a 3.63% improvement in mAP over the baseline detector. An oracle analysis further reveals the gap between learned and ideal motion representations, highlighting the potential of motion priors when represented and integrated effectively within the perception pipeline.
Together, the contributions of this dissertation show that the limitations of radar based perception can be effectively addressed through a hybrid design that combines classical tracking with targeted learning-based enhancements. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components — object detection, state estimation, data association, and downstream integration — through representations designed specifically for radar data. More importantly, the results demonstrate that the value of learning-based methods is not limited to improving individual tasks. These methods also produce informative intermediate representations. In turn, these representations enable more reliable and consistent interaction across the different stages of the perception pipeline. By systematically bridging perception components through these representations, this work moves radar perception beyond isolated module-level improvements toward a more integrated and robust system design. Overall, this dissertation establishes automotive radar as a robust and scalable sensing modality capable of delivering strong perception performance while complementing other sensors in future multi-modal perception systems. Furthermore, the presented research provides a principled foundation for the design of next-generation radar-based perception and tracking systems for autonomous driving. ...
Autonomous driving relies on the ability of vehicles to perceive and interpret their surroundings reliably under a wide range of operating conditions. Among the sensing modalities used in modern perception systems, automotive radar holds a distinctive position. It operates robustly in adverse weather, directly measures radial velocity, and provides long-range sensing capabilities. These properties make radar a key component for reliable perception in real-world environments. However, radar-based perception remains fundamentally challenging. Radar measurements are sparse and noisy, have limited spatial resolution, and are affected by clutter, multipath reflections, and other artifacts. These characteristics result in weak object representations, unreliable motion estimates, and limited cues for robust data association.
This dissertation addresses these challenges in the context of radar-based multi object tracking consisting of object detection, state estimation, and data association. Within this framework, the limitations of radar perception stem from key weaknesses across the pipeline: object detection is unreliable due to thresholding-induced information loss; velocity estimation is inaccurate under sparse and noisy measurements; and data association is challenged by limited geometric observability and unstable classification. In addition, the effective use of intermediate representations across stages remains underexplored, limiting system-level performance. To address these limitations, this thesis adopts a hybrid perspective in which learning-based components are selectively integrated into a classical tracking pipeline. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components through representations designed specifically for radar data. These components include object detection, state estimation, data association, and downstream integration.
The first contribution, presented in Chapter3 ,addresses radar object detection. Conventional CFAR-based pipelines convert radar spectra into sparse point clouds, thereby discarding localized spectral information that is important for representing extended objects. To overcome this limitation, a spectrum-augmented object detection frame work is proposed in which localized regions of interest are extracted from the radar spectrum and processed using convolutional neural networks to enrich point-level representations. These features are then integrated into a point-based detection pipeline, resulting in stronger object representations and improved detection accuracy. Experimental results show substantial improvements, including a 21.07% increase in mAP on a simulated dataset and an 8.18% increase in mAP on the K-Radar dataset. These results demonstrate that selectively reintroducing radar-specific spectral information can significantly enhance object detection performance.
Building on improved object-level observations, Chapter 4 focuses on state estimation through learning-based instantaneous velocity estimation. Conventional approaches rely on heuristic outlier rejection and least-squares fitting, which are highly sensitive to noise and sparsity. To address these limitations, deep learning-based methods are proposed to improve robustness under sparse and noisy measurements. These methods enable more reliable use of radar motion information in downstream tracking. The proposed approach achieves up to 59% reduction in MAE for velocity estimation and up to 41% reduction in velocity RMSE when integrated into a tracking framework.
Chapter 5 addresses data association and semantic consistency in radar-based multi-object tracking. Because radar measurements provide limited geometric information and single-frame semantic predictions are often unstable, conventional tracking approaches can suffer from unreliable data association and track fragmentation. To address these challenges, a hybrid tracking framework is developed that combines class-agnostic tracking, feature-based data association, and multi-frame track-level classification. Class-agnostic tracking reduces dependence on noisy single-frame class predictions, learned feature representations improve the discriminative power of data association, and temporal aggregation stabilizes semantic estimates over time. Experimental results demonstrate consistent gains in tracking performance, including MOTA improvements of 5.67%, 5.55%, and 12.77% for cars, pedestrians, and cyclists, respectively.
Finally, Chapter6 investigates system-level integration by examining how intermediate motion representations, obtained through learning-based motion prior estimation, can be incorporated into downstream object detection. Motion priors are integrated using two strategies: filtering-based point removal and feature-based augmentation. The results show that filtering-based integration degrades performance because it removes useful structural information from the input representation. In contrast, feature-based integration preserves the full point cloud while enriching it with motion cues, resulting in a 3.63% improvement in mAP over the baseline detector. An oracle analysis further reveals the gap between learned and ideal motion representations, highlighting the potential of motion priors when represented and integrated effectively within the perception pipeline.
Together, the contributions of this dissertation show that the limitations of radar based perception can be effectively addressed through a hybrid design that combines classical tracking with targeted learning-based enhancements. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components — object detection, state estimation, data association, and downstream integration — through representations designed specifically for radar data. More importantly, the results demonstrate that the value of learning-based methods is not limited to improving individual tasks. These methods also produce informative intermediate representations. In turn, these representations enable more reliable and consistent interaction across the different stages of the perception pipeline. By systematically bridging perception components through these representations, this work moves radar perception beyond isolated module-level improvements toward a more integrated and robust system design. Overall, this dissertation establishes automotive radar as a robust and scalable sensing modality capable of delivering strong perception performance while complementing other sensors in future multi-modal perception systems. Furthermore, the presented research provides a principled foundation for the design of next-generation radar-based perception and tracking systems for autonomous driving.
This dissertation addresses these challenges in the context of radar-based multi object tracking consisting of object detection, state estimation, and data association. Within this framework, the limitations of radar perception stem from key weaknesses across the pipeline: object detection is unreliable due to thresholding-induced information loss; velocity estimation is inaccurate under sparse and noisy measurements; and data association is challenged by limited geometric observability and unstable classification. In addition, the effective use of intermediate representations across stages remains underexplored, limiting system-level performance. To address these limitations, this thesis adopts a hybrid perspective in which learning-based components are selectively integrated into a classical tracking pipeline. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components through representations designed specifically for radar data. These components include object detection, state estimation, data association, and downstream integration.
The first contribution, presented in Chapter3 ,addresses radar object detection. Conventional CFAR-based pipelines convert radar spectra into sparse point clouds, thereby discarding localized spectral information that is important for representing extended objects. To overcome this limitation, a spectrum-augmented object detection frame work is proposed in which localized regions of interest are extracted from the radar spectrum and processed using convolutional neural networks to enrich point-level representations. These features are then integrated into a point-based detection pipeline, resulting in stronger object representations and improved detection accuracy. Experimental results show substantial improvements, including a 21.07% increase in mAP on a simulated dataset and an 8.18% increase in mAP on the K-Radar dataset. These results demonstrate that selectively reintroducing radar-specific spectral information can significantly enhance object detection performance.
Building on improved object-level observations, Chapter 4 focuses on state estimation through learning-based instantaneous velocity estimation. Conventional approaches rely on heuristic outlier rejection and least-squares fitting, which are highly sensitive to noise and sparsity. To address these limitations, deep learning-based methods are proposed to improve robustness under sparse and noisy measurements. These methods enable more reliable use of radar motion information in downstream tracking. The proposed approach achieves up to 59% reduction in MAE for velocity estimation and up to 41% reduction in velocity RMSE when integrated into a tracking framework.
Chapter 5 addresses data association and semantic consistency in radar-based multi-object tracking. Because radar measurements provide limited geometric information and single-frame semantic predictions are often unstable, conventional tracking approaches can suffer from unreliable data association and track fragmentation. To address these challenges, a hybrid tracking framework is developed that combines class-agnostic tracking, feature-based data association, and multi-frame track-level classification. Class-agnostic tracking reduces dependence on noisy single-frame class predictions, learned feature representations improve the discriminative power of data association, and temporal aggregation stabilizes semantic estimates over time. Experimental results demonstrate consistent gains in tracking performance, including MOTA improvements of 5.67%, 5.55%, and 12.77% for cars, pedestrians, and cyclists, respectively.
Finally, Chapter6 investigates system-level integration by examining how intermediate motion representations, obtained through learning-based motion prior estimation, can be incorporated into downstream object detection. Motion priors are integrated using two strategies: filtering-based point removal and feature-based augmentation. The results show that filtering-based integration degrades performance because it removes useful structural information from the input representation. In contrast, feature-based integration preserves the full point cloud while enriching it with motion cues, resulting in a 3.63% improvement in mAP over the baseline detector. An oracle analysis further reveals the gap between learned and ideal motion representations, highlighting the potential of motion priors when represented and integrated effectively within the perception pipeline.
Together, the contributions of this dissertation show that the limitations of radar based perception can be effectively addressed through a hybrid design that combines classical tracking with targeted learning-based enhancements. Rather than replacing the perception pipeline with a fully end-to-end solution, the proposed approach strengthens its key components — object detection, state estimation, data association, and downstream integration — through representations designed specifically for radar data. More importantly, the results demonstrate that the value of learning-based methods is not limited to improving individual tasks. These methods also produce informative intermediate representations. In turn, these representations enable more reliable and consistent interaction across the different stages of the perception pipeline. By systematically bridging perception components through these representations, this work moves radar perception beyond isolated module-level improvements toward a more integrated and robust system design. Overall, this dissertation establishes automotive radar as a robust and scalable sensing modality capable of delivering strong perception performance while complementing other sensors in future multi-modal perception systems. Furthermore, the presented research provides a principled foundation for the design of next-generation radar-based perception and tracking systems for autonomous driving.
In smart office environments, intelligent control of IoT devices based on human movement and the monitoring of sedentary behavior are crucial. To address object occlusion problem, this thesis proposes an innovative solution: integrating UWB radars into ceiling-mounted office lights, in contrast to conventional wall-mounted setups. The core contribution of this work is the development of an indoor human localization and activity recognition system using ceiling-mounted radars. A novel method is proposed for joint human tracking and activity recognition. The system and the proposed method is validated in a 3.5 m × 6 m office environment with three ceiling-mounted Novelda X4 radars. The experimental results demonstrate high performance: trajectory tracking achieves a root mean square error (RMSE) as low as 0.23 m, representing a 52.6% improvement over the baseline method, while activity classification for walking, standing, and sitting reaches an accuracy of up to 98%. These findings demonstrate the feasibility and effectiveness of ceiling-mounted UWB radars for accurate human tracking and activity recognition in office-like environments.
...
In smart office environments, intelligent control of IoT devices based on human movement and the monitoring of sedentary behavior are crucial. To address object occlusion problem, this thesis proposes an innovative solution: integrating UWB radars into ceiling-mounted office lights, in contrast to conventional wall-mounted setups. The core contribution of this work is the development of an indoor human localization and activity recognition system using ceiling-mounted radars. A novel method is proposed for joint human tracking and activity recognition. The system and the proposed method is validated in a 3.5 m × 6 m office environment with three ceiling-mounted Novelda X4 radars. The experimental results demonstrate high performance: trajectory tracking achieves a root mean square error (RMSE) as low as 0.23 m, representing a 52.6% improvement over the baseline method, while activity classification for walking, standing, and sitting reaches an accuracy of up to 98%. These findings demonstrate the feasibility and effectiveness of ceiling-mounted UWB radars for accurate human tracking and activity recognition in office-like environments.
Automotive radar can provide robust measurements that are crucial for autonomous driving localization and perception tasks, especially under adverse weather, low-light, and long-range conditions. This thesis proposes two radar-only, combined estimation methods that both provide ego-motion information and track multiple extended objects. The key to achieve this is addressing the challenge of distinguishing static and moving targets, using either radar raw signals or radar point clouds.
The raw signal based method is validated in its theoretical feasibility through simulations, while the point cloud based method is shown to improve ego-motion estimation in practical dynamic driving scenarios. With simulation of such scenarios generated using a dedicated MATLAB tool, the proposed approach outperforms the RANSAC-based baseline by reducing the APE metric by 2.00 m/s and the RTE metric by 1.58 m, and achieves accurate position and size estimates for tracked objects. Further validations on the experimental RadarScenes dataset show average APE reductions of 25.9% over entire scenes and 56.9% in critical 10-second segments across 7 dynamic traffic scenes. These results demonstrate the effectiveness of radar-only combined estimation of ego-motion and multiple object tracks, for robust localization and perception in complex traffic scenarios.
...
Automotive radar can provide robust measurements that are crucial for autonomous driving localization and perception tasks, especially under adverse weather, low-light, and long-range conditions. This thesis proposes two radar-only, combined estimation methods that both provide ego-motion information and track multiple extended objects. The key to achieve this is addressing the challenge of distinguishing static and moving targets, using either radar raw signals or radar point clouds.
The raw signal based method is validated in its theoretical feasibility through simulations, while the point cloud based method is shown to improve ego-motion estimation in practical dynamic driving scenarios. With simulation of such scenarios generated using a dedicated MATLAB tool, the proposed approach outperforms the RANSAC-based baseline by reducing the APE metric by 2.00 m/s and the RTE metric by 1.58 m, and achieves accurate position and size estimates for tracked objects. Further validations on the experimental RadarScenes dataset show average APE reductions of 25.9% over entire scenes and 56.9% in critical 10-second segments across 7 dynamic traffic scenes. These results demonstrate the effectiveness of radar-only combined estimation of ego-motion and multiple object tracks, for robust localization and perception in complex traffic scenarios.
This thesis presents a novel approach based on Joint Deep Probabilistic Subsampling with Cram´er-Rao Lower Bound Integration (J-DPSC) for sparse antenna array design in distributed radar systems. The method addresses the critical challenge of achieving high angular resolution in autonomous driving applications while maintaining low hardware complexity. The proposed J-DPSC framework extends Deep Probabilistic Subsampling with theoretical foundations based on worst-case dual-target Cram´er-Rao Lower Bound analysis. This enables simultaneous optimization of both transmitter and receiver arrays across mono-static, bi-static, and joint operational modes. A Neighborhood Masking Mechanism ensures physical realizability of the designed arrays. Extensive simulations validate that our proposed sparse arrays achieve performance comparable to fully populated arrays while using significantly fewer elements. The results demonstrate peak sidelobe levels up to −10 dB with minimal resolution sacrifice (≤ 0.1 degree) for two-sensor distributed radar networks. The approach allows extension to three-sensor configurations for potential applications with multi-node networks. Monte Carlo evaluations across varying SNR levels and source (i.e., targets) separations show consistent superiority over random selection methods.
Summarizing, this work advances sparse array design methodology by bridging machine learning techniques with estimation theory, enabling a more generalized and flexible design tool for distributed radar networks in next-generation autonomous driving applications. ...
Summarizing, this work advances sparse array design methodology by bridging machine learning techniques with estimation theory, enabling a more generalized and flexible design tool for distributed radar networks in next-generation autonomous driving applications. ...
This thesis presents a novel approach based on Joint Deep Probabilistic Subsampling with Cram´er-Rao Lower Bound Integration (J-DPSC) for sparse antenna array design in distributed radar systems. The method addresses the critical challenge of achieving high angular resolution in autonomous driving applications while maintaining low hardware complexity. The proposed J-DPSC framework extends Deep Probabilistic Subsampling with theoretical foundations based on worst-case dual-target Cram´er-Rao Lower Bound analysis. This enables simultaneous optimization of both transmitter and receiver arrays across mono-static, bi-static, and joint operational modes. A Neighborhood Masking Mechanism ensures physical realizability of the designed arrays. Extensive simulations validate that our proposed sparse arrays achieve performance comparable to fully populated arrays while using significantly fewer elements. The results demonstrate peak sidelobe levels up to −10 dB with minimal resolution sacrifice (≤ 0.1 degree) for two-sensor distributed radar networks. The approach allows extension to three-sensor configurations for potential applications with multi-node networks. Monte Carlo evaluations across varying SNR levels and source (i.e., targets) separations show consistent superiority over random selection methods.
Summarizing, this work advances sparse array design methodology by bridging machine learning techniques with estimation theory, enabling a more generalized and flexible design tool for distributed radar networks in next-generation autonomous driving applications.
Summarizing, this work advances sparse array design methodology by bridging machine learning techniques with estimation theory, enabling a more generalized and flexible design tool for distributed radar networks in next-generation autonomous driving applications.
Perception is a fundamental component of autonomous and self-driving vehicles, with reliable object detection and understanding of the environment being critical for safe operation. While lidar and camera based systems are widely used, radar remains a promising option due to its robustness in poor weather conditions and ability to directly measure the radial velocity of objects via the Doppler effect. However, radar’s sparse data and resulting limitations have constrained its potential. This thesis investigates the use of dual automotive radar setups to mitigate these limitations, and use the specific advantages of such a setup to improve full velocity vector estimation methods. A novel algorithm is proposed to achieve more accurate velocity estimation in non-ideal, real world conditions. The work further explores how improved velocity estimation can be used to improve classification performance using graph neural networks. Here it was found that including velocity information via a ground truth method did increase classification performance significantly, though the same result could not be obtained via the previously mentioned velocity estimation method. To support evaluation, a simplified simulation environment and ground truth velocity data for the RadarScenes dataset are developed. This research aims to close the performance gap between radar and other more data-dense sensors, offering a robust and more cost effective alternative, especially in conditions where optical systems under perform.
...
Perception is a fundamental component of autonomous and self-driving vehicles, with reliable object detection and understanding of the environment being critical for safe operation. While lidar and camera based systems are widely used, radar remains a promising option due to its robustness in poor weather conditions and ability to directly measure the radial velocity of objects via the Doppler effect. However, radar’s sparse data and resulting limitations have constrained its potential. This thesis investigates the use of dual automotive radar setups to mitigate these limitations, and use the specific advantages of such a setup to improve full velocity vector estimation methods. A novel algorithm is proposed to achieve more accurate velocity estimation in non-ideal, real world conditions. The work further explores how improved velocity estimation can be used to improve classification performance using graph neural networks. Here it was found that including velocity information via a ground truth method did increase classification performance significantly, though the same result could not be obtained via the previously mentioned velocity estimation method. To support evaluation, a simplified simulation environment and ground truth velocity data for the RadarScenes dataset are developed. This research aims to close the performance gap between radar and other more data-dense sensors, offering a robust and more cost effective alternative, especially in conditions where optical systems under perform.
Doctoral thesis
(2025)
-
N.C. Kruse, F. Fioranelli, Alexander Yarovoy
Radar sensors are an emerging technology in the context of non-contact monitoring of vulnerable individuals. Radar-based solutions ensure end-user privacy, whilst providing medical professionals and caregivers with key information concerning the subject's well-being. This thesis proposes novel methods for the classification of sequential human activities using a network of radar sensors. Accurate classification of Activities of Daily Life (ADL) can enable for instance the detection of falls and wandering amongst elderly individuals, and can be employed for the recognition of aggressive or otherwise anomalous behaviour for those receiving mental health care.
...
Radar sensors are an emerging technology in the context of non-contact monitoring of vulnerable individuals. Radar-based solutions ensure end-user privacy, whilst providing medical professionals and caregivers with key information concerning the subject's well-being. This thesis proposes novel methods for the classification of sequential human activities using a network of radar sensors. Accurate classification of Activities of Daily Life (ADL) can enable for instance the detection of falls and wandering amongst elderly individuals, and can be employed for the recognition of aggressive or otherwise anomalous behaviour for those receiving mental health care.
Ultra-Wideband (UWB) technology is currently widely in use in wireless communication and radar systems, with diverse applications ranging from imaging and tumor detection to multi-human detection and secure localization in mobile and automotive applications. In this context, mmWave systems are an emerging special class of UWB radar technology that employs short-wavelength electromagnetic waves.
With UWB radar technology, car cabin safety systems and tasks such as presence detection, occupancy localization, and driver health monitoring systems can be realized. Exploring effective means of presence detection and vital sign monitoring is crucial to improve automotive safety.
This thesis work presents a novel measurement setup for concurrent multi-radar, multi-band operation of both a 6-9 GHz UWB radar and a mmWave FMCW radar operating at 60 GHz. This addresses an unexplored area with the potential for advancements in non-contact vital sign monitoring that can leverage the advantages of each frequency range to improve performance. Furthermore, this work evaluates the benefits and trade-offs of the multi-band, multi-radar concurrent operation for presence detection, localization, and vital sign estimation applications, and evaluates the performance of the radars individually or in a combined, concurrent manner in the same use cases.
With a series of experimental results and a proposed processing pipeline for detection and association, the proposed concurrent multi-radar, multi-frequency operation achieves improved performance in presence detection by leveraging the unique advantages of each radar. Specifically, the high performance of TI radar in scenarios requiring angular resolution complements the strengths of Ranger5 in cases where one person occludes another.
...
With UWB radar technology, car cabin safety systems and tasks such as presence detection, occupancy localization, and driver health monitoring systems can be realized. Exploring effective means of presence detection and vital sign monitoring is crucial to improve automotive safety.
This thesis work presents a novel measurement setup for concurrent multi-radar, multi-band operation of both a 6-9 GHz UWB radar and a mmWave FMCW radar operating at 60 GHz. This addresses an unexplored area with the potential for advancements in non-contact vital sign monitoring that can leverage the advantages of each frequency range to improve performance. Furthermore, this work evaluates the benefits and trade-offs of the multi-band, multi-radar concurrent operation for presence detection, localization, and vital sign estimation applications, and evaluates the performance of the radars individually or in a combined, concurrent manner in the same use cases.
With a series of experimental results and a proposed processing pipeline for detection and association, the proposed concurrent multi-radar, multi-frequency operation achieves improved performance in presence detection by leveraging the unique advantages of each radar. Specifically, the high performance of TI radar in scenarios requiring angular resolution complements the strengths of Ranger5 in cases where one person occludes another.
...
Ultra-Wideband (UWB) technology is currently widely in use in wireless communication and radar systems, with diverse applications ranging from imaging and tumor detection to multi-human detection and secure localization in mobile and automotive applications. In this context, mmWave systems are an emerging special class of UWB radar technology that employs short-wavelength electromagnetic waves.
With UWB radar technology, car cabin safety systems and tasks such as presence detection, occupancy localization, and driver health monitoring systems can be realized. Exploring effective means of presence detection and vital sign monitoring is crucial to improve automotive safety.
This thesis work presents a novel measurement setup for concurrent multi-radar, multi-band operation of both a 6-9 GHz UWB radar and a mmWave FMCW radar operating at 60 GHz. This addresses an unexplored area with the potential for advancements in non-contact vital sign monitoring that can leverage the advantages of each frequency range to improve performance. Furthermore, this work evaluates the benefits and trade-offs of the multi-band, multi-radar concurrent operation for presence detection, localization, and vital sign estimation applications, and evaluates the performance of the radars individually or in a combined, concurrent manner in the same use cases.
With a series of experimental results and a proposed processing pipeline for detection and association, the proposed concurrent multi-radar, multi-frequency operation achieves improved performance in presence detection by leveraging the unique advantages of each radar. Specifically, the high performance of TI radar in scenarios requiring angular resolution complements the strengths of Ranger5 in cases where one person occludes another.
With UWB radar technology, car cabin safety systems and tasks such as presence detection, occupancy localization, and driver health monitoring systems can be realized. Exploring effective means of presence detection and vital sign monitoring is crucial to improve automotive safety.
This thesis work presents a novel measurement setup for concurrent multi-radar, multi-band operation of both a 6-9 GHz UWB radar and a mmWave FMCW radar operating at 60 GHz. This addresses an unexplored area with the potential for advancements in non-contact vital sign monitoring that can leverage the advantages of each frequency range to improve performance. Furthermore, this work evaluates the benefits and trade-offs of the multi-band, multi-radar concurrent operation for presence detection, localization, and vital sign estimation applications, and evaluates the performance of the radars individually or in a combined, concurrent manner in the same use cases.
With a series of experimental results and a proposed processing pipeline for detection and association, the proposed concurrent multi-radar, multi-frequency operation achieves improved performance in presence detection by leveraging the unique advantages of each radar. Specifically, the high performance of TI radar in scenarios requiring angular resolution complements the strengths of Ranger5 in cases where one person occludes another.
Master thesis
(2024)
-
L.M.S.J. de Ghellinck d'Elseghem, C. Frenkel, F. Fioranelli, Federico Corradi
Radar-based sensors are used to perceive their environment and objects of interest in a contactless manner and with robust performance in all weather and light conditions. One of the main drawbacks is the energy needed for the processing of radar data in order to extract its valuable information. Spiking neural networks are an emerging type of neural networks that aim to reduce the energy footprint of their computations while maintaining acceptable performance. To do so, the data is encoded through time in binary spikes to help leverage the low cost of additions. This is in stark opposition to the much higher cost of multiplications that are highly present in conventional artificial neural networks. The drawback of this energy gain is that the rate encoding adds an extra time dimension, hence increasing the latency between the acquisition of the radar data and the recognition of the corresponding gesture class.
More specifically, this work uses an air-marshalling dataset from the literature to exemplify a gesture recognition problem. The first step is to replicate the well-known radar processing pipeline, and classification approach based on conventional neural networks to reach high classification accuracies. A validation accuracy of 98.5% and a test accuracy of 59.8% are reached on the full dataset (11 classes) and 86.7% on their 5 best classes (test set), which is about the same performance reported in the original dataset baseline.
The following steps propose an adaptation of this non-spiking pipeline to its spiking equivalent by optimising the trade-off between the model’s latency, its memory requirements and its accuracy. This work also develops a strategy to tune spiking networks’ thresholds to make the process of developing a spiking equivalent more efficient. For example, the spiking network can reach 94.1% validation accuracy using 100 encoding steps and only 4.7% of the initial memory requirements, and reach 46.8% on the test set. However, this trade-off can be shifted towards lower latency, lower memory, or higher accuracy according to the desired requirements. ...
More specifically, this work uses an air-marshalling dataset from the literature to exemplify a gesture recognition problem. The first step is to replicate the well-known radar processing pipeline, and classification approach based on conventional neural networks to reach high classification accuracies. A validation accuracy of 98.5% and a test accuracy of 59.8% are reached on the full dataset (11 classes) and 86.7% on their 5 best classes (test set), which is about the same performance reported in the original dataset baseline.
The following steps propose an adaptation of this non-spiking pipeline to its spiking equivalent by optimising the trade-off between the model’s latency, its memory requirements and its accuracy. This work also develops a strategy to tune spiking networks’ thresholds to make the process of developing a spiking equivalent more efficient. For example, the spiking network can reach 94.1% validation accuracy using 100 encoding steps and only 4.7% of the initial memory requirements, and reach 46.8% on the test set. However, this trade-off can be shifted towards lower latency, lower memory, or higher accuracy according to the desired requirements. ...
Radar-based sensors are used to perceive their environment and objects of interest in a contactless manner and with robust performance in all weather and light conditions. One of the main drawbacks is the energy needed for the processing of radar data in order to extract its valuable information. Spiking neural networks are an emerging type of neural networks that aim to reduce the energy footprint of their computations while maintaining acceptable performance. To do so, the data is encoded through time in binary spikes to help leverage the low cost of additions. This is in stark opposition to the much higher cost of multiplications that are highly present in conventional artificial neural networks. The drawback of this energy gain is that the rate encoding adds an extra time dimension, hence increasing the latency between the acquisition of the radar data and the recognition of the corresponding gesture class.
More specifically, this work uses an air-marshalling dataset from the literature to exemplify a gesture recognition problem. The first step is to replicate the well-known radar processing pipeline, and classification approach based on conventional neural networks to reach high classification accuracies. A validation accuracy of 98.5% and a test accuracy of 59.8% are reached on the full dataset (11 classes) and 86.7% on their 5 best classes (test set), which is about the same performance reported in the original dataset baseline.
The following steps propose an adaptation of this non-spiking pipeline to its spiking equivalent by optimising the trade-off between the model’s latency, its memory requirements and its accuracy. This work also develops a strategy to tune spiking networks’ thresholds to make the process of developing a spiking equivalent more efficient. For example, the spiking network can reach 94.1% validation accuracy using 100 encoding steps and only 4.7% of the initial memory requirements, and reach 46.8% on the test set. However, this trade-off can be shifted towards lower latency, lower memory, or higher accuracy according to the desired requirements.
More specifically, this work uses an air-marshalling dataset from the literature to exemplify a gesture recognition problem. The first step is to replicate the well-known radar processing pipeline, and classification approach based on conventional neural networks to reach high classification accuracies. A validation accuracy of 98.5% and a test accuracy of 59.8% are reached on the full dataset (11 classes) and 86.7% on their 5 best classes (test set), which is about the same performance reported in the original dataset baseline.
The following steps propose an adaptation of this non-spiking pipeline to its spiking equivalent by optimising the trade-off between the model’s latency, its memory requirements and its accuracy. This work also develops a strategy to tune spiking networks’ thresholds to make the process of developing a spiking equivalent more efficient. For example, the spiking network can reach 94.1% validation accuracy using 100 encoding steps and only 4.7% of the initial memory requirements, and reach 46.8% on the test set. However, this trade-off can be shifted towards lower latency, lower memory, or higher accuracy according to the desired requirements.
Extreme precipitation, like floods and landslides, poses major risks to safety and the economy, underscoring the need for sophisticated weather forecasting to predict these events accurately, enhancing readiness and resilience. Nowcasting, which uses real-time atmospheric data to predict short-term weather, is key in addressing this challenge. Traditional nowcasting systems, reliant on extrapolation from rainfall radar observations and constrained by simplistic physical assumptions, often struggle to detect complex, nonlinear weather patterns. This gap has opened the door for deep learning models, which have shown significant promise in improving the accuracy and reliability of short-term weather predictions, making them a focal point of recent research and the basis of this thesis's approach.
This thesis introduces a deep generative model designed for the nowcasting of extreme precipitation events up to 3 hours ahead, utilizing a Vector-Quantized Variational Autoencoder (VQ-VAE) to compress radar data into a low-dimensional latent representation, and an Autoregressive Transformer for predicting future radar images. Additionally, a binary classifier works in conjunction with the Autoregressive Transformer to identify extreme versus non-extreme weather events, using these classifications to inform an Extreme Value Loss (EVL) function. This loss function aims to improve the accuracy of predicting extreme weather events by addressing the data imbalance between normal and extreme precipitation occurrences. The proposed model displays comparable performance with the state-of-the-art conventional methods and other deep learning nowcasting models in predicting extreme events. ...
This thesis introduces a deep generative model designed for the nowcasting of extreme precipitation events up to 3 hours ahead, utilizing a Vector-Quantized Variational Autoencoder (VQ-VAE) to compress radar data into a low-dimensional latent representation, and an Autoregressive Transformer for predicting future radar images. Additionally, a binary classifier works in conjunction with the Autoregressive Transformer to identify extreme versus non-extreme weather events, using these classifications to inform an Extreme Value Loss (EVL) function. This loss function aims to improve the accuracy of predicting extreme weather events by addressing the data imbalance between normal and extreme precipitation occurrences. The proposed model displays comparable performance with the state-of-the-art conventional methods and other deep learning nowcasting models in predicting extreme events. ...
Extreme precipitation, like floods and landslides, poses major risks to safety and the economy, underscoring the need for sophisticated weather forecasting to predict these events accurately, enhancing readiness and resilience. Nowcasting, which uses real-time atmospheric data to predict short-term weather, is key in addressing this challenge. Traditional nowcasting systems, reliant on extrapolation from rainfall radar observations and constrained by simplistic physical assumptions, often struggle to detect complex, nonlinear weather patterns. This gap has opened the door for deep learning models, which have shown significant promise in improving the accuracy and reliability of short-term weather predictions, making them a focal point of recent research and the basis of this thesis's approach.
This thesis introduces a deep generative model designed for the nowcasting of extreme precipitation events up to 3 hours ahead, utilizing a Vector-Quantized Variational Autoencoder (VQ-VAE) to compress radar data into a low-dimensional latent representation, and an Autoregressive Transformer for predicting future radar images. Additionally, a binary classifier works in conjunction with the Autoregressive Transformer to identify extreme versus non-extreme weather events, using these classifications to inform an Extreme Value Loss (EVL) function. This loss function aims to improve the accuracy of predicting extreme weather events by addressing the data imbalance between normal and extreme precipitation occurrences. The proposed model displays comparable performance with the state-of-the-art conventional methods and other deep learning nowcasting models in predicting extreme events.
This thesis introduces a deep generative model designed for the nowcasting of extreme precipitation events up to 3 hours ahead, utilizing a Vector-Quantized Variational Autoencoder (VQ-VAE) to compress radar data into a low-dimensional latent representation, and an Autoregressive Transformer for predicting future radar images. Additionally, a binary classifier works in conjunction with the Autoregressive Transformer to identify extreme versus non-extreme weather events, using these classifications to inform an Extreme Value Loss (EVL) function. This loss function aims to improve the accuracy of predicting extreme weather events by addressing the data imbalance between normal and extreme precipitation occurrences. The proposed model displays comparable performance with the state-of-the-art conventional methods and other deep learning nowcasting models in predicting extreme events.
Traditional target tracking using monostatic radar systems typically relies on centralized or decentralized architectures, where all data is transmitted to a fusion center for processing the position and velocity of mobile targets. This approach not only introduces a single point of failure and can lead to increased data transmission times, particularly when the fusion center is distant from individual radar nodes, but it also faces scalability issues and potential bottlenecks when data accumulates at the fusion center. To address these challenges, we introduce a Distributed Alternating Direction Method of Multipliers (DADMM) for target localization within a radar network, wherein each radar node shares its observed data only with immediate neighboring nodes, achieving consensus on the estimated target locations and velocities. Our simulations, which incorporate critical parameters such as the number of radar nodes, radar geometry, and Signal-to-Noise Ratio (SNR), assess their impact on estimation accuracy and convergence speed. The results demonstrate that the proposed DADMM not only effectively eliminates the single point of failure, but also enhances system efficiency and robustness. We also incorporate two distinct stopping criteria for position and velocity estimations, enabling us to promptly fix the first accurately estimated parameter and reallocate computational resources to more effectively refine the remaining parameters, which streamlines computational efforts by focusing on unresolved parameters. We highlight the additional benefits of our proposed framework, and present directions for future work.
...
Traditional target tracking using monostatic radar systems typically relies on centralized or decentralized architectures, where all data is transmitted to a fusion center for processing the position and velocity of mobile targets. This approach not only introduces a single point of failure and can lead to increased data transmission times, particularly when the fusion center is distant from individual radar nodes, but it also faces scalability issues and potential bottlenecks when data accumulates at the fusion center. To address these challenges, we introduce a Distributed Alternating Direction Method of Multipliers (DADMM) for target localization within a radar network, wherein each radar node shares its observed data only with immediate neighboring nodes, achieving consensus on the estimated target locations and velocities. Our simulations, which incorporate critical parameters such as the number of radar nodes, radar geometry, and Signal-to-Noise Ratio (SNR), assess their impact on estimation accuracy and convergence speed. The results demonstrate that the proposed DADMM not only effectively eliminates the single point of failure, but also enhances system efficiency and robustness. We also incorporate two distinct stopping criteria for position and velocity estimations, enabling us to promptly fix the first accurately estimated parameter and reallocate computational resources to more effectively refine the remaining parameters, which streamlines computational efforts by focusing on unresolved parameters. We highlight the additional benefits of our proposed framework, and present directions for future work.
4D millimeter-wave radar is increasingly important in advanced driver-assistance systems due to its ability to capture Doppler/velocity information and robustness in low-light or adverse weather conditions. Unlike traditional 3D radar, 4D radar provides elevation information, enhancing 3D spatial perception. Most of the perception tasks using 4D radar tensors focus on target classification using bounding boxes. In contrast, although semantic segmentation is typically used to process images and LiDAR point clouds, it has not been well explored for 4D radar tensors.
This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.
...
This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.
...
4D millimeter-wave radar is increasingly important in advanced driver-assistance systems due to its ability to capture Doppler/velocity information and robustness in low-light or adverse weather conditions. Unlike traditional 3D radar, 4D radar provides elevation information, enhancing 3D spatial perception. Most of the perception tasks using 4D radar tensors focus on target classification using bounding boxes. In contrast, although semantic segmentation is typically used to process images and LiDAR point clouds, it has not been well explored for 4D radar tensors.
This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.
This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.
Doctoral thesis
(2024)
-
W. Bouwmeester, Alexander Yarovoy , F. Fioranelli
Due to recent advances in miniaturisation and cost reductions of radar systems, an increasing amount of vehicles is equipped with automotive radar systems to enable advanced driver assistance systems (ADAS) which improve driving comfort and road safety. Examples of these systems include automatic braking as well as adaptive cruise control. To advance the capabilities of these systems and to take another step towards enabling autonomous driving, the next generation of radar sensors aims to enhance the situational awareness of the driver and the vehicle by, amongst others, classifying objects in the surrounding area of the car. Arguably, the most important objects to correctly classify in this process are vulnerable road users (VRUs).
With the largely completed transition of automotive radar systems from the previously used 24 GHz band to the new 77 GHz band, the performance of automotive radar systems has improved further due to larger available bandwidths and reduced wavelengths. The reduced wavelengths allow for an increased angular resolution with the same aperture size while the increased bandwidth results in enhanced range resolution. However, all currently commercially available automotive radar systems are single-polarised and therefore do not consider an important source of information, namely the polarisation state of the backscattered radiation, which can benefit classification of objects greatly.
To gain better insight in the benefits of this factor, this thesis is concerned with investigating the effectiveness of polarisation information for classification of objects in the automotive scenario. In particular, two types of objects are considered, namely road surfaces and their conditions, as well as vulnerable road users.
Chapter 2 considers the scattering of electromagnetic waves from road surfaces. Measurements on multiple different road surface materials are performed and their electrical properties are determined. These measurement results are subsequently used to simulate scattering from road surfaces at mm-wave frequencies using numerical methods. Also, a new method for computing the range-Doppler signature of road surfaces in dynamic scenarios is proposed.
In chapter 3, to accompany the numerical results from the previous chapter, a method is developed to determine statistical polarimetric radar cross section models of road surfaces at mm-wave frequencies. Measurements are performed and the radar cross section models of several types of surfaces and road surface conditions are determined. These models are subsequently used to simulate polarimetric backscattering from road surfaces as well as to determine the optimal sensing wave polarisation to be used for single-polarised radar systems.
Chapter 4 considers the classification of road surfaces and road surface conditions using H, α, and A features. For the first time, these features are adapted for road surface classification purposes in the automotive scenario and it is shown that these features can benefit surface classification.
Chapter 5 shifts the attention from classification of road surface conditions to classification of vulnerable road users. In order to do this, a novel automotive polarimetric MIMO radar system with corresponding signal processing algorithms has been developed and is presented in this chapter. This radar is subsequently used for measurements of moving VRUs and several features are considered to classify them. It is demonstrated that polarimetric information is indeed able to increase classification perfomance of VRUs compared to single polarised radar.
This thesis demonstrates that polarimetric radar is useful in the automotive scenario besides the fields in which it has already been established such as synthetic aperture radar and meteorological radar applications. The methods proposed within this dissertation can be used to enhance classification accuracy of road surface condition which may lead to improvements in road safety by for example enhancing the performance of anti-lock breaking systems. Also, the methods proposed in this thesis to use polarimetric information for classifying vulnerable road users are shown to improve classification performance of this type of objects which could help enable autonomous driving in the future. ...
With the largely completed transition of automotive radar systems from the previously used 24 GHz band to the new 77 GHz band, the performance of automotive radar systems has improved further due to larger available bandwidths and reduced wavelengths. The reduced wavelengths allow for an increased angular resolution with the same aperture size while the increased bandwidth results in enhanced range resolution. However, all currently commercially available automotive radar systems are single-polarised and therefore do not consider an important source of information, namely the polarisation state of the backscattered radiation, which can benefit classification of objects greatly.
To gain better insight in the benefits of this factor, this thesis is concerned with investigating the effectiveness of polarisation information for classification of objects in the automotive scenario. In particular, two types of objects are considered, namely road surfaces and their conditions, as well as vulnerable road users.
Chapter 2 considers the scattering of electromagnetic waves from road surfaces. Measurements on multiple different road surface materials are performed and their electrical properties are determined. These measurement results are subsequently used to simulate scattering from road surfaces at mm-wave frequencies using numerical methods. Also, a new method for computing the range-Doppler signature of road surfaces in dynamic scenarios is proposed.
In chapter 3, to accompany the numerical results from the previous chapter, a method is developed to determine statistical polarimetric radar cross section models of road surfaces at mm-wave frequencies. Measurements are performed and the radar cross section models of several types of surfaces and road surface conditions are determined. These models are subsequently used to simulate polarimetric backscattering from road surfaces as well as to determine the optimal sensing wave polarisation to be used for single-polarised radar systems.
Chapter 4 considers the classification of road surfaces and road surface conditions using H, α, and A features. For the first time, these features are adapted for road surface classification purposes in the automotive scenario and it is shown that these features can benefit surface classification.
Chapter 5 shifts the attention from classification of road surface conditions to classification of vulnerable road users. In order to do this, a novel automotive polarimetric MIMO radar system with corresponding signal processing algorithms has been developed and is presented in this chapter. This radar is subsequently used for measurements of moving VRUs and several features are considered to classify them. It is demonstrated that polarimetric information is indeed able to increase classification perfomance of VRUs compared to single polarised radar.
This thesis demonstrates that polarimetric radar is useful in the automotive scenario besides the fields in which it has already been established such as synthetic aperture radar and meteorological radar applications. The methods proposed within this dissertation can be used to enhance classification accuracy of road surface condition which may lead to improvements in road safety by for example enhancing the performance of anti-lock breaking systems. Also, the methods proposed in this thesis to use polarimetric information for classifying vulnerable road users are shown to improve classification performance of this type of objects which could help enable autonomous driving in the future. ...
Due to recent advances in miniaturisation and cost reductions of radar systems, an increasing amount of vehicles is equipped with automotive radar systems to enable advanced driver assistance systems (ADAS) which improve driving comfort and road safety. Examples of these systems include automatic braking as well as adaptive cruise control. To advance the capabilities of these systems and to take another step towards enabling autonomous driving, the next generation of radar sensors aims to enhance the situational awareness of the driver and the vehicle by, amongst others, classifying objects in the surrounding area of the car. Arguably, the most important objects to correctly classify in this process are vulnerable road users (VRUs).
With the largely completed transition of automotive radar systems from the previously used 24 GHz band to the new 77 GHz band, the performance of automotive radar systems has improved further due to larger available bandwidths and reduced wavelengths. The reduced wavelengths allow for an increased angular resolution with the same aperture size while the increased bandwidth results in enhanced range resolution. However, all currently commercially available automotive radar systems are single-polarised and therefore do not consider an important source of information, namely the polarisation state of the backscattered radiation, which can benefit classification of objects greatly.
To gain better insight in the benefits of this factor, this thesis is concerned with investigating the effectiveness of polarisation information for classification of objects in the automotive scenario. In particular, two types of objects are considered, namely road surfaces and their conditions, as well as vulnerable road users.
Chapter 2 considers the scattering of electromagnetic waves from road surfaces. Measurements on multiple different road surface materials are performed and their electrical properties are determined. These measurement results are subsequently used to simulate scattering from road surfaces at mm-wave frequencies using numerical methods. Also, a new method for computing the range-Doppler signature of road surfaces in dynamic scenarios is proposed.
In chapter 3, to accompany the numerical results from the previous chapter, a method is developed to determine statistical polarimetric radar cross section models of road surfaces at mm-wave frequencies. Measurements are performed and the radar cross section models of several types of surfaces and road surface conditions are determined. These models are subsequently used to simulate polarimetric backscattering from road surfaces as well as to determine the optimal sensing wave polarisation to be used for single-polarised radar systems.
Chapter 4 considers the classification of road surfaces and road surface conditions using H, α, and A features. For the first time, these features are adapted for road surface classification purposes in the automotive scenario and it is shown that these features can benefit surface classification.
Chapter 5 shifts the attention from classification of road surface conditions to classification of vulnerable road users. In order to do this, a novel automotive polarimetric MIMO radar system with corresponding signal processing algorithms has been developed and is presented in this chapter. This radar is subsequently used for measurements of moving VRUs and several features are considered to classify them. It is demonstrated that polarimetric information is indeed able to increase classification perfomance of VRUs compared to single polarised radar.
This thesis demonstrates that polarimetric radar is useful in the automotive scenario besides the fields in which it has already been established such as synthetic aperture radar and meteorological radar applications. The methods proposed within this dissertation can be used to enhance classification accuracy of road surface condition which may lead to improvements in road safety by for example enhancing the performance of anti-lock breaking systems. Also, the methods proposed in this thesis to use polarimetric information for classifying vulnerable road users are shown to improve classification performance of this type of objects which could help enable autonomous driving in the future.
With the largely completed transition of automotive radar systems from the previously used 24 GHz band to the new 77 GHz band, the performance of automotive radar systems has improved further due to larger available bandwidths and reduced wavelengths. The reduced wavelengths allow for an increased angular resolution with the same aperture size while the increased bandwidth results in enhanced range resolution. However, all currently commercially available automotive radar systems are single-polarised and therefore do not consider an important source of information, namely the polarisation state of the backscattered radiation, which can benefit classification of objects greatly.
To gain better insight in the benefits of this factor, this thesis is concerned with investigating the effectiveness of polarisation information for classification of objects in the automotive scenario. In particular, two types of objects are considered, namely road surfaces and their conditions, as well as vulnerable road users.
Chapter 2 considers the scattering of electromagnetic waves from road surfaces. Measurements on multiple different road surface materials are performed and their electrical properties are determined. These measurement results are subsequently used to simulate scattering from road surfaces at mm-wave frequencies using numerical methods. Also, a new method for computing the range-Doppler signature of road surfaces in dynamic scenarios is proposed.
In chapter 3, to accompany the numerical results from the previous chapter, a method is developed to determine statistical polarimetric radar cross section models of road surfaces at mm-wave frequencies. Measurements are performed and the radar cross section models of several types of surfaces and road surface conditions are determined. These models are subsequently used to simulate polarimetric backscattering from road surfaces as well as to determine the optimal sensing wave polarisation to be used for single-polarised radar systems.
Chapter 4 considers the classification of road surfaces and road surface conditions using H, α, and A features. For the first time, these features are adapted for road surface classification purposes in the automotive scenario and it is shown that these features can benefit surface classification.
Chapter 5 shifts the attention from classification of road surface conditions to classification of vulnerable road users. In order to do this, a novel automotive polarimetric MIMO radar system with corresponding signal processing algorithms has been developed and is presented in this chapter. This radar is subsequently used for measurements of moving VRUs and several features are considered to classify them. It is demonstrated that polarimetric information is indeed able to increase classification perfomance of VRUs compared to single polarised radar.
This thesis demonstrates that polarimetric radar is useful in the automotive scenario besides the fields in which it has already been established such as synthetic aperture radar and meteorological radar applications. The methods proposed within this dissertation can be used to enhance classification accuracy of road surface condition which may lead to improvements in road safety by for example enhancing the performance of anti-lock breaking systems. Also, the methods proposed in this thesis to use polarimetric information for classifying vulnerable road users are shown to improve classification performance of this type of objects which could help enable autonomous driving in the future.
Doctoral thesis
(2024)
-
S. Yuan, Alexander Yarovoy , F. Fioranelli
Autonomous driving is one of the most popular research topics. Radar technology is used for many applications of ADAS and is considered one of the key technologies for HAD. It has unique advantages compared with other sensors, especially its capabilities during adverse weather conditions and Doppler information extraction. Required by autonomous applications, radar has to change its historical role from a simple detector to an imaging sensor, which requires not only the range and Doppler resolution ability but also a high spatial resolution, i.e., the azimuth and the elevation angle resolution. To address this problem, in this thesis, new signal processing algorithms are proposed, which pave the way to improved performance of the automotive radar sensor.
The FMCW waveform is widely used in current automotive applications due to its low cost and simplicity. MIMO array techniques exploit the spatial diversity of transmit and receive antenna arrays and have been exploited in current automotive radar because of their ability to achieve high angular resolution with a few antennas. Platform movement is one of themain characteristics of automotive radar, which introduces movement uncertainty compared with radars at fixed locations but provides an opportunity to use the movement to boost the angular resolution. Thus, FMCW waveform and MIMO antenna array are the main research subjects in this thesis..... ...
The FMCW waveform is widely used in current automotive applications due to its low cost and simplicity. MIMO array techniques exploit the spatial diversity of transmit and receive antenna arrays and have been exploited in current automotive radar because of their ability to achieve high angular resolution with a few antennas. Platform movement is one of themain characteristics of automotive radar, which introduces movement uncertainty compared with radars at fixed locations but provides an opportunity to use the movement to boost the angular resolution. Thus, FMCW waveform and MIMO antenna array are the main research subjects in this thesis..... ...
Autonomous driving is one of the most popular research topics. Radar technology is used for many applications of ADAS and is considered one of the key technologies for HAD. It has unique advantages compared with other sensors, especially its capabilities during adverse weather conditions and Doppler information extraction. Required by autonomous applications, radar has to change its historical role from a simple detector to an imaging sensor, which requires not only the range and Doppler resolution ability but also a high spatial resolution, i.e., the azimuth and the elevation angle resolution. To address this problem, in this thesis, new signal processing algorithms are proposed, which pave the way to improved performance of the automotive radar sensor.
The FMCW waveform is widely used in current automotive applications due to its low cost and simplicity. MIMO array techniques exploit the spatial diversity of transmit and receive antenna arrays and have been exploited in current automotive radar because of their ability to achieve high angular resolution with a few antennas. Platform movement is one of themain characteristics of automotive radar, which introduces movement uncertainty compared with radars at fixed locations but provides an opportunity to use the movement to boost the angular resolution. Thus, FMCW waveform and MIMO antenna array are the main research subjects in this thesis.....
The FMCW waveform is widely used in current automotive applications due to its low cost and simplicity. MIMO array techniques exploit the spatial diversity of transmit and receive antenna arrays and have been exploited in current automotive radar because of their ability to achieve high angular resolution with a few antennas. Platform movement is one of themain characteristics of automotive radar, which introduces movement uncertainty compared with radars at fixed locations but provides an opportunity to use the movement to boost the angular resolution. Thus, FMCW waveform and MIMO antenna array are the main research subjects in this thesis.....