A. Palffy
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
6 records found
1
Radar-based perception has been gaining traction in recent years, supported by improvements in deep learning techniques. Low-level radar perception focuses on utilizing the denser radar signal data instead of the conventional point-cloud. Despite the recent focus on this data representation, a lack of public datasets has limited the scope of research, especially for scene segmentation. In this paper, we address this challenge by recording a novel low-level radar dataset that includes diverse environments, sensors and complex scenarios. We propose Swin-FFM, a Swin transformer based network for free-road segmentation using the complex-valued range-Doppler signal. On our dataset, Swin-FFM achieves an IoU of 86.8%, demonstrating its ability to successfully output accurate free-road boundaries even in challenging settings. In addition to this, we compare our network with baselines for both low-level and point-cloud formats. Finally, we demonstrate the network’s ability to work with any low-level radar representation, highlighting its benefit for low-level radar perception.
...
Radar-based perception has been gaining traction in recent years, supported by improvements in deep learning techniques. Low-level radar perception focuses on utilizing the denser radar signal data instead of the conventional point-cloud. Despite the recent focus on this data representation, a lack of public datasets has limited the scope of research, especially for scene segmentation. In this paper, we address this challenge by recording a novel low-level radar dataset that includes diverse environments, sensors and complex scenarios. We propose Swin-FFM, a Swin transformer based network for free-road segmentation using the complex-valued range-Doppler signal. On our dataset, Swin-FFM achieves an IoU of 86.8%, demonstrating its ability to successfully output accurate free-road boundaries even in challenging settings. In addition to this, we compare our network with baselines for both low-level and point-cloud formats. Finally, we demonstrate the network’s ability to work with any low-level radar representation, highlighting its benefit for low-level radar perception.
Radar (Radio Detection and Ranging) sensors are cost-efficient and robust under adverse weather conditions, making them an attractive component in modern automated driving perception systems, but they provide significantly sparser information about the environment than camera or LiDAR sensors. Thus, to fully exploit radars in perception solutions, it is crucial to exploit not only local but also global contextual information of the scene. However, existing 4D radar models often struggle to fully exploit both types of information, resulting in suboptimal performance. This paper proposes DRIFT, a dual-representation model that effectively captures and fuses both local and global contexts through a dual-path architecture. The model incorporates a point path to aggregate fine-grained local features and a pillar path to encode coarse-grained global features. These two parallel paths are inter-fused via novel feature-sharing layers at multiple stages, enabling full utilization of both representations. DRIFT is evaluated on the widely used View-of-Delft (VoD) dataset and an internal dataset, demonstrating its state-of-the-art performance across multiple tasks, including object detection and free-road segmentation. Notably, DRIFT achieves a mean average precision (mAP) of 52.6% (compared to 45.4% from the CenterPoint baseline) on the VoD dataset, surpassing existing methods.
...
Radar (Radio Detection and Ranging) sensors are cost-efficient and robust under adverse weather conditions, making them an attractive component in modern automated driving perception systems, but they provide significantly sparser information about the environment than camera or LiDAR sensors. Thus, to fully exploit radars in perception solutions, it is crucial to exploit not only local but also global contextual information of the scene. However, existing 4D radar models often struggle to fully exploit both types of information, resulting in suboptimal performance. This paper proposes DRIFT, a dual-representation model that effectively captures and fuses both local and global contexts through a dual-path architecture. The model incorporates a point path to aggregate fine-grained local features and a pillar path to encode coarse-grained global features. These two parallel paths are inter-fused via novel feature-sharing layers at multiple stages, enabling full utilization of both representations. DRIFT is evaluated on the widely used View-of-Delft (VoD) dataset and an internal dataset, demonstrating its state-of-the-art performance across multiple tasks, including object detection and free-road segmentation. Notably, DRIFT achieves a mean average precision (mAP) of 52.6% (compared to 45.4% from the CenterPoint baseline) on the VoD dataset, surpassing existing methods.
Accurate sensor calibration is a critical challenge in the development of automated vehicles, especially in dynamic and modular sensor configurations. Traditional target-based methods, while precise, are limited in scalability and adaptability. In this work, we propose a modular, targetless, ego-motion-based calibration framework for multi-modal sensors, including a monocular camera, LiDAR, and 4D radar. The framework leverages odometry trajectories for extrinsic calibration, incorporating temporal alignment, trajectory scaling, and both pairwise and joint optimization techniques to achieve robust and accurate sensor alignment. Experimental validation using the View-of-Delft (VoD) dataset demonstrates the framework’s
robustness across diverse sensor setups, adaptability to real-world conditions. Our results underscore the potential of scalable, targetless calibration approaches to enhance the reliability and flexibility of automated systems, supporting implementation in real-world scenarios. ...
robustness across diverse sensor setups, adaptability to real-world conditions. Our results underscore the potential of scalable, targetless calibration approaches to enhance the reliability and flexibility of automated systems, supporting implementation in real-world scenarios. ...
Accurate sensor calibration is a critical challenge in the development of automated vehicles, especially in dynamic and modular sensor configurations. Traditional target-based methods, while precise, are limited in scalability and adaptability. In this work, we propose a modular, targetless, ego-motion-based calibration framework for multi-modal sensors, including a monocular camera, LiDAR, and 4D radar. The framework leverages odometry trajectories for extrinsic calibration, incorporating temporal alignment, trajectory scaling, and both pairwise and joint optimization techniques to achieve robust and accurate sensor alignment. Experimental validation using the View-of-Delft (VoD) dataset demonstrates the framework’s
robustness across diverse sensor setups, adaptability to real-world conditions. Our results underscore the potential of scalable, targetless calibration approaches to enhance the reliability and flexibility of automated systems, supporting implementation in real-world scenarios.
robustness across diverse sensor setups, adaptability to real-world conditions. Our results underscore the potential of scalable, targetless calibration approaches to enhance the reliability and flexibility of automated systems, supporting implementation in real-world scenarios.
3D semantic understanding is essential for a wide range of robotics applications. Availability of datasets is a strong driver for research, and whilst obtaining unlabeled data is straightforward, manually annotating this data with semantic labels is time-consuming and costly. Recently, foundation models have facilitated open-set semantic segmentation, potentially aiding automatic labeling. However, these models have largely been limited to 2D images. This work introduces Label Any Pointcloud (LeAP), which leverages 2D Vision Foundation Models (VFMs) to automatically label 3D data with any set of classes in any kind of application. VFMs are used to create image labels for the desired classes which are then projected to 3D points. Using the Bayesian update, point-wise labels are combined into voxels to improve label consistency, and label points outside the camera fustrum. A novel Cross-Modal Self-Training (CM-ST) approach further enhances label quality. Through extensive experiments, we demonstrate that our method can generate high-quality 3D semantic labels across diverse fields without any manual 3D labeling. Models adapted to new application domains using our labels show up to 3.7× (12.9 → 47.1) mIoU improvement compared to the unadapted baselines. This ability to provide labels for any domain can help accelerate 3D perception research.
...
3D semantic understanding is essential for a wide range of robotics applications. Availability of datasets is a strong driver for research, and whilst obtaining unlabeled data is straightforward, manually annotating this data with semantic labels is time-consuming and costly. Recently, foundation models have facilitated open-set semantic segmentation, potentially aiding automatic labeling. However, these models have largely been limited to 2D images. This work introduces Label Any Pointcloud (LeAP), which leverages 2D Vision Foundation Models (VFMs) to automatically label 3D data with any set of classes in any kind of application. VFMs are used to create image labels for the desired classes which are then projected to 3D points. Using the Bayesian update, point-wise labels are combined into voxels to improve label consistency, and label points outside the camera fustrum. A novel Cross-Modal Self-Training (CM-ST) approach further enhances label quality. Through extensive experiments, we demonstrate that our method can generate high-quality 3D semantic labels across diverse fields without any manual 3D labeling. Models adapted to new application domains using our labels show up to 3.7× (12.9 → 47.1) mIoU improvement compared to the unadapted baselines. This ability to provide labels for any domain can help accelerate 3D perception research.
Automotive radars are increasingly used in automated driving systems due to their cost effectiveness, ease of integration, and ability to withstand adverse weather conditions. Semantic segmentation for radar point clouds is a crucial step in radar pre-processing, which can be used on almost all downstream tasks of radars, such as detection and tracking. Radar point clouds are noisier compared to LiDAR point clouds due to sensor noise and the multi-path propagation, which makes the segmentation task for radar more challenging. In this paper, we address the problem of segmentation in noisy radar point clouds in terms of ghost target vs. real detection, moving vs. static objects as well as semantic segmentation of moving road users. We demonstrate how these three tasks can be performed in a single, unified pipeline using an auto-labeled radar dataset. Our approach, called Real, Moving, and Semantic Segmentation Network (RMSNet), is able to output point-wise labels for all three tasks simultaneously. On our dataset, RMSNet attains an IoU of 82.5% for real detection segmentation, 66.9% IoU for moving object segmentation, and 64.9% mIoU for semantic segmentation. We also did a live demonstration, using the model on two different radars. The result shows that the model is capable of running inference in real-time and demonstrates good ability in model generalization.
...
Automotive radars are increasingly used in automated driving systems due to their cost effectiveness, ease of integration, and ability to withstand adverse weather conditions. Semantic segmentation for radar point clouds is a crucial step in radar pre-processing, which can be used on almost all downstream tasks of radars, such as detection and tracking. Radar point clouds are noisier compared to LiDAR point clouds due to sensor noise and the multi-path propagation, which makes the segmentation task for radar more challenging. In this paper, we address the problem of segmentation in noisy radar point clouds in terms of ghost target vs. real detection, moving vs. static objects as well as semantic segmentation of moving road users. We demonstrate how these three tasks can be performed in a single, unified pipeline using an auto-labeled radar dataset. Our approach, called Real, Moving, and Semantic Segmentation Network (RMSNet), is able to output point-wise labels for all three tasks simultaneously. On our dataset, RMSNet attains an IoU of 82.5% for real detection segmentation, 66.9% IoU for moving object segmentation, and 64.9% mIoU for semantic segmentation. We also did a live demonstration, using the model on two different radars. The result shows that the model is capable of running inference in real-time and demonstrates good ability in model generalization.
Autonomous driving systems require robust and reliable perception across diverse environmental conditions, yet current approaches to 3D semantic occupancy prediction face challenges in adverse weather and lighting. In this paper, we present the first study on fusing 4D radar and camera for 3D semantic occupancy prediction. This fusion offers significant advantages for robust and accurate perception as 4D radar provides reliable range, velocity, and angle information, even in challening conditions, complementing rich semantic and texture details from cameras. Additionally, we demonstrate that incorporating depth cues with camera image pixels supports the process of lifting 2D images to 3D, enhancing the accuracy of scene reconstruction. Secondly, we introduce a fully automatically labeled dataset specifically designed for training semantic occupancy models, demonstrating its ability to substantially reduce the need for costly manual annotation. Our results highlight the robustness of 4D radar in a wide range of challenging scenarios, showcasing its potential to advance perception for autonomous vehicles.
...
Autonomous driving systems require robust and reliable perception across diverse environmental conditions, yet current approaches to 3D semantic occupancy prediction face challenges in adverse weather and lighting. In this paper, we present the first study on fusing 4D radar and camera for 3D semantic occupancy prediction. This fusion offers significant advantages for robust and accurate perception as 4D radar provides reliable range, velocity, and angle information, even in challening conditions, complementing rich semantic and texture details from cameras. Additionally, we demonstrate that incorporating depth cues with camera image pixels supports the process of lifting 2D images to 3D, enhancing the accuracy of scene reconstruction. Secondly, we introduce a fully automatically labeled dataset specifically designed for training semantic occupancy models, demonstrating its ability to substantially reduce the need for costly manual annotation. Our results highlight the robustness of 4D radar in a wide range of challenging scenarios, showcasing its potential to advance perception for autonomous vehicles.