This article presents the first subject-specific head pose estimation approach using only one frequency-modulated continuous wave radar data frame. Specifically, the proposed method incorporates a deep learning framework to estimate head pose rotation and orientation frame-by-frame by combining a convolutional neural network operating on range-angle radar plots and a PeakConv network. The proposed method is validated with an in-house collected dataset, including annotated head movements that varied in roll, pitch, and yaw, and these were recorded in two different indoor environments. It is shown that the proposed model can estimate head poses with a relatively small error of approximately 6.7°–14.4° for all rotational axes and is capable of generalizing to unseen, new environments when trained in one scenario (e.g., lab) and tested in another (e.g., office), including in the cabin of a car.

Classification of human activities performed sequentially and with unconstrained durations using radar sensors has been studied in this work. A novel processing pipeline comprising a sequence segmentation stage, a segment processing stage, and a classification stage has been proposed to address this challenge. Specifically, the segmentation stage has been implemented by monitoring Rényi entropy for fluctuations in the radar data, with the entropy, derived from micro-Doppler spectrograms, functioning as a descriptive quantity of the activity being performed. The method has been experimentally verified on a challenging, publicly available dataset collected with a network of five simultaneously operating pulsed ultrawideband radars. Classification performance has been compared to reference works in the literature on the same dataset, and a test accuracy and macro F1-score of 89.3% and 82.0% have been, respectively, demonstrated.

The role of radar for building situation awareness in (semi)autonomous vehicles is severely restricted by its low angular resolution. The physical size of the radar, which determines its antenna aperture size and thus the radar angular resolution, is often a subject of stringent limitations to physically fit the system in the vehicles. Multiple input multiple output systems are used to increase the achievable angular resolution, and these are often combined in the literature with algorithms inspired by synthetic aperture radar techniques that exploit the velocity of the vehicle itself for finer resolution. Some of the most common approaches are reviewed, in this context, with a specific focus on challenges for the implementation of data collected in real driving scenarios. Key experimental results using representative algorithms and driving data collected in the city of Delft, The Netherlands, are presented and discussed.

The problem of radar-based tracking of groups of people moving together and counting their numbers in indoor environments is considered here. A novel processing pipeline to track groups of people moving together and count their numbers is proposed and validated. The pipeline is specifically designed to deal with frequent changes of direction and stop-and-go movements typical of indoor activities. The proposed approach combines a tracker with a classifier to count the number of grouped people; this uses both spatial features extracted from range-azimuth (RA) maps and Doppler frequency features extracted with wavelet decomposition. Thus, the pipeline outputs over time both the location and the number of people present. The proposed approach is verified with experimental data collected with a 24-GHz frequency-modulated continuous-wave (FMCW) radar. It is shown that the proposed method achieves 93.15% accuracy in terms of counting the number of people and a tracking metric optimal subpattern assignment (OSPA) of 0.335. Furthermore, the performance is analyzed as a function of different relevant variables such as feature combinations and scenarios.

Monitoring gait symmetry reliably is crucial, as it is an early indicator of Parkinson's disease (PD). In this work, a method is presented to analyze gait asymmetries using a 24-GHz frequency-modulated continuous-wave (FMCW) multiple-input-multiple-output (MIMO) radar in a nonclinical environment. The proposed method is validated by analyzing the gait of 60 people recorded in an environment that presents multiple challenges such as multipath, gait interruption events, and different trajectories, which cause aspect angle variability. This article presents algorithms to address these challenges to extract useful gait information from the radar data and discusses the performance of the proposed system, analyzing the ratio of correct feet identifications and the accuracy of the asymmetry gait parameters. It achieved a mean absolute error of the symmetry ratio below 8% for all gait parameters. This shows that the proposed system and algorithms are robust for both clinical and in-home implementations.

In this paper, an automatic labelling process is presented for automotive datasets, leveraging on complementary information from LiDAR and camera. The generated labels are then used as ground truth with the corresponding 4D radar data as inputs to a proposed semantic segmentation network, to associate a class label to each spatial voxel. Promising results are shown by applying both approaches to the publicly shared RaDelft dataset, with the proposed network achieving over 65% of the LiDAR detection performance, improving 13.2% in vehicle detection probability, and reducing 0.54 m in terms of Chamfer distance, compared to variants inspired from the literature.

Traditional target tracking using monostatic radar systems typically rely on centralized or decentralized architectures, where all data is transmitted to a fusion center for estimating the position and velocity of mobile agents. This approach introduces a single point of failure and can significantly increase communication costs, particularly when the fusion center is far from individual radar nodes. To overcome these issues, we introduce a distributed Alternating Direction Method of Multipliers (ADMM) for target localization using a radar network, wherein each radar node shares its observed data only with its immediate neighboring nodes, and achieves consensus with the radar network on the estimated target locations and velocities. We perform simulations incorporating critical system parameters such as the number of radar nodes and Signal-to-Noise Ratio (SNR) to assess their impact of estimation accuracy and convergence speed of the proposed distributed ADMM algorithm. We highlight the additional benefits of our proposed solution, and present directions for future work.

In this article, the classification of dynamic vulnerable road users (VRUs) using polarimetric automotive radar is considered. To this end, a signal processing pipeline for polarimetric automotive MIMO radar is proposed, including a method to enhance angular resolution by combining data from all polarimetric channels. The proposed signal processing pipeline is applied to measurement data of three different types of VRUs and a car, collected with a custom automotive polarimetric radar, developed in collaboration with Huber+Suhner AG. Several polarimetric features are estimated from the range-velocity signatures of the measured targets and are subsequently analyzed. A Bayesian classifier and a convolutional neural network (CNN) using these estimated polarimetric features are proposed and their performance is compared against their single-polarized counterparts. It is found that for the Bayesian classifier, a significant increase in classification performance is achieved, compared to the same classifier using single polarized information. For the CNN-based classifier, utilizing the distribution of polarimetric features of the target’s range-velocity signatures also increases classification performance, compared to its single-polarized version. This shows that polarimetric information is valuable for classification of VRUs and objects of interest in automotive radar.

In this paper, the problem of formulating effective processing pipelines for indoor human tracking is investigated, with the usage of a Multiple Input Multiple Output (MIMO) Frequency Modulated Continuous Wave (FMCW) radar. Specifically, two processing pipelines starting with detections on the Range-Azimuth (RA) maps and the Range-Doppler (RD) maps are formulated and compared, together with subsequent clustering and tracking algorithms and their relevant parameters. Experimental results are presented to validate and assess both pipelines, using a 24 GHz commercial radar platform with 250 MHz bandwidth and 15 virtual channels. Scenarios where 1 and 2 people move in an indoor environment are considered, and the influence of the number of virtual channels and detectors' parameters is discussed. The characteristics and limitations of both pipelines are presented, with the approach based on detections on RA maps showing in general more robust results.

The effect of different time-frequency (TF) resolution values is analyzed in the context of Human Activity Recognition (HAR) using multiple radars distributed in a network. Specifically, different spectrograms computed with various Short-Time Fourier Transform (STFT) window lengths and Morse wavelet transform are compared as input representation to a Convolutional Neural Network (CNN), together with a coherent combination of multiple spectrograms. The study emphasizes the importance of selecting appropriate window sizes for TF analysis and for classification, balancing the observation time with the physical duration of the diverse activities, and also avoiding correlation between different data samples that may compromise the generalization ability of the method. The results employing this coherent sensor fusion demonstrate the efficacy of the investigated method, achieving an F1 score of 0.943 on a challenging public dataset containing 9 activities performed by 15 participants.

In this paper, we address the limitations of traditional constant false alarm rate (CFAR) target detectors in automotive radars, particularly in complex urban environments with multiple objects that appear as extended targets. We propose a data-driven radar target detector exploiting a highly efficient 2D CNN backbone inspired by the computer vision domain. Our approach is distinguished by a unique cross-sensor supervision pipeline, enabling it to learn exclusively from unlabeled synchronized radar and lidar data, thus^eliminating the need for costly manual object annotations. Using a novel large-scale, real-life multi-sensor dataset recorded in various driving scenarios, we demonstrate that the proposed detector generates dense, lidar-like point clouds, achieving a lower Chamfer distance to the reference lidar point clouds than CFAR detectors. Overall, it significantly outperforms CFAR baselines detection accuracy.

In this study, the problem of multipath in radar sensor networks for human activity recognition (HAR) has been examined. Traditionally considered as a source of additional clutter, the multipath is being investigated for its potential to be exploited through the creation of virtual radar nodes. These virtual nodes are conceptualized to observe targets from aspect angles that differ from those of physically existing radars. To realize this idea, an innovative processing pipeline is proposed that extracts information from multipath signals to improve HAR. The pipeline isolates and tracks the line-of-sight (LOS) and multipath components of a moving human target performing continuous sequences of activities observed by a network of three radar sensors. Furthermore, the method has been verified with experimental data consisting of six activities and 14 volunteers by comparing classification metrics with the use of a single radar as well as only the LOS components of the three radars in the network. A 12-layer convolutional neural network (CNN) classifier has been designed to operate on range-Doppler (RD) images derived from the LOS and multipath components, extracted by the proposed method. A substantial performance improvement using the leave-one-person-out (L1Po) test set is demonstrated in the order of +11% by exploiting a multiradar network with its LOS and multipath components.

Mutual interference between automotive frequency-modulated continuous-wave (FMCW) radar systems has been a concern over recent years. Several interference mitigation (IM) techniques have been proposed to mitigate this phenomenon, which is deemed to grow in severity as more systems are deployed on the road. In this article, an inexpensive technique, based on well-known moving target indicator (MTI) processing, is proposed to separate interference from target signals. It exploits the contrast after stretch processing between uncorrelated FMCW interference (sparse and chirp-like) and beat signals (stable sinusoids). The interference is therefore marked and subtracted. This eliminates interference at the cost of introducing a distortion dependent on the relative radial velocity of the targets. To validate the proposed approach, called MTI-IM, numerical simulations and experiments with commercial-grade radars have been performed, with comparisons between MTI-IM and other IM techniques. Results show good capabilities to fight the uncorrelated interference coming from more than one interfering radar. This is achieved at reduced computational cost, which is a key limiting factor in automotive systems.

The problem of human activity classification using a distributed network of radar sensors has been considered. A novel sensor fusion method has been proposed that processes data from a network of radar sensors and yields 3-D representations of both reflection intensity and velocity distribution. The formulated method has been verified in an experimental case study, where activity classification was performed using data collected with 14 participants moving in diverse, unconstrained trajectories and executing nine activities. The classification performance of the proposed method has been compared to alternative fusion methods on the same dataset, and a test accuracy and macro $F1$ -score of, respectively, 87.4% and 81.9% have been demonstrated. A feasibility study has also been performed to demonstrate the ability of the proposed method to generate 3-D distributions of intensity and target velocity.

The problem of detection and localization of multiple people using a network of Ultra-Wide Band (UWB) radar nodes in the cabin of a vehicle is addressed in this paper. Specifically, an algorithm for decentralized vital signs detection is proposed, based on the analysis of a novel model for radar signatures of vital signs. Additionally, a centralized association processing block is used to fuse the detections from all radars in the network using a dedicated cost-matrix computation. The performance of the proposed processing pipeline is tested experimentally with a multistatic radar network, and a simulation framework is also developed to evaluate the performance in alternative topologies of the radar network beyond the experiments. It is shown that the detection and localization of humans within the car environment is possible with the proposed processing pipeline, with localization Root Mean Square Error.

The application of distributed radar to human motion monitoring is considered. A novel sensor fusion method has been proposed that yields a two-dimensional map of reflection intensity and a vector field of reconstructed velocities in lieu of conventional Doppler spectrograms or radial velocity components. The method has been verified using experimental datasets in two case studies involving fall detection in sequences of activities, and arm motion discrimination for in-place activities. A true positive rate and precision of respectively 99.3 % and 93.0 % have been demonstrated for the fall detection task, and the output of the proposed method for arm motion characterisation indicates suitability for classification in future research.

Fall detection systems can play an important role in assuring safe independent living for vulnerable people. These sensors not only have to detect falls but also have to recognize uncritical, normal activities of daily living in order to differentiate them from falls. Radar sensors are very attractive for human activity recognition thanks to their contactless capabilities and lack of plain videos recorded. In this article, a novel approach to recognize single activities in a continuous stream of radar data is proposed, whereby the stream is divided into windows of fixed length and, then, multilabel classification is used to recognize all activities taking place in these time segments. While the initial feasibility of this approach was presented in an earlier contribution presented at the 2023 IEEE SENSORS conference, in this extended work, additional in-depth studies on critical parameters are performed. Specifically, multiple combinations of different radar data domains/representations (e.g., range-time maps, range-Doppler maps, and spectrograms) and different radar nodes in a network of five cooperating sensors are considered as inputs to two considered multilabel classification networks. In addition, a parametric study on the probability thresholds of the networks to assign labels to specific classes is also performed.

In this paper, an algorithm to generate a sparse linear antenna array for Direction of Arrival (DoA) estimation that works well in combination with Bayesian Compressive Sensing (BCS) is proposed. The proposed algorithms rely on the provided information inherent to BCS, i.e., the entropy of the recovered estimation vector, to place new sensor antenna elements in an initially empty array, so that the most additional information is gathered about the observed scene. It is shown by means of simulation and radar measurements that BCS methods for DoA estimation using sparse sensor arrays provide promising results in terms of detection probability and estimation accuracy. Furthermore, the proposed algorithms are able to generate sparse sensor arrangements which provide an improved performance when compared against randomly generated sparse arrays.

A neural network (NN) based multi-frame classification approach is proposed to solve the problem of classification of tracked objects. Initially, a baseline tracker is implemented that uses the classification output of an object detection network for classification. Afterwards, two approaches for multi-frame classification are applied to perform classification of tracked objects. The first approach aggregates points from multiple frames and applies a single frame NN for classification, whereas the second approach uses bidirectional long short term memory (BiLSTM) layers to process points from multiple frames. Extensive experiments on the opensource 2D RadarScenes dataset showed a consistent increase in track performance when using either of the two techniques for multi-frame classification.

The problem of 3D ego-motion velocity estimation using multichannel Frequency Modulated Continuous Wave (FM CW) radar sensors has been studied. Special attention is given to presence of moving targets in the scene. These targets are first distinguished by the difference between the measured Doppler, and the Doppler calculated with an initial rough estimation of the vehicle ego-velocity. Then, an iterative algorithm is proposed to reduce the influence of the moving targets in the ego-motion estimation procedure, thus improving the overall accuracy. The performance of the proposed algorithm is compared with state-of-the-art alternatives based on simulated data, and superior performance has been demonstrated.