Automotive perception involves understanding the external driving environment and the internal state of the vehicle cabin and occupants using sensor data. It is critical to achieving high levels of safety and autonomy in driving. This article provides an overview of different sensor modalities, such as cameras, radars, and light detection and ranging (LiDAR) used commonly for perception, along with the associated data processing techniques. Critical aspects of perception are considered, such as architectures for processing data from single or multiple sensor modalities, sensor data processing algorithms and the role of machine learning techniques, methodologies for validating the performance of perception systems, and safety. The technical challenges for each aspect are analyzed, emphasizing machine learning approaches, given their potential impact on improving perception. Finally, future research opportunities in automotive perception for their wider deployment are outlined. @en

In this paper, we discuss an "open radar initiative" aimed at promoting the sharing of radar datasets and a common framework for acquiring data. The framework is based on widely available and affordable short-range radar hardware (automotive FMCW radar transceivers). This framework and initiative are intended to create and promote access to a common shared dataset for the development and benchmarking of algorithms. While this is the norm for image processing and speech processing research, there has been reluctance in the radar community so far to create common datasets of shared data, often due to justified intellectual property or security classification reasons. Notable exceptions do exist, such as the MSTAR dataset of SAR images, which enabled great progress across the radar research community for a number of years. With this initiative, we hope to stimulate discussion and, with time, changes of practice in the radar research community. The main contribution of this work relative to previously shared datasets of radar data is that the proposed framework consists of a complete, integrated and replicable hardware and software pipeline, allowing users to not only download existing data, but also to acquire their own data with a compatible format that allows expansion and enrichment of the common dataset.@en

Deep neural networks have become increasingly popular in radar micro-Doppler classification; yet, a key challenge, which has limited potential gains, is the lack of large amounts of measured data that can facilitate the design of deeper networks with greater robustness and performance. Several approaches have been proposed in the literature to address this problem, such as unsupervised pre-training and transfer learning from optical imagery or synthetic RF data. This work investigates an alternative approach to training which involves exploitation of “datasets of opportunity” - micro-Doppler datasets collected using other RF sensors, which may be of a different frequency, bandwidth or waveform - for the purposes of training. Specifically, this work compares in detail the cross-frequency training degradation incurred for several different training approaches and deep neural network (DNN) architectures. Results show a 70% drop in classification accuracy when the RF sensors for pre-training, fine-tuning, and testing are different, and a 15% degradation when only the pre-training data is different, but the fine-tuning and test data are from the same sensor. By using generative adversarial networks (GANs), a large amount of synthetic data is generated for pre-training. Results show that cross-frequency performance degradation is reduced by 50% when kinematically-sifted GAN-synthesized signatures are used in pre-training.@en