Continuous Improvement of Driving Automation
Using Safety Performance Indicators and Hazardous Scenario Identification
M.M. Selva Kumar (TU Delft - Electrical Engineering, Mathematics and Computer Science)
RR Venkatesha Prasad – Mentor (TU Delft - Networked Systems)
Burcu Kulahcioglu Ozkan – Graduation committee member (TU Delft - Software Engineering)
Andrei Terechko – Mentor (NXP Semiconductors)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
The rapid advancement of automated vehicles (AVs) can potentially improve transportation. However, ensuring the safety and reliability of Automated Driving Systems (ADS) remains a critical challenge, particularly when facing the expansion of Operational Design Domains (ODDs) and the continuous emergence of unknown hazardous scenarios. This thesis aims to address these challenges by developing a framework for monitoring the safety of multi-channel ADS and identifying hazardous scenarios using Safety Performance Indicators (SPIs) and Hazardous Scenario Identification (HSI) techniques.
The proposed SPI framework, based on the principles outlined in the UL 4600 standard, encompasses a comprehensive set of metrics for assessing the safety and performance of ADS. These metrics cover various critical functionalities, such as ego localization, object detection, trajectory planning, and overall ADS behaviour. By defining appropriate thresholds for each SPI, the framework enables the identification of potential safety issues and supports the continuous monitoring and improvement of ADS.
The HSI module, developed as part of this thesis, leverages the SPI framework and the NXP Daruma cross-channel analysis to detect hazardous scenarios. The HSI module's performance is evaluated using the CARLA simulator and advanced ADS software stacks (LAV and TFUSE) across diverse driving scenarios. The results demonstrate the HSI module's effectiveness in identifying hazardous scenarios such as ego vehicle tailgating, inconsistent ego localization, and ego vehicle being tailgated. However, our analysis also reveals challenges in terms of false positives and negatives, highlighting the need for further improvements in the ADS's perception and localization functionalities and in tuning the SPI thresholds appropriately based on testing as well as the characteristics of the ADS.
This thesis contributes to advancing ADS safety by developing a comprehensive SPI framework and implementing a proof of concept HSI module. We propose an architecture that integrates these components in a closed-loop process involving vehicle fleet data collection, cloud-based analysis, and targeted software updates. This framework enables the identification of areas for improvement and supports generating OpenSCENARIO files for reproducing and analyzing hazardous scenarios ad hoc. The findings from the experimental evaluation provide valuable insights into the performance and limitations of the SPI safety monitoring and HSI techniques, guiding the safe deployment and continuous improvement of ADS. This research ultimately paves the way for the widespread adoption of automated vehicles (AVs) in driving environments.