A scenario-based testing approach can reduce the time required to obtain statistically significant evidence of the safety of Automated Driving Systems (ADS). Identifying these scenarios in an automated manner is a challenging task. Most methods on scenario classification do not work for complex scenarios with diverse environments (highways, urban) and interaction with other traffic agents. This is mirrored in their approaches which model an individual vehicle in relation to its environment, but neglect the interaction between multiple vehicles (e.g. cut-ins, stationary lead vehicle). Furthermore, existing datasets lack diversity and do not have per-frame annotations to accurately learn the start and end time of a scenario. We propose a method for complex traffic scenario classification that is able to model the interaction of a vehicle with the environment, as well as other agents. We use Graph Convolutional Networks to model spatial and temporal aspects of these scenarios. Expanding the nuScenes and Argoverse 2 driving datasets, we introduce a scenario-labeled dataset, which covers different driving environments and is annotated per frame. Training our method on this dataset, we present a promising baseline for future research on perframe complex scenario classification

This research introduces a novel approach for 3D object detection utilizing language models, with a particular focus on addressing the challenges that have been encountered in the autonomous vehicle domain. The primary objective revolves around addressing the constraints associated with object detection models that rely heavily on labeled data and are resource-intensive, while also showing limited proficiency in recognizing new, unseen objects. The thesis presents the PointGLIP model, a novel integration of zero-shot and few-shot learning methodologies, that utilizes language models to augment object detection in 3D point cloud data. PointGLIP utilizes GLIP encoders, which are known for their capability to combine textual and visual data. This methodology facilitates the transfer of pre-trained information from a 2D domain to a 3D domain by converting point clouds into depth maps. This conversion process enables object detection without the need for extensive or any prior 3D training. The model’s ability to generalize and transfer learning from 2D to 3D environments is demonstrated by experiments conducted utilizing the nuScenes dataset. The results revealed that the model shows poor performance in the context of both zero-shot and few-shot learning. This shows that the model struggles to perform detections with depth maps which indicates a struggle in the transfer of knowledge from 2D to 3D contexts. In 2D, the integration of descriptive text through language models provides a unique approach to contextual understanding, though the outcomes demonstrated that further refinements are necessary for consistent reliability.

Recent progress in multiple micro aerial vehicle (MAV) systems has demonstrated autonomous navigation in static environments. Yet, there are limited works regarding the autonomous navigation of multiple MAVs in dynamic and unknown environments. The challenge arises from the complexity of the motion planning problem, which requires the MAVs to coordinate with each other while avoiding dynamic obstacles. This thesis presents a novel risk-aware decentralized multi-agent motion planning framework to address this issue. For perception, we rely on a particle-based dynamic map, which utilizes particles to represent dynamic obstacles and predict future states of dynamic obstacles. Leveraging this map representation and the shared trajectory from other agents, we evaluate the future collision risk with dynamic obstacles and other agents in a coupled manner. During the planning phase, a risk-aware kino-dynamic A* algorithm tailored to the particle-based map representation is developed, ensuring dynamically feasible paths with risks under a given safe level. Subsequently, spatio-temporal safety corridors with maximum volume are optimized by inflating from path segments, taking map particles as constraints. These corridors act as constraints for the trajectory optimization problem, which is simplified to a convex optimization problem by using Bézier splines. The proposed method is thoroughly evaluated in simulation environments featuring various quantities and shapes of dynamic obstacles. Comparative results with state-of-the-art multi-agent planners that rely on precise obstacle observations demonstrate the efficiency and safety of the proposed method. Furthermore, the effectiveness of the proposed method is validated in a more realistic simulation environment with pedestrians, using depth cameras onboard for perception.

Behaviour trees (BTs) serve as a powerful hierarchical structure for task execution, simplifying complex tasks but posing challenges in their manual design. The automatic generation of BTs addresses this concern, yet often lacks robust failure recovery options. This study presents Failure Recovery with Ontologically Generated Behaviour Trees (FROGBT), a novel approach bridging this gap by integrating ontological reasoning into the process of automatically generating BTs. This integration establishes a profound link between an agent’s knowledge and its capabilities, offering contextual insights into the agent’s skills. FROGBT enhances skill representation for planning and recovery. The approach’s effectiveness is indicated by its efficiency compared to the state-of-the-art framework for skill-based control, SkiROS, in a similar task. It showcases generality, uniting diverse skills, developed by various engineers, for recurring tasks, and introduces innovative failure recovery strategies. FROGBT highlights ontological reasoning’s potential to enhance BT generation with context-awareness and reasoning abilities, paving the way for future research on failure recovery concepts in generated BTs.