The Sim2Real gap poses significant challenges for testing autonomous vehicles, often becoming apparent only during high-risk real-world deployments. This research proposes a novel pipeline that leverages both high-fidelity (CARLA) and low-fidelity (Gym-Duckietown) simulators to estimate this gap prior to deployment. The results reveal a strong corelation between performance in Gym Duckietown and real-world outcomes, suggesting it can serve as potential estimation for real world performance and the Sim2Real gap. Nonetheless, real-world testing remains an essential part of the validation process. Future work should build on these findings to further explore and validate the approach.

Testing Deep Reinforcement Learning agents for safety and performance failures is critical but computationally expensive, requiring efficient methods to discover failure-inducing scenarios. Indago, a state-of-the-art testing framework, addresses this by using a Multi-Layer Perceptron (MLP) surrogate model to guide a search algorithm towards potential failures. However, the MLP considers only static initial environment configurations. We propose replacing the static MLP with a Long Short-Term Memory (LSTM) network to leverage the temporal data from agent interaction sequences during training. We evaluated our LSTM-based surrogate against the original MLP within the Indago framework on the HighwayEnv autonomous driving benchmark. Although the best LSTM exhibited lower predictive precision on a held-out test set (11.5% vs. 24%), its integration into the search-based testing pipeline led to a higher failure rate and input diversity. The LSTM guided search discovered more failures on average (34.6% vs. 30%) and explored the input space more thoroughly, achieving 99% cluster coverage and a tenfold increase in input entropy compared to the MLP. Interestingly, training on very short sequences of only two timesteps produced the most effective models, a finding that challenges the initial hypothesis that longer temporal contexts are beneficial for improved performance.

Deep Reinforcement Learning (DRL) is a powerful framework for training autonomous agents in complex environments. However, testing these agents is still prohibitively expensive due to the need for extensive simulations and the rarity of failure events, such as collisions or timeouts, where the agent fails to complete its task safely or correctly. Existing surrogate models, such as Multi-Layer Perceptrons (MLPs), are a promising improvement by predicting failures without requiring full simulation runs. However, prior research has focused almost exclusively on MLPs, leaving it unclear whether other, more expressive machine learning models could improve performance. In this paper, we investigate whether Bayesian Neural Networks (BNNs), which incorporate probabilistic reasoning into neural architectures, can be more effective surrogates for failure prediction in DRL environments. We developed, trained, and evaluated a BNN surrogate and compared it against a pre-trained MLP baseline, using the HighwayEnv car parking environment as our test case. Our evaluation focused on comparing the predictive accuracy, precision, recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) using training data, as well as assessing the models' effectiveness in the DRL parking environment. The results show that the BNN surrogate outperforms the MLP baseline in terms of practical utility for failure discovery. These findings suggest that BNNs can be a more effective surrogate model for prioritising failure scenarios in DRL testing.

In recent years, Deep Reinforcement Learning (DRL) has moved away from playing games to more practical tasks like autonomous parking. This transition has created a need for efficient testing of DRL agents. To evaluate an agent, we need to run a simulation of the task and let the agent decide what actions to perform. Running these simulations is expensive and time-consuming. The problem of testing is that we need hundreds of different test scenarios, and each simulation takes from seconds to minutes depending on the use case, which is why choosing the right initial state of the simulation is critical. Current solutions leverage the idea of surrogate models that can approximate how difficult it will be for an agent to complete a given environment without running the tests. Existing work has explored the use of surrogate models for DRL tasks, creating a Multi-Layer Perceptron to act as a surrogate model for a DRL agent attempting to park a car in a parking lot. Parking scenarios are inherently spatial problems, and MLPs are not able to take advantage of that spatial structure. To address this limitation, we used Convolutional Neural Networks (CNNs), which are designed to handle spatial information more effectively, and should therefore improve prediction performance over MLPs. Our approach transforms tabular data into a low-resolution grid-like image representing a parking lot. This approach guides a genetic algorithm to discover a mean of 25.76 failures per run, a 72% improvement over the 14.98 failures found by the MLP baseline. Our model also achieves significantly higher scores on diversity metrics of generated environments.

Testing Deep Reinforcement Learning (DRL) agents is computationally expensive and inefficient, especially when trying to identify environment configurations where the agent fails to reach its objective. Recent work proposes the use of a Multi-Layer-Perceptron (MLP) as a surrogate model to predict whether a given environment configuration is likely to be a failing environment without running the tests. However, this raises the question of whether different surrogate models can perform this task better and whether we can improve the training of these models by fine-tuning their hyperparameters. In this work, we seek to understand how XGBoost performs as an alternative to the MLP for predicting DRL failures, together with grid search as a hyperparameter optimization technique. We evaluated our approach on the Parking environment from the HighwayEnv Simulator using a DRL agent trained using Hindsight Experience Replay (HER), where a failing environment is one in which the vehicle collides with a parked vehicle or a timeout occurs. This evaluation is based on how well the model can classify failing environments and how effective it can guide a Genetic Algorithm (GA) to find failing environments (failure search). We compared the performance of XGBoost with the MLP and found that XGBoost significantly outperforms the MLP baseline across key classification metrics such as F1-score and AUC-ROC. Furthermore, during failure search, the XGBoost-guided GA yields more failing environments with greater coverage and entropy, indicating increased diversity and effectiveness. These findings suggest that XGBoost is a strong candidate for surrogate modelling in DRL testing and offers a more reliable alternative to an MLP-based approach.

Machine Learning (ML) systems are increasingly used in high-stakes, socially impactful domains, requiring attention to improve explainability and trust. However, current Requirements Engineering (RE) techniques often fail to address these human-centered qualities. This research investigates how Behavior-Driven Development (BDD) and Goal-Oriented Requirements Engineering (GORE) can improve the identification and visualization of requirements for explainability and trust. We conducted expert interviews and a survey to see how different stakeholders rate certain BDD scenarios and what they think of conceptual GORE models. Our results show that participants value concise, human-readable BDD scenarios and particularly like the GORE framework of i* to understand stakeholder relationships and system behaviors. The other GORE framework, GR4ML, was found to align more with business goals and addresses other stakeholder perspectives less. We conclude that BDD and GORE can improve explainability in ML system development. Future work should refine modeling tools to better integrate ethical and fairness considerations.