Autonomous driving is attracting growing attention because of the potential advantages it poses on safety, leisure, energy efficiency, reduced emissions, and traffic reduction. Current research is focusing on areas related to artificial intelligence to solve complex planning and decision-making tasks, object detection, or simultaneous localization and mapping. However, training and testing these methods in the real world is rather unsafe and can be harmful to other road users. For that reason, the focus is set on developing such methods first in a simulated and controlled environment and then transferring the control strategy to the target domain. The issue with this approach is that the performance drops when the domain changes, a problem known as sim2real. In addition to this, certain methods cannot be deployed to real-time applications as the computational complexity of the techniques is too large to produce outcomes in a constrained time frame. To date, research on reducing the sim2real for autonomous driving gap has focused on domain adaptation or domain randomization techniques, however, none has combined them with a high-fidelity vehicle dynamics model. This work aims to validate a safe transfer learning approach that reduces the reality gap in a zero-shot manner by combining the advantages of using simulated and real-world data, a digital twin of the vehicle, and a traffic scenario simulator. To test this framework reinforcement learning agents are trained to track different paths. Further evaluation includes the influence of using a high-fidelity model and real-world data. Furthermore, the effect of the reward function on the control strategy of the learning agents is also examined. The training process of this transfer learning framework starts with virtually generated scenarios, which are simpler than their real-life counterparts and noise-free. The motion of the learning agent is simulated with the digital twin and in every episode, its parameters are randomized. Moreover, noise is added to the control action. These randomizations are useful for the controller to operate under uncertainty and avoid overfitting to model inaccuracies. Once the performance saturates, real-world logged data is included so that the learning agent adapts to the target distribution, i.e. on noise levels and driving style. After the performance stops improving, the results are tested in Model-in-the-Loop and Vehicle-in-the-Loop with the SimRod, an all-electric drive-by-wire vehicle. The results of this study indicate that using this zero-shot transfer learning framework yields much better tracking accuracy (about 30% on average) in the physical world with respect to controllers trained only with lower-fidelity models or synthetically generated data. This study stresses that a combination of the three sim2real methods and synthetically generated and real-world data are necessary to reduce the reality gap. Additional results include the influence of the reward function design and using different reinforcement learning algorithms (proximal policy optimization and soft actor-critic).

In recent years, imitation learning (IL) has been widely used in industry as the core of autonomous vehicle (AV) planning modules. However, previous work on IL planners shows sample inefficiency and low generalisation in safety-critical scenarios, on which they are rarely tested. As a result, IL planners can reach a performance plateau where adding more training data ceases to improve the learnt policy. First, our work presents an IL model using spline coefficient parameterisation and offline expert queries to enhance safety and training efficiency. Then, we expose the weakness of the learnt IL policy by synthetically generating critical scenarios through optimisation of parameters of the driver's risk field (DRF), a parametric human driving behaviour model implemented in a multi-agent traffic simulator based on the Lyft Prediction Dataset. To continuously improve the learnt policy, we retrain the IL model with augmented data. Thanks to the expressivity and interpretability of the DRF, the desired driving behaviours can be encoded and aggregated to the original training data. Our work constitutes a full development cycle that can efficiently and continuously improve the learnt IL policies in closed-loop. Finally, we show that our IL planner developed with 30 times less training resource still has superior performance compared to the previous state-of-the-art.