Automated lane changing using deep reinforcement learning: a user-acceptance case study

Abstract

Lane change decision-making is an important challenge for automated vehicles, urging the need for high performance algorithms that are able to handle complex traffic situations. Deep reinforcement learning (DRL), a machine learning method based on artificial neural networks, has recently become a popular choice for modelling the lane change decision-making process, outperforming various traditional rule-based models. So far, performance has often been expressed in terms of achieved average speed, absence of collisions or merging success rate. However, no studies have investigated how humans will react to the resulting behavior as potential occupants. This study addresses this research gap by validating a self-developed DRL-based lane changing model (trained using proximal policy optimization) from a technology acceptance perspective through an online crowdsourcing experiment. Participants (N=1085) viewed a random subset of 32 out of 120 videos of an automated vehicle driving on a three-lane highway with varying traffic densities featuring our proposed model or a baseline policy (i.e. a state-of-the-art rule-based model, MOBIL). They were tasked to press a response key if the decision-making was deemed undesirable and subsequently rated the vehicle's behavior along four acceptance constructs (performance expectancy, safety, human-likeness and reliability) on a scale of 1 to 5. Results showed that the proposed model caused a significantly lower amount of disagreements and was rated significantly higher on all four acceptance constructs compared to the baseline policy. Moreover, considerable differences between individual disagreement rates were observed for both models. Our findings offer prospects for the practical application of DRL-based lane change models in a use-case scenario, depending on the user. Further research is necessary to examine whether these observations hold in other (more complex) traffic situations. Additionally, we recommend combining DRL with other modelling techniques that allow for personalization of behavioral parameters, such as imitation learning.