Title
Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers
Author
Dong, Y. (TU Delft Transport and Planning)
Datema, T. (TU Delft Electrical Engineering, Mathematics and Computer Science)
Wassenaar, V. (TU Delft Electrical Engineering, Mathematics and Computer Science)
van de Weg, J.J. (TU Delft Electrical Engineering, Mathematics and Computer Science)
Kopar, C.T. (TU Delft Mechanical Engineering)
Suleman, H.I. (TU Delft Electrical Engineering, Mathematics and Computer Science)
Faculty
Electrical Engineering, Mathematics and Computer Science
Date
2023
Abstract
Developing and testing automated driving models in the real world might be challenging and even dangerous, while simulation can help with this, especially for challenging maneuvers. Deep reinforcement learning (DRL) has the potential to tackle complex decision-making and controlling tasks through learning and interacting with the environment, thus it is suitable for developing automated driving while not being explored in detail yet. This study carried out a comprehensive study by implementing, evaluating, and comparing the two DRL algorithms, Deep Q-networks (DQN) and Trust Region Policy Optimization (TRPO), for training automated driving on the highway-env simulation platform. Effective and customized reward functions were developed and the implemented algorithms were evaluated in terms of onlane accuracy (how well the car drives on the road within the lane), efficiency (how fast the car drives), safety (how likely the car is to crash into obstacles), and comfort (how much the car makes jerks, e.g., suddenly accelerates or brakes). Results show that the TRPO-based models with modified reward functions delivered the best performance in most cases. Furthermore, to train a uniform driving model that can tackle various driving maneuvers besides the specific ones, this study expanded the highway-env and developed an extra customized training environment, namely, ComplexRoads, integrating various driving maneuvers and multiple road scenarios together. Models trained on the designed ComplexRoads environment can adapt well to other driving maneuvers with promising overall performance. Lastly, several functionalities were added to the highway-env to implement this work. The codes are open on GitHub at https://github.com/alaineman/drlcarsim-paper.
To reference this document use:
http://resolver.tudelft.nl/uuid:7262e4d4-2415-4c39-a326-e0488b2b01d5
DOI
https://doi.org/10.1109/ITSC57777.2023.10422159
Publisher
IEEE
Embargo date
2024-03-28
ISBN
9798350399462
Source
2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)
Event
26th IEEE International Conference on Intelligent Transportation Systems, ITSC 2023, 2023-09-24 → 2023-09-28, Euskalduna Conference Centre, Bilbao, Spain
Series
IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2153-0009
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2023 Y. Dong, T. Datema, V. Wassenaar, J.J. van de Weg, C.T. Kopar, H.I. Suleman