Safe Reinforcement Learning for Automated Vehicles

Safe Reinforcement Learning for Automated Vehicles

Cornet, R. (TU Delft Mechanical, Maritime and Materials Engineering; TU Delft Cognitive Robotics)

Pan, W. (mentor)
Wisse, M. (graduation committee)
Shyrokau, B. (graduation committee)
Zheng, Y. (graduation committee)

Delft University of Technology

Mechanical Engineering | Vehicle Engineering

2020-08-27

Fully automated vehicles have the potential to increase road safety and improve traffic flow by taking the human element out of the driving loop. They can also provide mobility to people who are unable to operate a conventional vehicle. Safe automated vehicles must be able to respond in emergency situations or drive on slippery roads in bad weather conditions. Therefore it is crucial to have a safe and robust control strategy that can use the full handling capabilities of the vehicle.

This thesis presents how safe reinforcement learning can be used to design a steering policy that can drive an automated vehicle at the limit of friction.

The steering policies are trained using the Lyapunov Safe Actor-Critic (LSAC) algorithm. LSAC is a combination of the Soft Actor-Critic (SAC) algorithm and a Lyapunov stability analysis to solve constrained control problems.

The performance of LSAC is tested in a vehicle simulator against SAC and Model Predictive Control (MPC) in a series of tests that include changing lanes at different speeds, recovering from a destabilizing collision, and driving on a race track at the limit of friction.

The experiments show that LSAC outperforms MPC and SAC control strategies in terms of safety and vehicle stability. LSAC can recover from larger disturbances than MPC and SAC. A control strategy is presented that will keep the vehicle stable when driving at the limit of friction but can use the maneuverability of an unstable vehicle when is it necessary to avoid dangerous situations. Additionally, a policy is presented that can find the fastest way around a race track while staying within the track limits.

safe reinforcement learning
Reinforcement Learning
Automated Vehicles
Automated driving

http://resolver.tudelft.nl/uuid:7bedb60a-ced8-4fcf-97ca-80208861a413

Student theses

master thesis

© 2020 R. Cornet