Comparing Exploration Approaches in Deep Reinforcement Learning for Traffic Light Control

Comparing Exploration Approaches in Deep Reinforcement Learning for Traffic Light Control

Oren, Yaniv (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Intelligent Systems)

Starre, R.A.N. (mentor)
Oliehoek, F.A. (graduation committee)
Migut, M.A. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2020-06-22

Identifying the most efficient exploration approach for deep reinforcement learning in traffic light control is not a trivial task, and can be a critical step in the development of reinforcement learning solutions that can effectively reduce traffic congestion. It is common to use baseline dithering methods such as $\epsilon$-greedy. However, the value of more evolved exploration approaches in this setting has not yet been determined. This paper addresses this concern by comparing the performance of the popular deep Q-learning algorithm using one baseline and two state of the art exploration approaches, and their combination. Specifically, $\epsilon$-greedy is used as a baseline, and compared to the exploration approaches Bootstrapped DQN, randomized prior functions, and their combination. This is done in three different traffic scenarios, capturing different traffic profiles. The results obtained suggest that the higher the complexity of the traffic scenario, and the larger the size of the observation space of the agent, the larger the gain from efficient exploration. This is illustrated by the improved performance observed in the agents using efficient exploration and enjoying a larger observation space in the complex traffic scenarios.

Deep Reinforcement Learning
traffic light control
Exploration

http://resolver.tudelft.nl/uuid:a3858c40-466c-48d0-a23b-45dbdf1c57ec

Student theses

bachelor thesis

© 2020 Yaniv Oren