Analysis of the impact of traffic density on training of reinforcement learning based conflict resolution methods for drones

Analysis of the impact of traffic density on training of reinforcement learning based conflict resolution methods for drones

Groot, D.J. (TU Delft Control & Simulation)
Ellerbroek, Joost (TU Delft Control & Simulation)
Hoekstra, J.M. (TU Delft Control & Simulation)

2024

Conventional Air Traffic Control is still predominantly being done by human Air Traffic Controllers, however, as the traffic density increases, the workload of the controllers increases as well. Especially for the area of unmanned aviation, driven by the rise in drones, having human controllers might become unfeasible. One of the methods that is currently being investigated for replacing the conflict resolution task of Air Traffic Control is Reinforcement Learning. As violation of the required separation margins, also called an intrusion, is an event of relatively low frequency, using Reinforcement Learning for this task comes with difficulties that can potentially be attributed to data imbalance. This paper artificially increased the traffic density during the training phase of the Reinforcement Learning method to investigate what the importance is of a balanced data set on the performance of the Reinforcement Learning method. It was found that as the traffic density increased, the Reinforcement Learning methods started to outperform the analytical methods. Beyond this it was found that methods trained at higher traffic densities, but tested at lower traffic densities, outperformed the methods trained at that specific density. This indicates that it might be better to always ensure that the training scenarios are more complex than anticipated during the execution phase, even if that results in unrealistic scenarios.

BlueSky simulator
Conflict resolution
Deep reinforcement learning
High-traffic density
Simulation
Soft actor critic
Unmanned traffic management

http://resolver.tudelft.nl/uuid:c4997b49-e52b-4d5c-a7be-c3a26d941b72

https://doi.org/10.1016/j.engappai.2024.108066

0952-1976

Engineering Applications of Artificial Intelligence, 133

Institutional Repository

journal article

© 2024 D.J. Groot, Joost Ellerbroek, J.M. Hoekstra