Hierarchical Reinforcement Learning for Model-Free Flight Control

Hierarchical Reinforcement Learning for Model-Free Flight Control: A sample efficient tabular approach using Q(lambda)-learning and options in a traditional flight control structure

Hoogvliet, Jonathan (TU Delft Aerospace Engineering)

van Kampen, Erik-jan (mentor)

Delft University of Technology

Aerospace Engineering

2019-11-04

Reinforcement learning (RL) is a model-free adaptive approach to learn a non-linear control law for flight control. However, for flat-RL (FRL) the size of the search space grows exponentially with the number of states, resulting in low sample efficiency. This research aims to improve the efficiency with Hierarchical Reinforcement Learning (HRL). Performance in terms of the number of samples and the mean tracking error is evaluated on an altitude reference tracking task using a simulated F16 aircraft model. FRL is used as the baseline performance index. HRL is used to define a three-level learning structure, re-using an existing flight control structure. Finally, options is used with HRL to add temporal abstraction. It is shown that by re-using the flight control structure the learning process is made more sample efficient. Adding options further increases this efficiency, but does not lead to better tracking
performance.

Hierarchical Reinforcement Learning
Flight Control Systems
Reinforcement Learning
Control
Simulation
Aircraft
sample efficiency
flat reinforcement learning

http://resolver.tudelft.nl/uuid:d66efdb7-d7c7-4c44-9b50-64678ffdf60d

Student theses

master thesis

© 2019 Jonathan Hoogvliet