Hierarchical Reinforcement Learning for Model-Free Flight Control

A sample efficient tabular approach using Q(lambda)-learning and options in a traditional flight control structure

Master thesis (2019)

Authors

J.M. Hoogvliet Aerospace Engineering

Contributors

E. van Kampen (mentor)
Faculty
Aerospace Engineering, Aerospace Engineering
Copyright: © 2019 Jonathan Hoogvliet
More Info
expand_more

Published Date

04-11-2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Aerospace Engineering

Abstract

Reinforcement learning (RL) is a model-free adaptive approach to learn a non-linear control law for flight control. However, for flat-RL (FRL) the size of the search space grows exponentially with the number of states, resulting in low sample efficiency. This research aims to improve the efficiency with Hierarchical Reinforcement Learning (HRL). Performance in terms of the number of samples and the mean tracking error is evaluated on an altitude reference tracking task using a simulated F16 aircraft model. FRL is used as the baseline performance index. HRL is used to define a three-level learning structure, re-using an existing flight control structure. Finally, options is used with HRL to add temporal abstraction. It is shown that by re-using the flight control structure the learning process is made more sample efficient. Adding options further increases this efficiency, but does not lead to better tracking
performance.