Online Reinforcement Learning for Flight Control

An Adaptive Critic Design without prior model knowledge

Master thesis (2019)

Authors

D. Kroezen Aerospace Engineering

Contributors

E. van Kampen (supervisor 1)
G.C.H.E. de Croon (supervisor 2)
M.A. Mitici Air Transport & Operations - Aerospace Engineering (supervisor 2)
W. Pan Robot Dynamics - Mechanical, Maritime and Materials Engineering (supervisor 2)
Faculty
Aerospace Engineering
Copyright: © 2019 Dave Kroezen
Machine Learning Reinforcement Learning (RL) Online Learning Adaptive Control Adaptive Critic Designs Flight Control Systems Adaptive Flight Control
More Info
expand_more

Published Date

10-04-2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Aerospace Engineering

Abstract

Online Reinforcement Learning is a possible solution for adaptive nonlinear flight control. In this research an Adaptive Critic Design (ACD) based on Dual Heuristic Dynamic Programming (DHP) is developed and implemented on a simulated Cessna Citation 550 aircraft. Using an online identified system model approximation, the method is independent of prior model knowledge. The agent consists of two Artificial Neural Networks (ANNs) which form the Adaptive Critic Design and is supplemented with a Recursive Least Squares (RLS) online model estimation. The implemented agent is demonstrated to learn a near optimal control policy for different operating points, which is capable of tracking pitch and roll rate while actively minimizing the sideslip angle in a faster than real-time simulation. Providing limited model knowledge is shown to increase the learning, performance and robustness of the controller.