Longitudinal Flight Control by Reinforcement Learning

Longitudinal Flight Control by Reinforcement Learning: Online Adaptive Critic Design Approach to Altitude Control

Lee, Jun (TU Delft Aerospace Engineering)

van Kampen, Erik-jan (mentor)

Delft University of Technology

Aerospace Engineering

2019-12-10

Reinforcement learning is used as a type of adaptive flight control. Adaptive Critic Design (ACD) is a popular approach for online reinforcement learning control due to its explicit generalization of the policy evaluation and the policy improvement elements. A variant of ACD, Incremental Dual Heuristic Programming (IDHP) has previously been developed that allows fully online adaptive control by online identification of system and control matrices. Previous implementation attempts to a high fidelity Cessna Citation model have shown accurate simultaneous altitude and roll angle reference tracking results with outer loop PID and inner loop IDHP rate controllers after an online training phase. This paper presents an implementation attempt to achieve full IDHP altitude control under the influence of measurement noise and atmospheric gusts. Two IDHP controller designs are proposed with and without the cascaded actor structure. Simulation results with measurement noise indicate that the IDHP controller design without the cascaded actor structure can achieve high success ratios. It is demonstrated that IDHP altitude control under measurement noise and atmospheric gusts are achievable under four flight conditions.

Reinforcement Learning
Adaptive Critic Designs
Flight Control Systems

http://resolver.tudelft.nl/uuid:c1201f27-964c-4257-ad65-89224bef94a1

Student theses

master thesis

© 2019 Jun Lee