Uncertainty-Aware Reinforcement Learning for Flight Control

Uncertainty-Aware Reinforcement Learning for Flight Control: Mastering the Mystery of Flight

Homola, Marek (TU Delft Aerospace Engineering)

van Kampen, E. (mentor)
Li, Y. (mentor)

Delft University of Technology

Aerospace Engineering

2024-01-19

In the rapidly evolving aviation sector, the quest for safer and more efficient flight operations has historically relied on traditional Automatic Flight Control Systems (AFCS) based on high-fidelity models. However, such models not only incur high development costs but also struggle to adapt to new, complex aircraft designs and unexpected operational conditions. As an alternative, deep Reinforcement Learning (RL) has emerged as a promising solution for model-free, adaptive flight control. Yet, RL-based approaches pose significant challenges in terms of sample efficiency and safety assurance. Addressing these gaps, this paper introduces Returns Uncertainty-Navigated Distributional Soft Actor-Critic (RUN-DSAC). Designed to enhance the learning efficiency, adaptability, and safety of flight control systems, RUN-DSAC leverages the rich uncertainty information inherent in the returns distribution to refine the decision-making process. When applied to the attitude tracking task on a high-fidelity non-linear fixed-wing aircraft model, RUN-DSAC demonstrates superior performance in learning efficiency, adaptability to varied and unforeseen flight scenarios, and robustness in fault tolerance that outperforms the current state-of-the-art SAC and DSAC algorithms.

Autonomous Flight Control
Deep Distributional Reinforcement Learning
Uncertainty-Driven Control
Safety-Critical Systems
Sample Efficient Machine Learning
Reinforcement Learning
Autonomous System

http://resolver.tudelft.nl/uuid:1553d8c9-79df-44dd-8c9f-d2eb9f7017b8

Public code repository: https://github.com/mhomola/RUN-DSAC

Student theses

master thesis

© 2024 Marek Homola