Reinforcement Learning for Quadcopter Fault Tolerance

Reinforcement Learning for Quadcopter Fault Tolerance: Analyzing Hierarchical and Online Deep Reinforcement Learning for Quadcopter Fault Tolerant Control

Bello, Riccardo (TU Delft Aerospace Engineering; TU Delft Control & Simulation)

van Kampen, E. (mentor)

Delft University of Technology

Aerospace Engineering

2021-09-17

The demand of adding fault tolerance to quadcopter control systems has significantly increased with the rise of adoption of UAVs in numerous sectors. This work proposes and demonstrates the use of Hierarchical Reinforcement Learning to control a quadcopter subject to severe actuator fault. State-of-the-art algorithms are implemented, and a control architecture is designed with the aim of addressing sample efficiency, robustness and overall performance. The controller is able to learn the correct control policy in around 500 cumulative episodes, using a sampling frequency of 100 Hz and two neural networks of two 16 neuron layers each. Online learning is employed to provide robustness to changes in the environment and physical characteristics of the drone. As a result, vertical speed tracking improves in terms of settling time when the drone's mass is more than doubled. However, the usage of online learning results to be unpractical, due to the unwanted but necessary addition of noise to the controller output.

Reinforcement Learning
Machine Learning
Artificial Intelligence
Online Learning
Fault Tolerant Control
Quadcopter

http://resolver.tudelft.nl/uuid:b4a32863-810a-4a0f-8737-54b5a7912ec5

Student theses

master thesis

© 2021 Riccardo Bello