Reinforcement Learning for Quadcopter Fault Tolerance

Abstract

The demand of adding fault tolerance to quadcopter control systems has significantly increased with the rise of adoption of UAVs in numerous sectors. This work proposes and demonstrates the use of Hierarchical Reinforcement Learning to control a quadcopter subject to severe actuator fault. State-of-the-art algorithms are implemented, and a control architecture is designed with the aim of addressing sample efficiency, robustness and overall performance. The controller is able to learn the correct control policy in around 500 cumulative episodes, using a sampling frequency of 100 Hz and two neural networks of two 16 neuron layers each. Online learning is employed to provide robustness to changes in the environment and physical characteristics of the drone. As a result, vertical speed tracking improves in terms of settling time when the drone's mass is more than doubled. However, the usage of online learning results to be unpractical, due to the unwanted but necessary addition of noise to the controller output.