Continuous state and action Q-learning framework applied to quadrotor UAV control

Continuous state and action Q-learning framework applied to quadrotor UAV control

Naruta, Anton (TU Delft Aerospace Engineering)

van Kampen, Erik-jan (mentor)

Delft University of Technology

2017-09-08

This paper describes an implementation of a reinforcement learning-based framework applied to the control of a multi-copter rotorcraft. The controller is based on continuous state and action Q-learning. The policy is stored using a radial basis function neural network. Distance-based neuron activation is used to optimize the generalization algorithm for computational performance. The training proceeds off-line, using a reduced-order model of the controlled system. The model is identified and stored in the form of a neural network. The framework incorporates a dynamics inversion controller, based on the identified model. Simulated flight tests confirm the controller's ability to track the reference state signal and outperform a conventional proportional-derivative(PD) controller. The contributions of the developed framework are a computationally-efficient method to store a $\mathcal{Q}$-function generalization, continuous action selection based on local $\mathcal{Q}$-function approximation and a combination of model identification and offline learning for inner-loop control of a UAV system.

Reinforcement learning
Q-Learning
quadcopter
Neural Networks

http://resolver.tudelft.nl/uuid:d7fb9b06-a75e-46df-b324-015f22521bf0

Student theses

master thesis

© 2017 Anton Naruta