Reinforcement learning for control

Performance, stability, and deep approximators

Review (2018)

Authors

Lucían Busoniu Technical University of Cluj-Napoca
T.D. de Bruin Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
Domagoj Tolić RIT Croatia
J. Kober Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
Ivana Palunko University of Dubrovnik
Research Group
Learning & Autonomous Control (Mechanical, Maritime and Materials Engineering) (TU Delft)
Copyright: © 2018 Lucian Buşoniu, T.D. de Bruin, Domagoj Tolić, J. Kober, Ivana Palunko
DOI: https://doi.org/10.1016/j.arcontrol.2018.09.005
Deep learning Stability Reinforcement learning Optimal control Adaptive dynamic programming Function approximation
More Info
expand_more

Published Date

2018

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Cognitive Robotics

Research Group

Learning & Autonomous Control

Abstract

Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that – among other things – points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.