Interactive Learning with Corrective Feedback for Policies Based on Deep Neural Networks

Conference Paper (2020)
Author(s)

Rodrigo Perez-Dattari (Universidad de Chile)

Carlos Celemin (TU Delft - Learning & Autonomous Control)

Javier Ruiz-Del-Solar (Universidad de Chile)

J. Kober (TU Delft - Learning & Autonomous Control)

Research Group
Learning & Autonomous Control
Copyright
© 2020 R.J. Perez Dattari, Carlos Celemin, Javier Ruiz-del-Solar, J. Kober
DOI related publication
https://doi.org/10.1007/978-3-030-33950-0_31
More Info
expand_more
Publication Year
2020
Language
English
Copyright
© 2020 R.J. Perez Dattari, Carlos Celemin, Javier Ruiz-del-Solar, J. Kober
Research Group
Learning & Autonomous Control
Bibliographical Note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.@en
Pages (from-to)
353-363
ISBN (print)
978-3-030-33949-4
ISBN (electronic)
978-3-030-33950-0
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) strategy for training DNN policies based on human corrective feedback, with a method called Deep COACH (D-COACH). This approach not only takes advantage of the knowledge and insights of human teachers as well as the power of DNNs, but also has no need of a reward function (which sometimes implies the need of external perception for computing rewards). We combine Deep Learning with the COrrective Advice Communicated by Humans (COACH) framework, in which non-expert humans shape policies by correcting the agent’s actions during execution. The D-COACH framework has the potential to solve complex problems without much data or time required. Experimental results validated the efficiency of the framework in three different problems (two simulated, one with a real robot), with state spaces of low and high dimensions, showing the capacity to successfully learn policies for continuous action spaces like in the Car Racing and Cart-Pole problems faster than with DRL.

Files

Out.pdf
(pdf | 1.18 Mb)
- Embargo expired in 23-07-2020
License info not available