Towards Learning from Implicit Human Reward

(Extended Abstract)

Abstract (2016)

Authors

G. Li Ocean University of China, Universiteit van Amsterdam

H. Dibeklioglu Pattern Recognition and Bioinformatics -

S Whiteson University of Oxford

H.S. Hung Pattern Recognition and Bioinformatics -

Research Group

Pattern Recognition and Bioinformatics () (TU Delft)

Reinforcement learning Human agent interaction

More Info

expand_more

To reference this document use:

http://resolver.tudelft.nl/uuid:cc113785-e689-4519-9d0c-dc3dfaf49acc

Published Date

2016

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Intelligent Systems

Research Group

Pattern Recognition and Bioinformatics

Abstract

The TAMER framework provides a way for agents to learn to solve tasks using human-generated rewards. Previous research showed that humans give copious feedback early in training but very sparsely thereafter and that an agent's competitive feedback --- informing the trainer about its performance relative to other trainers --- can greatly affect the trainer's engagement and the agent's learning. In this paper, we present the first large-scale study of TAMER, involving 561 subjects, which investigates the effect of the agent's competitive feedback in a new setting as well as the potential for learning from trainers' facial expressions. Our results show for the first time that a TAMER agent can successfully learn to play Infinite Mario, a challenging reinforcement-learning benchmark problem. In addition, our study supports prior results demonstrating the importance of bi-directional feedback and competitive elements in the training interface. Finally, our results shed light on the potential for using trainers' facial expressions as reward signals, as well as the role of age and gender in trainer behavior and agent performance.