Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework

Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework

Van Vliet, B.
Caarls, W.
Schuitema, E.
Jonker, P.P.

Mechanical, Maritime and Materials Engineering

Biomechanical Engineering

2011-11-03

Reinforcement learning is a way to learn control tasks by trial and error. Even for simple motor control tasks, however, this can take a long time. We can speed up learning by using prior knowledge, but this is not always available, especially for an autonomous agent. One way to add limited prior knowledge is to use subgoals, defining points that the controller should aim for on the way to reaching the real goal. In this study, we use the MAXQ hierarchical framework to specify subgoals. This decreased the learning time by a factor two on a robot leg step-up task and we show that tests on a real robot give similar results. The worse end performance that is a result of the reduced solution space can be partially canceled out by hierarchical greedy execution. To our knowledge, this is the first time the MAXQ framework is applied to a real robot.

http://resolver.tudelft.nl/uuid:eaf340c0-5798-43ab-bc9d-e27c6dcd4e94

BNAIC 2011: 23rd Benelux Conference on Artificial Intelligence, Ghent, Belgium, 3-4 November 2011

Institutional Repository

conference paper

(c) 2011 The Author(s)