Influence-Based Abstraction in Deep Reinforcement Learning
Miguel Suau de Castro (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Elena Congeduti (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Rolf Starre (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Aleksander Czechowski (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Frans Oliehoek (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large systems into small regions. We explore this method in the context of deep reinforcement learning, showing that by keeping track of a small set of variables in the history of previous actions and observations we can learn policies that can effectively control a local region in the global system.