Influence-Augmented Local Simulators
a Scalable Solution for Fast Deep RL in Large Networked Systems
Miguel Suau (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Jinke He (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Matthijs T.J. Spaan (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Frans A. Oliehoek (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.