Event-Based Communication in Distributed Q-Learning

Conference Paper (2022)
Author(s)

D. Jarne Ornia (TU Delft - Team Manuel Mazo Jr)

M Mazo Jr. (TU Delft - Team Manuel Mazo Jr)

Research Group
Team Manuel Mazo Jr
Copyright
© 2022 D. Jarne Ornia, M. Mazo
DOI related publication
https://doi.org/10.1109/CDC51059.2022.9992660
More Info
expand_more
Publication Year
2022
Language
English
Copyright
© 2022 D. Jarne Ornia, M. Mazo
Research Group
Team Manuel Mazo Jr
Pages (from-to)
2379-2386
ISBN (print)
978-1-6654-6761-2
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

We present an approach to reduce the communication of information needed on a Distributed Q-Learning system inspired by Event Triggered Control (ETC) techniques. We consider a baseline scenario of a Distributed Q-Learning problem on a Markov Decision Process (MDP). Following an event-based approach, N agents sharing a value function explore the MDP and compute a trajectory-dependent triggering signal which they use distributedly to decide when to communicate information to a central learner in charge of computing updates on the action-value function. These decision functions form an Event Based distributed Q learning system (EBd-Q), and we derive convergence guarantees resulting from the reduction of communication. We then apply the proposed algorithm to a cooperative path planning problem, and show how the agents are able to learn optimal trajectories communicating a fraction of the information. Additionally, we discuss what effects (desired and undesired) these event-based approaches have on the learning processes studied, and how they can be applied to more complex multi-agent systems.

Files

Event_Based_Communication_in_D... (pdf)
(pdf | 1.26 Mb)
- Embargo expired in 10-07-2023
License info not available