Robust Event-Driven Interactions in Cooperative Multi-agent Learning

Conference paper (2022)

Authors

D. Jarne Ornia Team Manuel Mazo Jr - Mechanical, Maritime and Materials Engineering
M. Mazo Team Manuel Mazo Jr - Mechanical, Maritime and Materials Engineering
Research Group
Team Manuel Mazo Jr (Mechanical, Maritime and Materials Engineering) (TU Delft)
Copyright: © 2022 D. Jarne Ornia, M. Mazo
DOI
help_outline
https://doi.org/10.1007/978-3-031-15839-1_16
Reinforcement Learning Multi-Agent Systems Event-Triggered Communication
More Info
expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Delft Center for Systems and Control

Research Group

Team Manuel Mazo Jr

Abstract

We present an approach to safely reduce the communication required between agents in a Multi-Agent Reinforcement Learning system by exploiting the inherent robustness of the underlying Markov Decision Process. We compute robustness certificate functions (off-line), that give agents a conservative indication of how far their state measurements can deviate before they need to update other agents in the system with new measurements. This results in fully distributed decision functions, enabling agents to decide when it is necessary to communicate state variables. We derive bounds on the optimality of the resulting systems in terms of the discounted sum of rewards obtained, and show these bounds are a function of the design parameters. Additionally, we extend the results for the case where the robustness surrogate functions are learned from data, and present experimental results demonstrating a significant reduction in communication events between agents.