Multi-agent reinforcement learning via distributed MPC as a function approximator

Journal article (2024)

Authors

S.H. Mallick Team Bart De Schutter - Mechanical, Maritime and Materials Engineering
F. Airaldi Team Azita Dabiri - Mechanical, Maritime and Materials Engineering
A. Dabiri Team Azita Dabiri - Mechanical, Maritime and Materials Engineering
B.H.K. De Schutter Delft Center for Systems and Control
Research Group
Team Azita Dabiri (Mechanical, Maritime and Materials Engineering) (TU Delft)
Multi-agent reinforcement learning Networked systems ADMM Distributed model predictive control
More Info
expand_more

Published Date

2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Delft Center for Systems and Control

Research Group

Team Azita Dabiri

Abstract

This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on a numerical example.