Safe Multi-agent Learning via Trapping Regions

Safe Multi-agent Learning via Trapping Regions

Czechowski, A.T. (TU Delft Interactive Intelligence)
Oliehoek, F.A. (TU Delft Interactive Intelligence)

Elkind, Edith (editor)

2023

One of the main challenges of multi-agent learning lies in establishing convergence of the algorithms, as, in general, a collection of individual, self-serving agents is not guaranteed to converge with their joint policy, when learning concurrently. This is in stark contrast to most single-agent environments, and sets a prohibitive barrier for deployment in practical applications, as it induces uncertainty in long term behavior of the system. In this work, we apply the concept of trapping regions, known from qualitative theory of dynamical systems, to create safety sets in the joint strategy space for decentralized learning. We propose a binary partitioning algorithm for verification that candidate sets form trapping regions in systems with known learning dynamics, and a heuristic sampling algorithm for scenarios where learning dynamics are not known. We demonstrate the applications to a regularized version of Dirac Generative Adversarial Network, a four-intersection traffic control scenario run in a state of the art open-source microscopic traffic simulator SUMO, and a mathematical model of economic competition.

http://resolver.tudelft.nl/uuid:c4839d9c-b915-4598-8e01-fd1b35c3f08a

International Joint Conferences on Artificial Intelligence (IJCAI)

9781956792034

Proceedings of the 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023

32nd International Joint Conference on Artificial Intelligence, IJCAI 2023, 2023-08-19 → 2023-08-25, Macao, China

IJCAI International Joint Conference on Artificial Intelligence, 1045-0823, 2023-August

Institutional Repository

conference paper

© 2023 A.T. Czechowski, F.A. Oliehoek