DACOOP-A

Decentralized Adaptive Cooperative Pursuit via Attention

Journal article (2024)

Authors

Zheng Zhang Sun Yat-sen University
Dengyu Zhang Sun Yat-sen University
Qingrui Zhang Sun Yat-sen University
W. Pan Robot Dynamics - Mechanical, Maritime and Materials Engineering , The University of Manchester
Tianjiang Hu Sun Yat-sen University
Research Group
Robot Dynamics (Mechanical, Maritime and Materials Engineering) (TU Delft)
Reinforcement learning Multi-robot systems Attention mechanism Cooperative pursuit
More Info
expand_more

Published Date

2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Cognitive Robotics

Research Group

Robot Dynamics

Abstract

Integrating rule-based policies into reinforcement learning promises to improve data efficiency and generalization in cooperative pursuit problems. However, most implementations do not properly distinguish the influence of neighboring robots in observation embedding or inter-robot interaction rules, leading to information loss and inefficient cooperation. This letter proposes a cooperative pursuit algorithm named Decentralized Adaptive COOperative Pursuit via Attention (DACOOP-A) by empowering reinforcement learning with artificial potential field and attention mechanisms. An attention-based framework is developed to emphasize important neighbors by concurrently integrating the learned attention scores into observation embedding and inter-robot interaction rules. A KL divergence regularization is introduced to alleviate the resultant learning stability issue. Improvements in data efficiency and generalization are demonstrated through numerical simulations. Extensive quantitative analyses are performed to illustrate the advantages of the proposed modules. Real-world experiments are performed to justify the feasibility of DACOOP-A in physical systems.