Where to go next

Where to go next: Learning a Subgoal Recommendation Policy for Navigation in Dynamic Environments

Ferreira de Brito, B.F. (TU Delft Learning & Autonomous Control)
Everett, Michael (Massachusetts Institute of Technology)
How, Jonathan Patrick (Massachusetts Institute of Technology)
Alonso-Mora, J. (TU Delft Learning & Autonomous Control)

2021

Robotic navigation in environments shared with other robots or humans remains challenging because the intentions of the surrounding agents are not directly observable and the environment conditions are continuously changing. Local trajectory optimization methods, such as model predictive control (MPC), can deal with those changes but require global guidance, which is not trivial to obtain in crowded scenarios. This paper proposes to learn, via deep Reinforcement Learning (RL), an interaction-aware policy that provides long-term guidance to the local planner. In particular, in simulations with cooperative and non-cooperative agents, we train a deep network to recommend a subgoal for the MPC planner. The recommended subgoal is expected to help the robot in making progress towards its goal and accounts for the expected interaction with other agents. Based on the recommended subgoal, the MPC planner then optimizes the inputs for the robot satisfying its kinodynamic and collision avoidance constraints. Our approach is shown to substantially improve the navigation performance in terms of number of collisions as compared to prior MPC frameworks, and in terms of both travel time and number of collisions compared to deep RL methods in cooperative, competitive and mixed multiagent scenarios.

Collision avoidance
Deep Reinforcement Learning
Motion and Path Planning in Dynamic Environments or for Multi-robot Systems
Navigation
Planning
Robot kinematics
Robots
Training
Vehicle dynamics

http://resolver.tudelft.nl/uuid:9d6316ec-035d-44bc-8d76-85c8a6c74846

https://doi.org/10.1109/LRA.2021.3068662

2021-09-24

2377-3766

IEEE Robotics and Automation Letters, 6 (3), 4616-4623

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2021 B.F. Ferreira de Brito, Michael Everett, Jonathan Patrick How, J. Alonso-Mora