Curriculum-Based Deep Reinforcement Learning for Explosive Jumping in Quadruped Robots

Curriculum-Based Deep Reinforcement Learning for Explosive Jumping in Quadruped Robots

Atanassov, Vassil (TU Delft Mechanical, Maritime and Materials Engineering; TU Delft Cognitive Robotics)

Ding, J. (mentor)
Della Santina, C. (graduation committee)

Delft University of Technology

Mechanical Engineering | Vehicle Engineering | Cognitive Robotics

2023-08-21

Legged animals possess extraordinary agility with which they can gracefully traverse a wide range of environments, from running through grasslands to jumping across cliffs and climbing nearly vertical walls. Inspired by this, in this work, we use Deep Reinforcement Learning to give legged robots the ability to perform a diverse set of highly explosive and agile jumps. Unlike other works, our approach is not constrained to imitating a reference trajectory. We instead use curriculum-based learning to progressively learn more challenging tasks, starting from a vertical high jump and then generalising to forward and diagonal jumps. In the final curriculum stage, the robot learns to leap over barrier-like obstacles or to land on them, conditioned on the desired jumping distance and the object's dimensions. We show that such an approach can produce a wide range of robust and precise motions, which we thoroughly and successfully validated in several indoor and outdoor real-world experiments on the Unitree Go1 robot. In our real-world experiments, we show a forward jump of 90cm, exceeding previous records for similar robots reported in the literature. Additionally, we investigate the effects of incorporating bio-inspired parallel elastic actuators to improve the jumping performance further. This resulted in smoother motions, much softer landings with lower joint velocities and larger jumps. Finally, we present and analyse the limitations of our method and introduce exciting directions for future work to address them.

Learning
Legged locomotion
Reinforcement Learning
Jumping
Agile locomotion

http://resolver.tudelft.nl/uuid:44e049e6-2c7c-4ab1-81d6-496695d62aa4

Student theses

master thesis

© 2023 Vassil Atanassov