Learning Dexterous Grasping Policies via Neural Geometric Fabrics

Abstract

Geometric fabrics are a modern method to generate stable and safe motion. Desired behaviors are designed in low-dimensional task spaces. Neural Geometric Fabrics learn the components which construct these fabrics from demonstrations. However, the task spaces themselves are still designed by hand, which requires expert knowledge. We present Fully-Learned Neural Geometric Fabrics (FL-NGF), which replaces task spaces with neural encoders learned from the same demonstrations. We evaluate it on dexterous grasping with a simulated 22-degree-of-freedom Universal Robots UR5e and four-fingered Shadow Robot Dexterous Lite Hand.

To assess how original Neural Geometric Fabrics and fully learned Geometric Fabrics differ from each other, we train them on demonstrations with force production. We collect two teleoperated datasets with a SenseGlove R1 haptic glove, one with active force feedback to the operator and one without. Force feedback lowers the contact forces the operator applies by 1.62×, producing a gentler set of demonstrations that leave less force margin for a lift to succeed.

On the dataset recorded without force feedback, FL-NGF matches the hand-crafted baseline in pickup success and generalizes slightly better to novel object positions. This results in a system that is easier to implement, because no expert is needed in the design process. On the force feedback dataset, the two separate sharply, with the hand-designed task spaces losing most of their success rate while the learned ones hold up. Learned task spaces are therefore not uniformly better, but less sensitive to a change in demonstration quality than the original geometric fabric model. No architecture we tested manages both to reproduce the gentler demonstrated grasp and to grasp reliably.