Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration

Abstract

This paper proposes a novel optimal control scheme for constrained image based visual servoing of a robot manipulator. For a robot manipulator with an eye-on-hand configuration, visibility constraint is an essential requirement to avoid servo failure, while robot's actuator limits must also be satisfied. To ensure this, the constraints are modelled implicitly via learning the task and defining safe regions using expert human demonstrations via mixture of Dynamic Movement Primitives (DMPs). The visual servoing problem is then formulated as a closed-loop optimal control problem using these constraint model where a desired target (possibly time-varying) is obtained by acting upon the feedback from the real-time visual sensors. The visual servo control loop consists of a single network adaptive critic optimal tracking control scheme whose weights are tuned using Lyapunov stability criteria. The stability and the performance of the proposed control scheme is shown theoretically via Lyapunov approach and also verified experimentally using a seven degree of freedom (DOF) Franka Emika and six DOF Universal Robot (UR) 10 manipulator. The approach is also demonstrated on a use case scenarios in mock-up convenience store and warehouse setup.