Augmented Reality Interfaces in Robotic Manipulation

Abstract

This study investigated the human-robot interaction of three different augmented reality interfaces that enabled an operator to control a robot arm. The first control methods gives the operator complete manual control by moving a holographic object which the robot arm copies (Direct control). The second control method allows the operator to manually set holographic waypoints which the robot arm autonomously tries to execute (Waypoint control). And the third control method, gives the operator the capability to specify a target location which the robot autonomously moves towards (Command control). An experiment was designed to give 30 participants the capability to control a Franka Emika Research 3 robot arm with a Hololens. During the experiment, the participant had to use each control method to move the end-effector of the robot arm from a starting position to a goal position in 9 different environments. The 9 environments were divided into three manually created complexity levels (Easy, Medium, and Hard), which were based upon the performance of an informed RRT* path planner that was simulated in 67.500 unique environments. The experiment results showed that Command control outperformed the other two methods in the success rate, path length, and operation time. In terms of number of routes Command control had in general a lower number compared to Direct control, showcasing that Direct control has more flexibility in route selection. Furthermore, Direct control required more skill to get a similar performances in path length, operation time, and number of collisions as Command control. The design of Waypoint control made this method too unpredictable for the participants to use proficiently, which was underscored by the very low success rate, the long operation time, and large path length of this method. Comparing the performance of the control methods against the complexity levels did not show many significant differences, except for the success rate that decreased with a more complex environment. In summary, each control method will require unique features to completely unlock the possibilities of an augmented reality interface. The three control methods show different performances, each with their own strengths and weaknesses. And the choice of a control method is therefore dependent on the task at hand.