Grasping objects in a smooth humanlike motion, instead of the more typical pick-and-place approach, includes multiple aspects that need to be performed correctly for a successful grasp. These aspects involve moving the end-effector such that its surface makes and retains contact with the object while also coordinating the movement of the gripper to securely grasp the object. This work investigates how the intricate task of grasping may be learned from humans based on kinesthetic demonstrations. Due to the complexity of the task, these demonstrations are often slow and even slightly flawed, particularly at moments when multiple aspects (i.e. end-effector movement, orientation and gripper width) have to be demonstrated at once. Rather than training a person to provide faster demonstrations, non-expert users are provided with the ability to interactively modify the dynamics of their initial demonstration through teleoperated corrective feedback. This in turn allows them to teach motions outside of their own physical capabilities. In the end, the goal is to obtain a faster but reliable execution of the task. The presented framework learns the desired movement dynamics based on the current Cartesian position with Gaussian Processes, resulting in a reactive, time-invariant policy. Using Gaussian Processes also allows online interactive corrections and active disturbance rejection through epistemic uncertainty minimization. The experimental evaluation of the framework is carried out on a Franka-Emika Panda. Tests were performed to determine i) the framework's effectiveness in successfully learning how to grasp an object quickly, ii) ease of policy correction to environmental changes (i.e. different object shapes and mass), and iii) the framework’s usability for non-expert users.

Creating autonomous Micro Aerial Vehicles for executing complex missions poses various challenges, including safe navigation in the presence of external wind disturbances. Most current navigation methods handle external wind disturbances through real-time estimation and rejection algorithms in the control stage, but lack safety guarantees in strong winds. Recent robust methods provide safety guarantees but can be overly conservative. With the availability of more powerful computing devices, data-driven control algorithms are becoming increasingly feasible. Combining Gaussian Process models with Model Predictive Control has shown to enhance safety and performance in various control applications. Moreover, Model Predictive Control has been extended to solve more complex optimization algorithms that combine trajectory generation and tracking, preventing reference trajectories that are risky and challenging to control in the presence of disturbances. This research aims to improve quadrotor navigation in windy environments by using Gaussian Processes to model wind disturbances. The Gaussian Process model is integrated with a Model Predictive Contouring Control formulation that combines trajectory generation and control into one optimization problem. A nominal model of the quadrotor is derived and the Gaussian Process disturbance model is trained with the quadrotor position as input and wind disturbance as output. The wind disturbance map, along with the nominal model, is implemented in the Model Predictive Contouring Controller to consider both the mean and uncertainty of the resulting probabilistic model. This study validates the use of Gaussian Processes to model complex wind disturbances in quadrotor navigation. The wind disturbance map is trained from available state information, with data collected using an optimal exploration design to minimize uncertainty and reduce exploration times. The hyperparameters involved in training the Gaussian Process model are discussed and the implementation of sparse Gaussian Processes is outlined to make it real-time feasible for the Model Predictive Contouring Control formulation. Including the prediction model by incorporating chance constraints results in improved tracking and increased robustness in cluttered environments compared to the nominal Model Predictive Contouring Control formulation. The proposed algorithm is shown to outperform state-of-the-art methods for safe quadrotor navigation in windy and cluttered environments, being able to handle more complex wind fields than existing methods while also being less conservative.

This work applies interactive imitation learning for the navigation of a mobile robot. The algorithm"Learning Interactively to Resolve Ambiguity in Sensor Policy Fusion" (LIRA-SPF) is introduced in the field of machine learning for robot navigation. This algorithm extends on existing methods by allowing the ambiguity-free fusion of existing single-sensor policy behavior using an active and interactive querying of the human expert. The ambiguous situations investigated in this work are due the possible perspective mismatch of each sensor: LIRA-SPF aims to detect these situations and save the correct solution in a new fused policy. As a consequence, we provide an alternative to training a new behavior again from scratch, leveraging the knowledge of existing expert behaviors and reducing the required teacher’s effort. The algorithm is tested with different supervised and unsupervised disambiguation strategies thanks to its modular implementation. This paper summarizes multiple simulated and real robot tests, showing the advantages of the proposed disambiguation module on state of the art approaches. In particular, the analysis underlines the necessity of less human-robot interaction during the training process. Finally the conclusions reveal the missing blocks of the approach and how this could be beneficial in the sensor fusion procedure.

Robot dexterous manipulation research has drawn more attention in recent years since the development of various learning methods makes it possible for robots to achieve dexterity at the human level. Many attempts have been made to integrate human knowledge into Reinforcement Learning (RL) processes for faster learning speed and better performance. Despite their successes in many aspects, there are two open problems that still need to be carefully considered: 1. The effect of demonstrations gradually vanishes during RL. 2. In most cases, only imperfect demonstrations are available to robots. In this work, we proposed a new learning framework - Interactive Behavioural Cloning for faster Reinforcement Learning (IBC-RL), which could alleviate problems in complex manipulation tasks with long horizons. Different demonstrations are shown to robots at different learning stages. Robots learn complex tasks step by step with interactive demonstrations from human teachers. The framework is evaluated with four dexterous manipulation tasks simulated with the Isaac Gym engine. Human teachers perform demonstrations by controlling the simulated robot hands through a hand-tracking system. The results of the experiments could demonstrate the efficiency of IBC-RL in guiding and accelerating the learning processes with imperfect demonstrations.

Recent research has shown that a Learning from Demonstration (LfD) approach is useful for teaching robots flexible skills efficiently, and it opens the possibility for non-expert users to program these skills. When learning from demonstration data, learning frameworks should learn representations that are flexible and can generalize to unseen situations. Within the context of multi-reference frame skill learning, this work proposes a framework to learn such a representation without using task-specific heuristics or pre-segmentation of the demonstrations. Local policies are first learned by fitting the local dynamics with respect to each frame using Gaussian Processes (GP). A classifier that determines the relevance of each frame for every time step is then trained in a self-supervised manner. The uncertainty quantification capability of Gaussian Processes is exploited to improve the performance of the local policies and the self-supervised learning process of the classifier. The framework is validated through multi-frame tasks in simulation as well as on a robotic manipulator with a pick-and-place re-shelving task. Its performance is also compared to that of the Task-Parameterised Gaussian Mixture Model (TPGMM) with simulated and robotic data. In this comparison, the proposed model performs better according to metrics that quantify deviation from the goal at each reference frame and according to similarity measures between demonstrations and their corresponding reproductions.