A review on machine learning in flexible surgical and interventional robots: Where we are and where we are going

Minimally Invasive Procedures (MIPs) emerged as an alternative to more invasive surgical approaches, offering patient benefits such as smaller incisions, less pain, and shorter hospital stay. In one class of MIPs, where natural body lumens or small incisions are used to access deeper anatomical locations, Flexible Surgical and Interventional...

Robust Jumping with an Articulated Soft Quadruped via Trajectory Optimization and Iterative Learning

Quadrupeds deployed in real-world scenarios need to be robust to unmodelled dynamic effects. In this work, we aim to increase the robustness of quadrupedal periodic forward jumping (i.e., pronking) by unifying cutting-edge model-based trajectory optimization and iterative learning control. Using a reduced-order soft anchor model, the...

Visually-guided motion planning for autonomous driving from interactive demonstrations

The successful integration of autonomous robots in real-world environments strongly depends on their ability to reason from context and take socially acceptable actions. Current autonomous navigation systems mainly rely on geometric information and hard-coded rules to induce safe and socially compliant behaviors. Yet, in unstructured urban...

Learning to Pick at Non-Zero-Velocity from Interactive Demonstrations

This work investigates how the intricate task of a continuous pick & place (P&P) motion may be learned from humans based on demonstrations and corrections. Due to the complexity of the task, these demonstrations are often slow and even slightly flawed, particularly at moments when multiple aspects (i.e., end-effector movement,...

DeepKoCo: Efficient latent planning with a task-relevant Koopman representation

This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy...

Fine-tuning deep RL with gradient-free optimization

Deep reinforcement learning makes it possible to train control policies that map high-dimensional observations to actions. These methods typically use gradient-based optimization techniques to enable relatively efficient learning, but are notoriously sensitive to hyperparameter choices and do not have good convergence properties. Gradient...

Reinforcement learning based compensation methods for robot manipulators

Smart robotics will be a core feature while migrating from Industry 3.0 (i.e., mass manufacturing) to Industry 4.0 (i.e., customized or social manufacturing). A key characteristic of a smart system is its ability to learn. For smart manufacturing, this means incorporating learning capabilities into the current fixed, repetitive, task-oriented...

Reinforcement learning for control: Performance, stability, and deep approximators

Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the...

Experience selection in deep reinforcement learning for control

Experience replay is a technique that allows off-policy reinforcement-learning methods to reuse past experiences. The stability and speed of convergence of reinforcement learning, as well as the eventual performance of the learned policy, are strongly dependent on the experiences being replayed. Which experiences are replayed depends on two...

Mixture of Attractors: A novel movement primitive representation for learning motor skills from demonstrations

In this letter, we introduce Mixture of Attractors, a novel movement primitive representation that allows for learning complex object-relative movements. The movement primitive representation inherently supports multiple coordinate frames, enabling the system to generalize a skill to unseen object positions and orientations. In contrast to...

Reinforcement learning of potential fields to achieve limit-cycle walking

Reinforcement learning is a powerful tool to derive controllers for systems where no models are available. Particularly policy search algorithms are suitable for complex systems, to keep learning time manageable and account for continuous state and action spaces. However, these algorithms demand more insight into the system to choose a...

Improved deep reinforcement learning for robotics through distribution-based experience retention

Recent years have seen a growing interest in the use of deep neural networks as function approximators in reinforcement learning. In this paper, an experience replay method is proposed that ensures that the distribution of the experiences used for training is between that of the policy and a uniform distribution. Through experiments on a...