Learning from demonstration (LfD) has proved useful for teaching robots complex skills with high sample efficiency. However, teaching long-horizon tasks with multiple skills is challenging as deviations tend to accumulate, the distributional shift becomes more evident, and human teachers become fatigued over time, thereby increasing the likelihood of failure. To address these challenges, we introduce (ST)² a sequential method for learning long-horizon manipulation tasks that allows users to control teaching flow by specifying keypoints, enabling structured and incremental demonstrations. Using this framework, we study how users respond to two teaching paradigms: 1) a traditional monolithic approach in which users demonstrate the entire task trajectory at once, and 2) a sequential approach, in which the task is segmented and demonstrated step by step. We conducted an extensive user study on the restocking task with 16 participants in a realistic retail store environment, evaluating the user preferences and effectiveness of the methods. A user-level analysis showed superior performance for the sequential approach in most cases (10 users), compared with the monolithic approach (five users), with one tie. Our subjective results indicate that some teachers prefer sequential teaching - as it allows them to teach complicated tasks iteratively - or others prefer teaching in one go due to its simplicity.

Collision avoidance – involving a rapid threat detection and quick execution of the appropriate evasive maneuver – is a critical aspect of driving. However, existing models of human collision avoidance behavior are fragmented, focusing on specific scenarios or only describing certain aspects of the avoidance behavior, such as response times. This paper addresses these gaps by proposing a computational cognitive model of human collision avoidance behavior based on active inference. Active inference provides a unified approach to modeling human behavior: the minimization of free energy. Building on prior active inference work, our model incorporates established cognitive mechanisms such as evidence accumulation to simulate human responses in three distinct collision avoidance scenarios: front-to-rear lead vehicle braking, lateral incursion by an oncoming vehicle, and another vehicle failing to yield at an intersection. We demonstrate that our model explains a wide range of empirical findings on human collision avoidance behavior. Specifically, the model closely reproduces both aggregate results from meta-analyses previously reported in the literature and detailed, scenario-specific effects observed in two recent driving simulator studies, including response timing, maneuver selection, and execution. Our results highlight the potential of active inference as a generalizable framework for understanding and modeling human behavior in complex real-life driving tasks.

For an autonomous vehicle to operate reliably within real-world traffic scenarios, it is imperative to assess the repercussions of its prospective actions by anticipating the uncertain intentions exhibited by other participants in the traffic environment. Driven by the pronounced multi-modal nature of human driving behavior, this paper presents an approach that leverages Bayesian beliefs over the distribution of potential policies of other road users to construct a novel risk-aware probabilistic motion planning framework. In particular, we propose a novel contingency planner that outputs long-term contingent plans conditioned on multiple possible intents for other actors in the traffic scene. The Bayesian belief is incorporated into the optimization cost function to influence the behavior of the short-term plan based on the likelihood of other agents' policies. Furthermore, a probabilistic risk metric is employed to fine-tune the balance between efficiency and robustness. Through a series of closed-loop safety-critical simulated traffic scenarios shared with human-driven vehicles, we demonstrate the practical efficacy of our proposed approach that can handle multi-vehicle scenarios.

Wind turbines are getting larger to increase power capacity. Their longer blades sample a larger area of the spatially and temporally varying turbulent wind field, leading to increased periodic blade load and fatigue damage over time. Individual pitch control (IPC) has proven effective in alleviating these loads by pitching the blades. Conventional IPC fully attenuates the periodic blade loads, which requires excessive pitching, leading to additional stresses on the pitch system. To balance pitch actuation and load alleviation, bounds can be set on the pitch signal (input-constrained IPC), or on the load (output-constrained IPC). While input-constrained IPC has been abundantly researched, little research has focused on output-constrained IPC and on the trade-off when operating between full IPC and no IPC. Therefore, we propose an output-constrained IPC method using an adaptive leaky integrator. The natural frequency of the leaky integrator is adapted on the error between the reference and resultant blade moment. This allows the control scheme to attain every load alleviation level between full and no IPC. Furthermore, in realistic turbulent wind conditions, operating close to full IPC leads to diminishing returns, showing that the proposed controller achieves a superior trade-off between load reduction and actuator effort.

Sim2real, the transfer of control policies from simulation to the real world, is crucial for efficiently solving robotic tasks without the risks associated with real-world learning. How-ever, discrepancies between simulated and real environments, especially due to unmodeled dynamics and latencies, significantly impact the performance of these transferred policies. In this paper, we address the challenges of sim2real transfer caused by latency and asynchronous dynamics in real-world robotic systems. Our approach involves developing a novel framework, REX (Robotic Environments with jaX), that uses a graph-based simulation model to incorporate latency effects while optimizing for parallelization on accelerator hard-ware. Our framework simulates the asynchronous, hierarchical nature of real-world systems, while simultaneously estimating system dynamics and delays from real-world data and implementing delay compensation strategies to minimize the sim2real gap. We validate our approach on two real-world systems, demonstrating its effectiveness in improving sim2real performance by accurately modeling both system dynamics and delays. Our results show that the proposed framework supports both accelerated simulation and real-time processing, making it valuable for robot learning.

This paper introduces a new imitation learning framework based on energy-based generative models capable of learning complex, physics-dependent, robot motion policies through state-only expert motion trajectories. Our algorithm, called Noise-conditioned Energy-based Annealed Rewards (NEAR), constructs several perturbed versions of the expert's motion data distribution and learns smooth, and well-defined representations of the data distribution's energy function using denoising score matching. We propose to use these learnt energy functions as reward functions to learn imitation policies via reinforcement learning. We also present a strategy to gradually switch between the learnt energy functions, ensuring that the learnt rewards are always well-defined in the manifold of policy-generated samples. We evaluate our algorithm on complex humanoid tasks such as locomotion and martial arts and compare it with state-only adversarial imitation learning algorithms like Adversarial Motion Priors (AMP). Our framework sidesteps the optimisation challenges of adversarial imitation learning techniques and produces results comparable to AMP in several quantitative metrics across multiple imitation settings. Code and videos available at anishhdiwan.github.io/noise-conditionedenergy-based-annealed-rewards/.

Objectives: To develop and validate a questionnaire on dental students' self-efficacy with tooth removal, suitable for measuring the effectiveness of training methods. Methods: To prepare and validate this questionnaire, we used the Association of Medical Education in Europe (AMEE) stepwise guide for developing questionnaires for educational research. In the validation process, our study group conducted two pilot studies, the first for an exploratory factor analysis and the second for a confirmatory factor analysis. In addition, the questionnaire was tested for convergence with the neuroticism subscale of the NEO-Personality Inventory. Results: After an exploratory factor analysis, which used a total of 137 responses on 33 items, 15 items were left for confirmatory factor analysis. A total of 118 responses were available for the confirmatory factor analysis. Model fitness was tested using tests for exact fitness and fit indices such as the goodness of fit index (GFI), root mean square error of approximation (RMSEA) and standardised root mean squared residual (SRMR). An acceptable fit was found for 11 items divided over three factors: ‘self-perceived skill’, ‘tension’ and ‘dedication’. These 11 items did not converge with the neuroticism scale. Conclusion: This study showed the development steps and initial validation of a psychometric instrument, the Amsterdam Self-Efficacy Scale for Tooth Removal (ASES-TR), consisting of 11 items for testing dental students' self-efficacy in performing tooth removal procedures.

Human teaching effort is a significant bottleneck for the broader applicability of interactive imitation learning. To reduce the number of required queries, existing methods employ active learning to query the human teacher only in uncertain, risky, or novel situations. However, during these queries, the novice’s planned actions are not utilized despite containing valuable information, such as the novice’s capabilities, as well as corresponding uncertainty levels. To this end, we allow the novice to say: “I plan to do this, but I am uncertain.” We introduce the Active Skill-level Data Aggregation (ASkDAgger) framework, which leverages teacher feedback on the novice plan in three key ways: (1) S-Aware Gating (SAG): Adjusts the gating threshold to track sensitivity, specificity, or a minimum success rate; (2) Foresight Interactive Experience Replay (FIER), which recasts valid and relabeled novice action plans into demonstrations; and (3) Prioritized Interactive Experience Replay (PIER), which prioritizes replay based on uncertainty, novice success, and demonstration age. Together, these components balance query frequency with failure incidence, reduce the number of required demonstration annotations, improve generalization, and speed up adaptation to changing domains. We validate the effectiveness of ASkDAgger through language-conditioned manipulation tasks in both simulation and real-world environments. Code, data, and videos are available at https://askdagger.github.io.

In robot manipulation, Reinforcement Learning (RL) often suffers from low sample efficiency and uncertain convergence, especially in large observation and action spaces. Foundation Models (FMs) offer an alternative, demonstrating promise in zero-shot and few-shot settings. However, they can be unreliable due to limited physical and spatial understanding. We introduce ExploRLLM, a method that combines the strengths of both paradigms. In our approach, FMs improve RL convergence by generating policy code and efficient representations, while a residual RL agent compensates for the FMs' limited physical understanding. We show that Explorllm outperforms both policies derived from FMs and RL baselines in table-top manipulation tasks. Additionally, real-world experiments show that the policies exhibit promising zero-shot sim-to-real transfer. Supplementary material is available at https://explorllm.github.io.

Quadrupedal jumping has been intensively investigated in recent years. Still, realizing controlled jumping with soft landings remains an open challenge due to the complexity of the jump dynamics and the need to perform complex computations during the short time. This work tackles this challenge by leveraging trajectory optimization and behavior cloning. We generate an optimal jumping motion by utilizing dual-layered coarse-to-refine trajectory optimization. We combine this with a variable impedance control approach to achieve soft landing. Finally, we distill this computationally heavy jumping and landing policy into an efficient neural network via behavior cloning. Extensive simulation experiments demonstrate that, compared to classic model predictive control, the variable impedance control ensures compliance and reduces the stress on the motors during the landing phase. Furthermore, the neural network can reproduce jumping and landing behavior, achieving at least a 97.4% success rate. Hardware experiments confirm the findings, showcasing explosive jumping with soft landings and on-the-fly evaluation of the control actions.

Learning from demonstration is a promising approach for teaching robots new skills. However, a central challenge in the execution of acquired skills is the ability to recognize faults and prevent failures. This is essential because demonstrations typically cover only a limited set of scenarios and often only the successful ones. During task execution, unforeseen situations may arise, such as changes in the robot's environment or interaction with human operators. To recognize such situations, this paper focuses on teaching the robot situational awareness by using a camera input and labeling frames as safe or risky. We train a Gaussian Process (GP) regression model fed by a low-dimensional latent space representation of the input images. The model outputs a continuous risk score ranging from zero to one, quantifying the degree of risk at each timestep. This allows for pausing task execution in unsafe situations and directly adding new training data, labeled by the human user. Our experiments on a robotic manipulator show that the proposed method can reliably detect both known and novel faults using only a single example for each new fault. In contrast, a standard multi-layer perceptron (MLP) performs well only on faults it has encountered during training. Our method enables the next generation of cobots to be rapidly deployed with easy-to-set-up, vision-based risk assessment, proactively safeguarding humans and detecting misaligned parts or missing objects before failures occur.

Several recent works show impressive results in mapping language-based human commands and image scene observations to direct robot executable policies (e.g., pick and place poses). However, these approaches do not consider the uncertainty of the trained policy and simply always execute actions that are suggested by the current policy as the most probable ones. This makes them vulnerable to domain shift and inefficient in the number of required demonstrations. We extend previous works and present the PARTNR algorithm that can detect ambiguities in the trained policy by analyzing multiple modes in the probability distributio of pick and place poses using topological analysis. In this way uncertainty in action can be estimated with single inference (and training single model) instead of using ensemble of models. Additionally, PARTNR employs an adaptive, sensitivity-based, gating function that decides if additional user demonstrations are required. User demonstrations are aggregated to the dataset and used for subsequent training. In this way, the policy can adapt promptly to domain shift and it can minimize the number of required demonstrations for a well-trained policy. The adaptive threshold enables to achieve the user-acceptable level of ambiguity to execute the policy autonomously and in turn, increase the trustworthiness of our system. We demonstrate the performance of PARTNR in a table-top pick and place task.

Sim2real, that is, the transfer of learned control policies from simulation to the real world, is an area of growing interest in robotics because of its potential to efficiently handle complex tasks. The sim2real approach faces challenges because of mismatches between simulation and reality. These discrepancies arise from inaccuracies in modeling physical phenomena and asynchronous control, among other factors. To this end, we introduce Engine Agnostic Graph Environments for Robotics (EAGERx), a framework with a unified software pipeline for both real and simulated robot learning. It can support various simulators and aids in integrating state, action, and time scale abstractions to facilitate learning. EAGERx’s integrated delay simulation, domain randomization features, and proposed synchronization algorithm contribute to narrowing the sim2real gap. We demonstrate (in the context of robot learning and beyond) the efficacy of EAGERx in accommodating diverse robotic systems and maintaining consistent simulation behavior. EAGERx is open source, and its code is available at https://eagerx.readthedocs.io

To promote the research in compliant quadrupedal locomotion, especially with parallel elasticity, we present Delft E-Go, which is an easily accessible quadruped that combines the Unitree Go1 with open-source mechanical add-ons and control architecture. Implementing this novel system required a combination of technical work and scientific innovation. First, a dedicated parallel spring with adjustable rest length is designed to strengthen each actuated joint. Then, a novel 3-D dual spring-loaded inverted pendulum model is proposed to characterize the compliant locomotion dynamics, decoupling the actuation with parallel compliance. Based on this template model, trajectory optimization is employed to generate optimal explosive motion without requiring reference defined in advance. To complete the system, a torque controller with anticipatory compensation is adopted for motion tracking. Extensive hardware experiments in multiple scenarios, such as trotting across uneven terrains, efficient walking, and explosive pronking, demonstrate the system’s reliability, energy benefits of parallel compliance, and enhanced locomotion capability. Particularly, we demonstrate for the first time the controlled pronking of a quadruped with asymmetric legs.