Recent progress in visual imitation learning has shown that diffusion models are a powerful tool for training robots to perform complex manipulation tasks. While 3D Diffusion Policy uses a point cloud representation to improve spatial reasoning and sample efficiency, it still struggles to generalize across novel objects and environments due to spurious correlations learned from irrelevant visual features. In this work, a novel approach, Affordance-guided 3D Diffusion Policy (ADP3) is introduced, which integrates task-relevant affordance cues into the policy’s point cloud input. By conditioning the policy on 3D affordance heatmaps instead of raw point clouds, the policy is biased to attend to task-relevant object regions. Using affordance heatmaps reduced the success rate drop to just 3% on unseen objects in 4 Meta-World tasks, compared to a 35% drop when using raw point clouds. ADP3 also demonstrates impressive performance in our real-world experiments, showing resilience to cluttered scenes and novel object orientations.

This thesis explores how deliberate modifications to reward function design in the reinforcement learning can induce skill mutations in robotic reinforcement learning, specifically within a precision pouring task. Using a simulated Franka Emika Panda robot in NVIDIA Isaac Lab, we evaluate 25 distinct reward configurations composed of weighted terms for effort, accuracy, and velocity. The resulting policies exhibit a wide range of behaviors—from fast and efficient pours to novel skills such as rim cleaning, mixing, and watering—demonstrating that small adjustments in reward structure can yield significant variations in learned strategies. Our analysis demonstrates that even small changes in reward structure can lead to significant shifts in policy behavior, facilitating both task-optimal and creative, potentially transferable strategies. To validate this concept, we implement it using the Proximal Policy Optimization (PPO) algorithm, showing that reward design alone—without altering the learning architecture—can drive meaningful skill diversification. This approach offers promising directions for adaptive control, transfer learning, and multi-objective optimization in robotic systems.

Autonomous vessels offer potential benefits in safety, operational efficiency, and environmental impact, but require reliable perception systems to function independently. This thesis investigates the reliability of camera-based detection of maritime docks, a class of static but visually diverse objects that are often difficult to distinguish from their surroundings. While extensive research exists for detecting ships and buoys, docks remain underexplored, with no publicly available datasets containing a significant number of labeled instances.
To address this gap, the Dordrecht Dock Dataset was developed, containing 30,761 frames recorded under real-world maritime conditions across eight distinct docks. A custom interpolation-based annotation tool enabled efficient labeling, achieving annotation speeds of up to 3,000 frames per hour. Two deep learning models—Faster R-CNN and YOLO11n—were trained and evaluated on this dataset using a consistent pipeline. Evaluation followed a leave-one-dock-out cross-validation strategy with fixed test sets and standard performance metrics, including mAP50, mAP50–95, F1 score, and inference speed.
Faster R-CNN outperformed YOLO11n across nearly all accuracy metrics (mAP50 of 0.85 vs. 0.69), particularly at shorter ranges. YOLO11n demonstrated over twice the inference speed (37 FPS vs. 17 FPS), but exhibited weaker generalization and reduced reliability on visually different docks. Both models showed significant performance drops beyond 100–110 meters, emphasizing the need for high-resolution input or focus on short-range detection.

Introduction: Knee osteoarthritis (KOA) is a degenerative joint condition that affects both the medial and lateral femoral condyles, and is a leading cause of pain and reduced mobility worldwide. Low-impact exercises such as cycling are increasingly explored, as cycling reduces knee joint contact forces (KJCF) compared to high-impact activities, making it a suitable option for preserving joint health. Understanding the distribution of KJCF across the medial and lateral condyles during cycling is crucial for informing effective exercise interventions.

Objective: This study aims to validate an existing musculoskeletal cycling model using experimental electromyography (EMG) data, and to extend it to estimate the distribution of the tibiofemoral compressive force (TFC) across the medial and lateral condyles.

Methods: Eleven healthy recreational cyclists (mean age: 25.7 ± 1.7 years) participated in lab-based cycling trials. Reflective markers and EMG sensors captured lower limb kinematics and muscle activity at varying cadences (70–90 RPM) and power outputs (80–120 W). The musculoskeletal cycling model of Clancy et al. (2023) in OpenSim was modified to include separate medial and lateral knee compartments. A publicly available dataset was used to scale the modified model and run inverse kinematics (IK), static optimisation (SO), and joint reaction force (JRF) analysis. Predicted muscle activation was evaluated using five matched comparison pairs and cross-correlation analysis. Finally, the modified model was used to estimate the distribution of TFC across the medial and lateral condyles throughout the crank cycle.

Results: The modified model produced near-identical results compared to the original model in both muscle activation (R > 0.99) and total TFC (R > 0.98), verifying the model modifications.
Model-predicted muscle activations showed the highest correlation with EMG data for the vastus medialis (VM) (R = 0.88–0.99) and the vastus lateralis (VL) (R = 0.84–0.99). Lower correlations were observed for the rectus femoris (RF), gastrocnemii (GL, GM), biceps femoris (BF), and semitendinosus (ST). In all subjects, lateral TFC consistently exceeded medial TFC throughout most of the crank cycle.

Conclusion: This thesis successfully estimated compartmental TFC during cycling without affecting the muscle activations or total TFC predictions. Comparison with EMG data showed variable agreement, reflecting modelling limitations and inter-subject variability. The results provide valuable insight into asymmetrical knee joint loading during cycling and illustrate the potential of musculoskeletal modelling to guide more targeted rehabilitation strategies for individuals with KOA.

The psychological impact that robots have on workers in a physical human-robot collaboration is not well researched. This creates a risk of creating monotonous jobs for workers as more and more robots enter the workforce. To mitigate this risk, physical human-robot collaboration should be designed to have a positive psychological impact by facilitating a flow state. Flow is the experience of complete absorption into the current moment. Based on studies in other fields, a new method for designing a physical human-robot collaboration for flow has been developed. This new method was applied to an abstract blending task in which the robot assisted the human in blending. A controlled experiment was conducted to test the psychological impact that this design method had on participants. The findings provide evidence that designing for flow made the task more satisfying, as such indications were found that suggest participants were more motivated in their task, finding it more rewarding, causing them to do more of the task than required. However, no significant impact on flow was found because participants in the control group were equally focused as participants who had the task designed for flow. Nonetheless, the positive impact that designing the physical-human robot collaboration had on the motivation and satisfaction of the humans still created a positive psychological impact. This makes designing for flow a promising method for building towards a human-centered future of work.

Strategy games provide a compelling testbed for developing human-like computer agents, with applications that extend beyond gaming into fields requiring adaptive and socially intelligent AI. In these games, players tend to enjoy and engage more deeply with AI opponents that not only provide a challenge but also behave in ways that resemble human thinking and decision-making. However, despite progress in developing such agents, there is still no standard approach for evaluating how human-like these opponents truly are—making it difficult to assess and improve their design. Here I show that strategy game opponents having more human-like game-level playstyles does not necessarily lead to them being more believable (perceived as human-like by human players).

By developing a turn-based strategy game and evaluating Hierarchical Reinforcement Learning (HRL) agents of varying complexity, I assessed both their behavioural similarity to human players and how believable they were perceived to be by human players. This research introduces a new approach for understanding player behaviour using behaviour vectors composed of three high-level metrics—Aggressiveness, Management, and Exploration—consistent with existing literature. These metrics are designed to be broadly applicable across strategy games, enabling consistent comparison between human and AI opponents, as well as across different games and agents. The findings demonstrate that while HRL agents can replicate human-like playstyles without using human training data, players judge human-likeness more on perceived intelligence and fairness. This suggests that creating truly human-like AI opponents requires not just replicating human game-level playstyles, but designing agents that align with players' expectations for intelligent and fair decision-making.

Force/torque sensors are essential tools that enable robots to effectively interact with their environments. Existing calibration methods often fail to capture inter-axis nonlinearities and coupling effects, particularly when available calibration data are sparse and discrete. To address this challenge, the presented approach employs a Deep Neural Network (DNN) that learns both the scaling and the direction of the input-output relationship. The method works by extracting the absolute magnitude and unit vector from the raw N-dimensional sensor output values, which can vary among sensors. The DNN takes this N-dimensional input and produces a 7-dimensional output—comprising a corrected 6D unit vector representing the desired force-torque direction and a scaling factor. The final measurement is then constructed by combining the output unit vector, the learned scaling factor, and the original input magnitude. This approach simplifies the calibration problem to a linear mapping along one axis, enabling the model to generalize well under limited training conditions while leveraging the DNN’s strength in capturing nonlinear inter-axis relationships. The proposed DNN was trained and evaluated on both artificially generated and real-world datasets, and its performance was compared to two baseline models: a commonly used linear transformation model and a comparative DNN approach from the literature. On generated data, the proposed DNN achieved an RMSE of 36.9 ± 3.44, outperforming the comparative DNN (48.3 ± 4.47) and the linear transformation model (62.3±0.76). Similar improvements were observed on the real-world dataset. Although these results are promising, they are based on artificially generated data and a single real-world dataset from one specific sensor. Further validation and more extensive testing are necessary. Nonetheless, the gains indicated here suggest meaningful potential for improved calibration strategies in force-controlled robotic applications, even under limited training conditions.

Purpose. Patient-tailored cervical cancer brachytherapy applicators may improve treatment outcomes over clinical applicators. There is a need for automated design and treatment planning as this is a complex, time consuming process. The ARCHITECT project has developed fully automated 3D printed personalised applicators, matching and improving conventional treatment. However, small postprocessing modifications may often be needed as it is difficult to quantify all criteria used during treatment planning. Current treatment planning systems allow physicians to adjust dwell times, but do not support modifications to automated needle configurations. This work presents an approach for real-time needle replanning in patient-tailored applicators demonstrated through a graphical user interface. Methods. Our approach to perform real-time replanning consists of a heuristic convex-decomposition step followed by quadratic optimisation. Safe flight corridors (SFCs) were generated by inflating a symmetric ellipsoid around automated trajectories and iteratively selecting tangent planes to form a convex volume. Needle trajectories were represented by Bézier curves and optimised considering the boundary volume constraints. Replanning could be done by the user by modifying the interstitial part of the needles through a graphical user interface developed in MATLAB. The algorithm was tested with a database of 20 patients, a dosimetric study was carried out with 5 of those patients and the GUI was evaluated with a usability study involving 8 participants. Results. Replanning of individual needles could be performed in <1.0 s with optimisation times of ~0.01 s followed by a feasibility check of 0.5 s. SFC generation and trajectory optimisation were tested with 99 needle channels (20 patients) showing an error rate of 17%. Replanning in a case study to demonstrate dosimetric benefits in 5 patients was feasible, however, did not lead to clear improvements in dose conformity. The interface received a usability score of 73 (min=57.5, max=92.5) from participants. Conclusions. The algorithm and interface were validated and used successfully. However, their use in a clinical setting requires improvements or exploration of alternative approaches to increase robustness.

Designing robotic systems such as quadrupeds is challenging due to the intricate relationship between motion and design, particularly when aiming to replicate the agility, efficiency, and versatility of animals. Co-design simplifies robotic development by simultaneously optimizing physical design and control algorithms in an integrated way. While most prior work validates co-design approaches in simulation, our research bridges this gap by transitioning optimized designs to real-world implementation. To achieve this, we developed a modular quadruped platform with bio-inspired legs that enables the physical implementation of the optimized designs. Our design space, which includes leg segment lengths, spring stiffness, and engagement angle, was optimized to maximize energy efficiency for real-world tasks. We propose a simplified learning-based co-design framework that combines reinforcement learning to create a universal locomotion controller with Bayesian optimization to select the best design. Real-world tests demonstrate a significant reduction in the cost of transport—18.6% for inspection tasks and 35.7% for payload tasks—compared to the nominal design without springs. In simulations, the universal controller adapts well across robot configurations, and the optimization process remains consistent across runs. Although some discrepancies between simulation and real-world performance remain, our findings underscore the potential of co-design to address complex trade-offs in real-world robotic system design.

Exo devices allows users to decrease muscle effort by offloading it to the exo device. Compliance in the physical human-robot interface, originating from flexible cuffs and soft body tissue, can affect the forces transferred. The design of the cuff can alter the value of the pHEI compliance, but it is not yet known to what extent changes in pHEI compliance affect the muscle activation. Due to human variances and the difficulty of measuring objective performance metrics, it is difficult to experimentally pin point cause and effect. Therefore a novel model based method was used which combined experimentally obtained pHEI compliance data with musculoskeletal models to simulate the effect of pHEI compliance on muscle activation. A case study using this method was performed on a subject wearing a passive shoulder exotendon suit. Results indicate a large effect of cuff design on stiffness, damping and cuff migration values. Furthermore, optimising a rigid model and adding compliance afterwards resulted in an increase of total normalised muscle cost. Results from this compliant simulation let to results more closely representing EMG data from another study. Finally, optimising a compliant model results in a total normalised muscle cost equal to that of the rigid case, increased robustness of the exo to configuration errors and increased comfort. This indicates that compliance does not have a detrimental effect on optimised performance when taken into consideration during optimisation.

This paper introduces a new imitation learning framework based on energy-based generative models capable of generating complex, life-like, physics-dependent motions, through state-only expert motion trajectories. Our algorithm, called Noise-conditioned Energy-based Annealed Rewards (NEAR), constructs several perturbed versions of the expert's motion data distribution and learns smooth, and well-defined representations of the data distribution's energy function using denoising score matching. We propose to use these learnt energy functions as reward functions to learn imitation policies via reinforcement learning. We also present a strategy to gradually switch between the learnt energy functions, ensuring that the learnt rewards are always well-defined in the manifold of policy-generated samples, thereby improving the learnt policies. We evaluate our algorithm on complex humanoid tasks such as locomotion and martial arts and compare it with state-only adversarial imitation learning algorithms like Adversarial Motion Priors (AMP). Our framework sidesteps the optimisation challenges of conventional generative imitation learning techniques and produces results comparable to AMP in several quantitative metrics across multiple tasks. Finally, a portion of this paper also analyses the optimisation challenges of adversarial imitation learning algorithms, and discusses some previously under-explored failure modes, providing rigorous empirical results to back our argumentation. Code and videos are available at anishhdiwan.github.io/noise-conditioned-energy-based-annealed-rewards/

Surgical navigation involves transferring preoperative imaging data, along with preplanned information, onto the patient in the operating theater without using constant radiation. This technique has proven effective and is widely adopted across various surgical specialties. Research has shown a consistent trend in surgical robotics, with numerous initiatives using this technology to navigate the robot’s end-effector within the patient’s anatomy. For this purpose, commercially available surgical navigation systems are often employed. However, these systems, which are primarily dominated by optical tracking, are not necessarily suited for robotic systems and exhibit limitations such as low update frequency and line-of-sight issues. Additionally, performance reporting in current surgical robotic research is highly inconsistent, and clear guidelines are lacking. This research aims to develop a surgical robotic navigation system to work towards establishing a performance benchmark and systematically assess various error components as a first step toward guiding the field of surgical robotic navigation. To this end, two systems, the Haply System and the Dual-Robot System, have been developed and evaluated for technical accuracy and registration accuracy in both static and dynamic environments. Furthermore, sensor fusion methods have been explored to enhance performance in the Haply system. The results and analysis indicate that the Dual-Robot System is the most accurate in dynamic navigation and presents a viable alternative to optical tracking systems in terms of performance. However, its clinical adoptability remains questionable.

In this thesis, we introduce a novel approach aimed at enhancing the jumping and landing capabilities of quadruped robots. Our method integrates both model-based and model-free strategies and features a behavioral cloning framework designed to reduce computational delays often encountered in trajectory optimization.

Initially, we build upon an existing framework for quadruped jumps, where we refine the trajectory optimization (TO) algorithm and introduce a new Variable Impedance Control (VIC). The VIC is specifically developed to facilitate softer landings. This improved system was then utilized to generate a comprehensive synthetic dataset, including 11,000 samples that cover a diverse range of jumping scenarios. This dataset served as the foundation for training a neural network. The primary objective of the network is to emulate the performance of the model-based approach. Structurally, the network is designed to process the robot's current state as input and generate the corresponding control actions for its 12 motors as output.

The most significant achievement of this research is the neural network's ability to closely replicate the outcomes of the model-based solution. Notably, it ensures more compliant behavior and lower stress on the motors during the landing phase than an MPC. The neural network demonstrates a 97.4% success rate. This high level of performance underscores its potential for on-the-fly application in robotic systems. The effectiveness of our method is further validated through a series of simulations and practical tests conducted on a Go1 quadruped robot.

This paper presents a hierarchical reinforcement learning framework for efficient robotic manipulation in sequential contact tasks. We leverage this hierarchical structure to sequentially execute behavior primitives with variable stiffness control capabilities for contact tasks. Our proposed approach relies on three key components: an action space enabling variable stiffness control, an adaptive stiffness controller for dynamic stiffness adjustments during primitive execution, and affordance coupling for efficient exploration while encouraging compliance. Through comprehensive training and evaluation, our framework learns efficient stiffness control capabilities and demonstrates improvements in learning efficiency, compositionality in primitive selection, and success rates compared to the state-of-the-art. The training environments include block lifting, door opening, object pushing, and surface cleaning. Real world evaluations further confirm the framework’s sim2real capability. This work lays the foundation for more adaptive and versatile robotic manipulation systems, with potential applications in more complex contact-based tasks.