This study examines how fixed robot personalities (patient, impatient, leader, follower) influence co-learning in human-robot teams by answering the research question: How do different robot personalities influence co-learning. To do this, we implemented a reinforcement learning framework for a handover task where a robot and human participant co-learn to solve a task. The robot has personalities encoded along two axes: patient/impatient (via motion speed and stiffness) and leader/follower (via exploration rates and reward structures in phased Q-learning).

Through a within-subject design, we analyze policy metrics and human perceptions. While task success rates remain stable, strategy and internal policy metrics vary significantly. This underpins the key finding: robot personality does not affect task performance since humans can adapt to overcome subtle differences in robot personality. However, robot personality significantly affects how the collaboration is performed as human-robot teams adopt different strategies for different robot personalities.

Results demonstrate that robot personality is salient for differences in physical behaviour yet is unperceivable for modifications of internal parameters like exploration rate/decay and reward function for short interactions. This work bridges a critical gap in understanding how static robot traits shape collaborative adaptation, even when overt performance metrics remain unchanged.

Visual place recognition (VPR) is a form of visual localization. Current approaches are designed to handle common VPR challenges, such as appearance and viewpoint variations. With the introduction of DINOv2, vision foundation models have been used as feature extractors to improve performance for VPR techniques, as they show great generalizing capabilities for image representations. By fine-tuning these large models on VPR-specific datasets, performance increases even more. A problem with these big VPR datasets is the bias towards urban environments. To solve this problem, we propose to use a simple pipeline to fine-tune existing techniques on the reference databases of test datasets. Our experiments show that performance improves by reference database fine-tuning for multiple techniques on different datasets. To handle appearance and viewpoint variations as well, image augmentations can be used during training. With this complete pipeline, techniques improve performance. The experiments show improvement even if a large query-reference domain gap exists for that dataset given that a part of the test queries are know during fine-tuning.

Despite rapid advancements in Large Language Models (LLMs), they often produce hallucinated or detrimental outputs, necessitating alignment with human preferences. We address these challenges by introducing Step Chain-of-Thought (SCoT) to enhance semantic understanding by breaking down complex instructions. Additionally, we combine Direct Preference Optimization (DPO) with Low-Rank Adaptation (LoRA) to improve alignment with user intent. DPO optimizes outputs based on human feedback, while LoRA, alongside careful tuning of learning rates and beta values, mitigates repetition issues seen with DPO alone. Our findings show that models fine-tuned with DPO with LoRA achieve superior alignment compared to those using only Supervised Fine-Tuning (SFT). However, automated evaluators like LLM-as-a-Judge struggle with nuanced SCoT assessments, underscoring the necessity of human evaluation for capturing the complexities of alignment. In task alignment for robotics, Full Fine-Tuning (FFT) excels in familiar tasks, while LoRA significantly improves adaptability to new scenarios, increasing the robustness. Moreover, combining ground truth with synthetic data, especially when using LoRA, achieves a balance between accuracy and adaptability, revealing the limitations of relying solely on synthetic data. These conclusions highlight the critical importance of well-aligned datasets, fine-tuning strategies, and careful parameter tuning for LLM alignment.

The challenge of navigating uneven terrain is a critical obstacle in the advancement of robotic
locomotion. Traditional quadrupedal locomotion methods, such as walking, are often
insufficient for dynamic and complex environments. Agile skills like jumping are necessary and
must be adaptable over uneven terrain. This study addresses this issue by developing a policy
for executing jumps over uneven terrain using a single demonstration. Initially, the system
learns to imitate a forward jump based on a single demonstration from a SLIP trajectory
planner. It then generalizes its jumping abilities to various distances in both forward and lateral
directions. The study compares the performance of systems with and without parallel elasticity,
demonstrating the energetic benefits of using elastic actuation for quadrupedal jumping.
Results show that the system with parallel elastic actuation is 15.20% more energy-efficient
and experiences a 15.79% reduction in peak power compared to the system without parallel
elasticity. A policy trained using the proposed methodology successfully performs jumps of
variable distances over uneven terrain with height perturbations of +/-4 cm using only
proprioceptive information.

Previous work has shown that state abstraction can be an efficient way to plan in robotic environments with continuous actions and long task horizons. Although some of these works learn predicates for state abstractions, they often neglect an important part needed for generalization. Namely objects and their properties, our work shows that object properties can be exploited to increase the generalizability of learned models. We do this by extending a framework based on predicates for state abstraction and introducing affordances. Affordances allow us to group objects in a way that also considers possible actions. In order to learn affordance interpretations we make use of querying. First, the agent is tasked with solving a task made to test a specific affordance, then based on its uncertainty about an object's affordance interpretation it may query an expert. For example, when presented with a task where it needs to pick up an object it may query the expert on whether the object is actually grippable. We compare our approach with a baseline without affordances and show that our approach needs fewer operators to plan and that our approach is able to generalize to novel objects.

To offer engaging neurorehabilitation training to stroke patients, robotic exoskeletons have emerged as a beneficial tool to train motor tasks. However, in robotic therapy, the therapists are often limited in how they can interact with the patients. The training programs with exoskeletons are often standardized, not allowing the therapist to directly manipulate them and customize the training for specific patient needs. This paper presents a novel telerehabilitation system incorporating a therapist-in-the-loop haptic interface and immersive virtual reality (IVR), aimed at improving the interaction dynamics between patients and therapists during rehabilitation sessions. We conducted a human factors study involving 36 participants grouped in pairs, assigned as "Leaders" or "Followers," where the Leader had to guide the Follower (positioned in the exoskeleton) to a specific arm pose related to activities of daily living (ADL). This task was done under two conditions: demonstration and teleoperation. In the demonstration condition, the Follower could see the Leader displayed on a 2D screen in front of them showing them visually what was the correct pose whereas in the teleoperation condition, there was no visual aid but the Leader could guide the Follower using the haptic device. The study explored the efficiency, workload, and user experience of both modalities. Results indicated that teleoperation significantly reduced the time to complete tasks and facilitated more efficient and smoother movement patterns. While Leaders experienced reduced workload and increased motivation with the teleoperation condition, Followers reported less workload but also decreased motivation, suggesting a trade-off between task ease and engagement. In conclusion, the teleoperation system improves the ability for the therapist to interact with the exoskeleton but can disengage the patient.

With the current transition towards renewable and high-tech solutions, the world is becoming increasingly complex. Consequently, the challenges faced by firefighters also intensify. For that reason, firefighting robots are rising in popularity despite being far from perfect. An important area of improvement is the perception capabilities of those robots, given the fact that firefighting robots suffer from occluded camera views in environments filled with smoke. To overcome this challenge a LiDAR sensor may be used but experiments in this work show that even those point clouds are adversely affected by smoke. Consequently, this work presents a method for real-time reconstruction of ground surfaces in occluded environments filled with smoke. The developed method functions in ROS Noetic and merges segmented ground points, when available, with ground surfaces which are reconstructed based on information from segmented wall points. In this way, the method works even without the presence of ground points. To achieve this, a combination of established techniques from scientific literature, along with newly developed techniques were implemented. Doing so gives the robot’s operator an improved representation of the ground surface within environments filled with smoke. Ultimately the developed method may allow for autonomous navigation based on LiDAR data within environments filled with smoke. This research shows that a method consisting of techniques which tackle the independent sub-challenges arising from the use of LiDAR in indoor environments filled with smoke can effectively reconstruct the ground surfaces within those environments. Furthermore, the developed method has the potential to do so in a real-time manner.

Monitoring expeditions in endangered habitats are currently performed by human experts. However, this approach has several disadvantages, including the limited amount of experts, cost-intensive expeditions, and the dangers that are posed by exploring dangerous terrains. Therefore, one can look into using quadrupedal legged robots that would collaborate with the human operators, which would be able to assist them by performing extra measurements in these dangerous habitats. An open problem in quadrupedal legged locomotion is robust periodic forward jumping, a.k.a. pronking, specifically for quadrupedal robots that have flexible joints placed in series. In this paper, we therefore propose a novel framework that generates an energy efficient pronking motion for a quadrupedal series elastic actuated legged robot. This periodic pronking motion is generated using a reduced order model and the serial elasticity of the joints is taken into account using an template-anchor approach.
To minimize the trajectory error we use Functional Iterative Learning Control (fILC) as feedforward control in parallel with a proportional-derivative feedback controller. The advantage of using fILC is that the elasticity of the quadrupedal robot is preserved and that the controller is able to learn the pronking motion in a small number of iterations. This framework is validated on an eight degrees of freedom series elastic actuated robot. In- and outdoor experiments show that this framework is able to work in unknown terrains.

Soft robots are characterized by compliant elements that introduce heightened kinematic complexity compared to their rigid counterparts. Such systems, with infinite degrees of freedom, are inherently underactuated, making precise real-time shape regulation a challenging task. Model-based controllers, utilizing tractable reduced-order modelling methods, have emerged as promising solutions. However, practical implementations of these methods often rely on fully-actuated approximations, overlooking the underactuated nature of these continuum structures. In this study, we aim to experimentally validate model-based controllers that explicitly account for underactuation, surpassing the theoretical feasibility demonstrated in simulation. These controllers incorporate gravity cancellation and compliance compensation using the dynamic model of the robot to achieve superior real-time shape regulation compared to conventional PD/PID controllers. To facilitate this experimental validation, we have built a multi-segment soft robot research platform that includes a passively actuated segment, allowing for the utilization of both actuated and unactuated degrees of freedom in the control feedback loop. Through rigorous experimentation, we provide comprehensive evidence of the efficacy of this class of model-based controllers in controlling unconventionally actuated robotic systems. Consequently, our work bridges the gap between theory and practice, resulting in a practical real-time shape regulation framework that is adaptable to a vast variety of soft robotic systems.

Task and Motion Planning (TAMP) has progressed significantly in solving intricate manipulation tasks in recent years, but the robust execution of these plans remains less touched. Particularly, generalizing to diverse geometric scenarios is still challenging during execution. In this work, we propose a reactive TAMP method to deal with disturbances and geometric ambiguities by combining an active inference planner (AIP) for online action selection with a proposed multi-modal model predictive path integral controller (M3P2I) for low-level control. The AIP generates online alternative plans, each of which is translated into a cost function to be sampled for the proposed method. The proposed M3P2I then uses a parallelizable physics simulator for throwing different rollouts, leading to a coherent optimal solution by averaging the weighted samples based on their costs. Our method empowers real-time adaptation of action sequences to rectify failed plans, while also computing low-level motions to address dynamic obstacles or disturbances that could potentially invalidate the existing plan.

Theoretical findings are validated in simulation and in the real world. We show that the framework exhibits reactiveness in different scenarios, including battery charging, push-pull among obstacles, and pick-place with disturbances. We show that our framework outperforms an off-the-shelf RL method in the reactive pick-place task in terms of position error and orientation error. We also show that M3P2I is generalizable in combining different constraints, such as generating hybrid motions of push and pull for the mobile robot, and grasping objects with different grasping poses. The real-world experiments show that the system exhibits reactiveness and robustness against human disturbances in a variety of manipulation tasks.

The supporting videos can be found at https://sites.google.com/view/m3p2i-aip.

Intelligent manufacturing has become increasingly important in the food packaging industry due to the growing demand for enhanced productivity and flexibility while minimizing waste and lead times. This work explores the integration of such manufacturing in automated secondary robotic food packaging solutions that transfer food products into containers using pick-and-place robots. A major problem in these solutions is varying product supply caused by prior machinery. As a result productivity drops drastically when conveyor belt speeds are not optimally controlled. Conventional heuristic-based engineered approaches are used to address this issue but are inadequate, leading to noncompliance with industry's requirements. Reinforcement learning, on the other hand, has the potential of solving this problem by learning quick and predictive decision-making behavior based on experience. However, the lack of research in reinforcement learning for complex industrial robotic problems, limits its adoption in industry. Therefore, this work aims to investigate the feasibility of reinforcement learning in the robotic packaging industry. We propose a reinforcement learning framework, with policy inference in a highly complex control scheme, designed to optimize the conveyor belt speed of the secondary robotic packaging solution using real-world product supply data. The framework exceeds the 99.8 percent performance requirement and maintains quality at the required 100 percent when tested on real-world data. Compared to the current heuristic-based solution, our proposed framework improves productivity, has smoother control and reduces code execution time.

Unmanned Underwater Vehicles (UUVs) operate in complex environments and need to be able to adapt to sudden failures, or changes in the environment. To achieve autonomous operation, UUVs must have the ability to self-adapt in such cases. To effectively handle component failures and unexpected events, self-adaptation must be applied to both the architecture and task plan of the UUV. This allows the UUV to modify its architecture to accommodate component failures and adjust its task plan in response to unforeseen events that may render the current plan infeasible. The mutual dependencies between architectural adaptation and task planning pose a significant challenge when determining how to apply adaptation. As a result, the task planner must take into account the implications of architectural adaptation when generating a plan. This paper proposes Metaplan, a modular ROS2-based framework for applying architectural and task plan co-adaptation in a reusable way. Metaplan extends Metacontrol, an architectural self-adaptation framework, with a task planner based on Planning Domain Definition Language (PDDL). The effectiveness of Metaplan is demonstrated by evaluating it on SUAVE, an exemplar for evaluating self-adaptation frameworks for UUVs. Metaplan is shown to outperform a baseline which only makes use of a task planner. Architecture and task plan co-adaptation is demonstrated by presenting the UUV with a sudden drop in battery level, requiring the UUV to adapt both its architecture and its task plan. Furthermore, the reusability of Metaplan is showcased by applying it to a mobile manipulator scenario.

Recent research has shown that a Learning from Demonstration (LfD) approach is useful for teaching robots flexible skills efficiently, and it opens the possibility for non-expert users to program these skills. When learning from demonstration data, learning frameworks should learn representations that are flexible and can generalize to unseen situations. Within the context of multi-reference frame skill learning, this work proposes a framework to learn such a representation without using task-specific heuristics or pre-segmentation of the demonstrations. Local policies are first learned by fitting the local dynamics with respect to each frame using Gaussian Processes (GP). A classifier that determines the relevance of each frame for every time step is then trained in a self-supervised manner. The uncertainty quantification capability of Gaussian Processes is exploited to improve the performance of the local policies and the self-supervised learning process of the classifier. The framework is validated through multi-frame tasks in simulation as well as on a robotic manipulator with a pick-and-place re-shelving task. Its performance is also compared to that of the Task-Parameterised Gaussian Mixture Model (TPGMM) with simulated and robotic data. In this comparison, the proposed model performs better according to metrics that quantify deviation from the goal at each reference frame and according to similarity measures between demonstrations and their corresponding reproductions.

Many recent robot learning problems, real and simulated, were addressed using deep reinforcement learning. The developed policies can deal with high-dimensional, continuous state and action spaces, and can also incorporate machine-generated or human demonstration data. A great number of them depend on state-action value estimates, especially the ones in the actor-critic framework. Deriving unbiased estimates for these values is still an open research question, mostly since the connection between accurate value estimates and system performance is not yet well-understood. This thesis work has three main research contributions. Firstly, it analyzes the connection between value estimates and performance for the TD3 algorithm. Secondly, it derives theoretical bounds for the true value function when dealing with environments where a reward is only given for successful completion of a task (sparse/binary reward). Lastly, a deliberate underestimation objective is added to the TD3 algorithm together with the theoretical bounds to improve system performance when using human demonstration data that only covers a specific part of the state and action space. All the algorithms are tested and evaluated using simulated robot manipulation tasks in the robosuite environment, where the robot is first trained on the demonstration data and then can gather more experiences in the simulation. Results show that the deliberate underestimation together with the value bounds enable the robot to learn from human demonstration, which was not possible for the standard TD3. Additionally, applying just the value bounds speeds up the learning process when using machine-generated datasets.

During the transfer of calli in plant regeneration, the repetitive work of pick and placement of calli in new agar is still done by human operators. This process can be automated to eliminate labour-intensive work. The thesis focuses on the development of a robotic gripper to automate the transfer of calli. To be able to grip calli, first its physical properties have to be investigated. With different weights, the maximum allowable force before calli damage was determined.
Subsequently a methodological design process was performed, which consisted of formulating system requirements and conceptual design analysis with the use of the Kesselring method. In order to finalize the design, the two highest scoring gripper concepts were chosen to be further explored. Both concepts were transformed in a 3D design and printed to validate its effectiveness for gripping calli. Of those two concepts, the best performing gripper was further optimized and used as prototype for additional validation. To validate this new gripper prototype, calli of different shapes and sizes were transferred to a new petri dish. To correctly grip the calli, a script was created to adequately detect and determine the size of the calli. The plastic prototype had high slack and inaccuracies due its the material. Therefore, the prototype was made of aluminium, with the use of Computer Numerical Control (CNC). Experiments revealed that the aluminium prototype had a success rate of 98% when gripping calli. In conclusion, the designed aluminium gripper was able to grip and release the calli without damage and has therefore great potential to automate calli transfer and thereby improve the quality and quantity of plant regeneration.

Pile driving is a widely used technique for the construction of buildings and infrastructure. A popular technique is to vibrate the pile into the sediment. However, since building sites are increasingly being located in metropolitan areas, there is a growing concern about the environmental impact that vibrations may cause during driving. Therefore, pile driving assessment and prediction have become important for several reasons, including reducing the risk of damage to nearby structures and limiting disturbance to adjacent properties. In particular, pile driveability (i.e., the penetration rate) assessment is immensely useful prior to installation as it increases the construction performance and subsequently reduces costs and environmental impact.

The important factors influencing the penetration rate include vibrator characteristics, pile properties, and soil conditions. However, due to assumptions and the lack of methods that accurately represent the complex phenomena at play during vibratory driving, a disparity is obtained between the predictions of modern pile behavior programs and the observed penetration rate.

Recently, the registration of pile driving data has increased significantly. This extended amount of measurement data can potentially be leveraged for an improvement of the prediction of the penetration rate in future projects. Literature review on the application of machine learning (ML) within pile driving, geotechnical engineering and drilling revealed that the artificial neural network (ANN) is a promising alternative method for the prediction of the driveability of vibratory driven piles (i.e., vibro-driveability).

In this work, machine learning methods and the traditional model were utilized to predict vibro-driveability. Promising ML techniques include the multilayer perceptron neural network (MLPNN) and radial basis function neural network (RBFNN). These neural networks were trained with the particle swarm optimization (PSO) algorithm. The backpropagation (BP) algorithm was also incorporated to train the MLPNN and RBFNN models as a conventional method. Based on results obtained with the aforementioned methods, we propose a new model, the Vibratory Driveability (VD) model, that combines the fruitful characteristics of the MLPNN and RBFNN.

The performance of the five different models was compared with the performance of contemporary vibro-driveability prediction software for three test sets. This was done using different performance indices including the mean squared error (MSE), mean absolute error (MAE) and the weighted average percentage error (WAPE). Additionally, the desired characteristics of the predictions based on the geo-engineer's input were examined and compared with the obtained predictions. It was demonstrated that the ANN-based methods achieved drastic improvements in prediction performance and consequently outperformed the traditional model, making ANN-based methods the preferred alternative for the prediction of vibro-driveability. Among the ANN models, the VD model produced the highest performance, as it reflected the desired prediction behavior for all three test cases and showed competitive prediction performance in terms of the performance metrics.

This work leads to the first-ever published research on the application of artificial neural networks for the prediction of vibro-driveability. As such, it could form the foundation for the development of new (vibratory) pile driving behavior assessment and prediction software. The development of these commercial applications could lead to a considerable reduction in costs and environmental impact.

In this work, we propose a method for monitoring and management of rotator-cuff tendon strains in human-robot collaborative physical therapy for rotator cuff rehabilitation. The proposed approach integrates a complex offline biomechanical model with a collaborative, industrial robot arm and an impedance controller. The model is used for computing rotator-cuff tendon strain as a function of human shoulder configuration, muscle activation and external forces. This subject- and injury-specific data is stored in \textit{strain maps}, which represent the relationship between the strains and shoulder DoFs. In our previous work, we implemented strain maps to preplan minimal strain, safe trajectories using two shoulder DoFs, and used the corresponding robot-mediated movement for passive trajectory following for healthy subjects. This work expands on that by implementing two novel functionalities: 1) patient-led movement, and 2) adding the third shoulder DoF and the corresponding control complexities, while still controlling for safe rotator-cuff tendon strains. For patient-led movement, we precomputed unsafe zones for each strain map by clustering and fitting ellipses to the clusters. These unsafe areas with increased risk of (re-)injury are then used to set the impedance control parameters and reference pose for real-time biomechanical safety control. By linearly interpolating between strain maps, smooth and safe movement of the third shoulder DoF is added. The resulting robot control torques guide the patient away from unsafe, high strain shoulder poses in real-time during patient-led movement. The proposed method has the potential to improve the safety, Range of Motion, and muscle activity that the patients receive through robot-mediated physical therapy. The main advantage of this approach is that the patient is free to use and explore their full shoulder RoM, while the robot controls and manages biomechanical safety in real-time. To validate the proposed method, we performed two experiments showcasing two novel functionalities, and a third experiment as proof-of-concept displaying the full method.

Manipulating soft and fragile objects is a challenging task in robotic grasping. The key challenge for robotic grasping is to exert enough grip force to prevent slipping while being gentle enough to prevent damage to an object. Existing grippers used for processes like automatic harvesting of fruits, either apply excessive grip force leading to object damage or react to slip resulting in object release from the gripper. The aim of this study is to develop a grip force controller that uses tactile feedback to maintain a constant frictional safety margin over the minimum required grip force, called Safety Margin Control. Tactile sensors can provide information on friction, which is used to predict slip. An optical tactile sensor is modeled and used in simulations where Safety Margin Control regulates the grip force during interaction with various virtual objects. The deformation of the sensor’s soft viscoelastic membrane is described by local frictional behavior and used to estimate the safety margin. The desired safety margin is set to 30%, based on comparison to the way humans control grip force in their fingertips. The desired value can be tuned to favor release over damage and vice versa. Safety Margin Control is compared to two baseline controllers: React To Slip and Conservative Control. The performance is evaluated based on maximum pressure and total lateral displacement of the object relative to the sensor. Safety Margin Control results in a pressure decrease of 44% on average compared to Conservative Control, and no significant pressure change was observed compared to React To Slip. The total lateral displacement for Safety Margin Control is 0 mm, as opposed to 1.3 mm for React To Slip. Safety Margin Control provides a way forward for automated harvesting as the pressure exerted on an object can be reduced while no slip occurs.

Soft robots are made of compliant materials, which increase their flexibility but also presents modeling challenges. The difficulty mainly comes from material nonlinearity, infinite degrees of freedom, uncertain parameters, and complex calculations. This project uses physics-inspired neural networks to solve the last two problems. Based on the piece-wise constant curvature approximation, this work modifies Lagrangian and Hamiltonian neural networks for dynamic modeling of soft robots. In particular, local linear damping and actuator models are added to deep Lagrangian and Hamiltonian neural networks, and a one-step integration algorithm is used in the loss calculation. This project justifies the modified network structure and uses these new neural networks to learn the dynamic model of soft manipulators.