Circular Image

L. Peternel

info

Please Note

61 records found

Journal article (2026) - H.A. Jekel, A. Díaz Rosales, L. Peternel
The paper presents a visio-verbal teleimpedance interface for commanding 3D stiffness ellipsoids to the remote robot with a combination of the operator’s gaze and verbal interaction. The gaze is detected by an eye-tracker, allowing the system to understand the context in terms of what the operator is currently looking at in the scene. Along with verbal interaction, a Vision-Language Model (VLM) processes this information, enabling the operator to communicate their intended action or provide corrections. Based on these inputs, the interface can then generate appropriate stiffness matrices for different physical interaction actions. To validate the proposed visio-verbal teleimpedance interface, we conducted a series of experiments on a setup including a Force Dimension Sigma.7 haptic device to control the motion of the remote Kuka LBR iiwa robotic arm. The human operator’s gaze is tracked by Tobii Pro Glasses 2, while human verbal commands are processed by a VLM using GPT-4o. The first experiment explored the optimal prompt configuration for the interface. The second and third experiments demonstrated different functionalities of the interface on a slide-in-the-groove task ...
This paper presents a multi-modal dynamic workspace re-indexing method for addressing operator ergonomics and workspace limitations. The proposed method has two interactive modes: pose-to-pose mode, which is active when the operator is within an ergonomic workspace of comfortable arm postures, and ergonomic workspace drift mode, which activates after the operator makes an excursion beyond the boundaries of the ergonomic workspace when trying to reach more distant targets with the remote robot. In the ergonomic workspace drift mode, the operator temporarily stays slightly outside these boundaries, while the offset between the local and remote workspace drifts with a velocity proportional to the excursion distance. This dynamically re-indexes the remote workspace toward the distant target, and the operator can remain in a comfortable posture while the remote robot moves toward the intended target where the task is. To construct the ergonomic workspace, we employed the Rapid Upper Limb Assessment method. To validate the proposed method, we conducted experiments on a teleoperation setup involving a Force Dimension Sigma7 haptic device controlling a Kuka LBR iiwa robotic arm. The results show that the proposed controller successfully addresses workspace limitations by dynamically reindexing the follower's workspace towards target objects, while maintaining good operator ergonomics. ...
Journal article (2025) - C. Nandkumar, L. Peternel
This paper presents the design and evaluation of a comprehensive system to develop voice-based interfaces to support users in supermarkets. These interfaces enable shoppers to convey their needs through both generic and specific queries. Although customisable state-of-the-art systems like GPTs from OpenAI are easily accessible and adaptable, featuring low-code deployment with options for functional integration, they still face challenges such as increased response times and limitations in strategic control for tailored use cases and cost optimization. Motivated by the goal of crafting equitable and efficient conversational agents with a touch of personalisation, this study advances on two fronts: 1) a comparative analysis of four popular off-the-shelf speech recognition technologies to identify the most accurate model for different genders (male/female) and languages (English/Dutch) and 2) the development and evaluation of a novel multi-LLM supermarket chatbot framework, comparing its performance with a specialized GPT model powered by the GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback. Our findings reveal that OpenAI’s Whisper leads in speech recognition accuracy between genders and languages and that our proposed multi-LLM chatbot architecture, which outperformed the benchmarked GPT model in performance, user satisfaction, user-agent partnership, and self-image enhancement, achieved statistical significance in these four key areas out of the 13 evaluated aspects that all showed improvements. The paper concludes with a simple method for supermarket robot navigation by mapping the final chatbot response to the correct shelf numbers to which the robot can plan sequential visits. Later, this enables the effective use of low-level perception, motion planning, and control capabilities for product retrieval and collection. We hope that this work encourages more efforts to use multiple specialized smaller models instead of always relying on a single powerful model. ...
Journal article (2025) - N. Mol, J.M. Prendergast, David Abbink, L. Peternel
In this letter, we investigate whether classical function allocation—the principle of assigning tasks to either a human or a machine—holds for physical Human-Robot Collaboration, which is important for providing insights for Industry 5.0 to guide how to best augment rather than replace workers. This study empirically tests the applicability of Fitts' List within physical Human-Robot Collaboration, by conducting a user study (N=26, within-subject design) to evaluate four distinct allocations of position/force control between human and robot in an abstract blending task. We hypothesize that the function in which humans control the position achieves better performance and receives higher user ratings. When allocating position control to the human and force control to the robot, compared to the opposite case, we observed a significant improvement in preventing overblending. This was also perceived better in terms of physical demand and overall system acceptance, while participants experienced greater autonomy, more engagement and less frustration. An interesting insight was that the supervisory role (when the robot controls both position and force) was rated second best in terms of subjective acceptance. Another surprising insight was that if position control was delegated to the robot, the participants perceived much lower autonomy than when the force control was delegated to the robot. These findings empirically support applying Fitts' principles to static function allocation for physical collaboration, while also revealing important nuanced user experience trade-offs, particularly regarding perceived autonomy when delegating position control. ...
Journal article (2025) - A.J. Becoy, K. Khomenko, L. Peternel, R.T. Rajan
This article proposes a novel method of coverage path planning for the purpose of scanning an unstructured environment autonomously. The method uses the morphological skeleton of a prior 2D navigation map via SLAM to generate a sequence of points of interest (POIs). This sequence is then ordered to create an optimal path based on the robot’s current position. To control the high-level operation, a finite state machine (FSM) is used to switch between two modes: navigating toward a POI using Nav2 and scanning the local surroundings. We validate the method in a leveled, indoor, obstacle-free, non-convex environment, evaluating time efficiency and reachability over five trials. The map reader and path planner can quickly process maps of widths and heights ranging between [196,225] pixels and [185,231] pixels in 2.52ms and 1.7ms, respectively. Their computation time increases with 22.0ns/pixel and 8.17 μs/pixel, respectively. The robot managed to reach 86.5% of all waypoints across the five runs. The proposed method suffers from drift occurring in the 2D navigation map. ...
Journal article (2025) - Kevin Haninger, L. Peternel
For successful goal-directed human-robot interaction, the robot should adapt to the intentions and actions of the collaborating human. This can be supported by musculoskeletal or data-driven human models, where the former are limited to lower-level functioning such as ergonomics, and the latter have limited generalizability or data efficiency. What is missing, is the inclusion of human motor control models that can provide generalizable human behavior estimates and integrate into robot planning methods. We use well-studied models from human motor control based on the speed-accuracy and cost-benefit trade-offs to plan collaborative robot motions. In these models, the human trajectory minimizes an objective function, a formulation we adapt to numerical trajectory optimization. This can then be extended with constraints and new variables to realize collaborative motion planning and goal estimation. We deploy this model, as well as a multi-component movement strategy, in physical collaboration with uncertain goal-reaching and synchronized motion tasks, showing the ability of the approach to produce human-like trajectories over a range of conditions. ...
Combining biomechanical modeling with robotic physiotherapy is a promising direction to provide real-time insights during the rehabilitation of patients with musculoskeletal injuries, such as rotator-cuff tears. One aspect is to prevent re-injuries caused by high strain in the injured tissues while allowing patients to perform the required rehabilitation exercises. In this paper, we propose a novel shared control method for robots to limit unsafe patient movements, through physical guidance based on a strain-space representation of the human rotator cuff. The method provides motion corrections through two complementary predictive modules. The first module exerts a lower degree of intervention and is analogous to rumble strips or speed bumps for cars on the road. In this case, an impedance controller induces variable damping to slow down the patient's movement when a danger zone is approached. The second module produces a higher degree of intervention and is analogous to lane-assist in cars. In this case, the robot plans an optimal deflection trajectory and temporarily takes over control of the movement to avoid an unsafe situation. We performed experiments with a healthy participant acting as a patient and evaluated the effect of different human-robot interaction modalities on the resulting human movement in terms of avoidance of high-strain areas of the rotator-cuff tendons and contact forces exchanged. ...
Rhizome 1.0 and 2.0 are European Space Agency (ESA) co-funded projects that have been implemented with a team from the Architecture, Mechanical Engineering, and Aerospace Engineering Faculties, TU Delft, and various industrial partners. The focus is on the development of a Martian habitat using 3D-printed components and in situ resource utilization. This paper presents a new multi-modal method developed for the collaborative assembly of building components with the support of Computer Vision (CV) and Human-Robot Interaction (HRI) using compliant robotic collaborative manipulators. The building components are Voronoi-based and are fabricated using Design-to-Robotic-Production and -Assembly (D2RP&A). During the collaborative assembly, the robot uses CV to detect the fabricated component and generate autonomous actions to perform the pick-and-place movement. Between the autonomous robot actions, HRI is used by the human to physically guide the robot when grasping and positioning. To evaluate the proposed method, lab experiments were conducted using robotically milled mock-up components from Styrofoam, which were assembled with a collaborative robotic arm. The results indicate that robots can assist humans during the assembly process to implement tasks that are beyond their physical abilities, while robots benefit from human cognitive capabilities during more complex actions. ...
Despite recent advancements in physical humanrobot collaboration, measuring and distinguishing between forces applied by humans and robots remains challenging, limiting our understanding of force dynamics during collaboration. Our proposed solution addresses this gap with a low-cost, lightweight design that integrates directly at the robot endeffector level. The interface employs a three-ring mechanical structure with strategically positioned load cells and a Sarrus mechanism to constrain movement to the z-axis only, enabling tool mounting for real-world collaborative tasks such as blending or sanding operations. Validation experiments demonstrate excellent force decoupling capabilities with minimal crossinterference, achieving Weighted Root Mean Squared Errors of 0.14 N for robot-applied forces and 0.08 N for human-applied forces compared to ground truth measurements in steadystate for loads ranging from 0 N up to 23 N. The Maximum Absolute Error in these experiments is 0.33 N, confirming high measurement accuracy. This affordable and integrated solution lowers the threshold for employing decoupled force sensing in collaborative tasks, making it more accessible for investigating force dynamics and developing adaptive control strategies in both research and practical applications of physical humanrobot collaboration. ...
Conference paper (2025) - Arwin Hidding, Tom Lim, Henriette Bier, Luka Peternel
Constructing a Martian habitat presents significant challenges due to extreme temperature variations and a low-density and -pressure atmosphere. To address these challenges a habitat constructed from prefabricated, interlocking Voronoi-based components that are assembled by human-robot collaboration has been explored in the Rhizome projects at TU Delft. In this paper, we propose a combined robot motion planning and learning method that can optimize human involvement in assembly tasks in on-site construction. The proposed hybrid approach exploits motion planning to create motion trajectories for aspects of the task where robot autonomy is capable of solving the problem on its own using sensors and intelligence. When the task becomes too difficult for existing planning capabilities, the human can step in and teach motion trajectories via kinaesthetic demonstration using Dynamic Movement Primitives (DMPs). The trajectories are then executed on the low level by an impedance controller to handle the physical interaction with the environment during the assembly. The decision-making process is managed by a behavior tree. ...

Requirements, Method, and Test With Handover Task

Journal article (2024) - Emma M. Van Zoelen, Hugo Veldman-Loopik, Karel van den Bosch, Mark Neerincx, David A. Abbink, Luka Peternel
Despite a large body of research on robot learning, it has not yet been thoroughly studied how collaborating humans and robots learn reciprocally. In such situations, both humans and robots continuously learn about each other and the task through interaction. This letter addresses the research question: "How can human-robot co-learning be facilitated in physically embodied collaborative tasks?". First, we derived five requirements for successful human-robot co-learning from literature: shared goal, synchrony, interdependence, adaptability, and transparency. Based on these requirements, we designed a collaborative human-robot handover task and a robot Q-learning method. In an evaluation with six human participants co-learning was indeed found to emerge in the hand-over task. Particularly, for three of the human-robot dyads, our designed setup proved to facilitate co-learning in a way that met all five requirements. The task and robot learning method presented in this letter demonstrate how human-robot co-learning can be enabled in physically embodied tasks. ...

Insights on orientation and impedance

Journal article (2024) - Fares Abu-Dakka, Matteo Saveriano, Luka Peternel
Many daily tasks exhibit a periodic nature, necessitating that robots possess the ability to execute them either alone or in collaboration with humans. A widely used approach to encode and learn such periodic patterns from human demonstrations is through periodic Dynamic Movement Primitives (DMPs). Periodic DMPs encode cyclic data independently across multiple dimensions of multi-degree of freedom systems. This method is effective for simple data, like Cartesian or joint position trajectories. However, it cannot account for various geometric constraints imposed by more complex data, such as orientation and stiffness. To bridge this gap, we propose a novel periodic DMP formulation that enables the encoding of periodic orientation trajectories and varying stiffness matrices while considering their geometric constraints. Our geometry-aware approach exploits the properties of the Riemannian manifold and Lie group to directly encode such periodic data while respecting its inherent geometric constraints. We initially employed simulation to validate the technical aspects of the proposed method thoroughly. Subsequently, we conducted experiments with two different real-world robots performing daily tasks involving periodic changes in orientation and/or stiffness, i.e., operating a drilling machine using a rotary handle and facilitating collaborative human–robot sawing. ...
Journal article (2024) - A. van den Berg, Jelle Hofland, Cock J.M. Heemskerk, David Abbink, L. Peternel
In recent years, providing additional visual feedback about the interaction forces has been found to offer benefits to haptic-assisted teleoperation. However, there is limited insight into the effects of the design of force feedback-related visual cues and the type of visual display on the performance of teleoperation of robotic arms executing industrial tasks. In this study, we provide new insights into this interaction by extending these findings to the haptic assistance teleoperation of a simulated robotic arm in a virtual environment, in which the haptic assistance is comprised of a set of virtual fixtures. We design a novel method for providing visual cues about the interaction forces to complement the haptic assistance and augment visual feedback in virtual reality with a head-mounted display. We evaluate the visual cues method and head-mounted display method through human factors experiments in a teleoperated dross removal use case. The results show that both methods are beneficial for task performance, each of them having stronger points in different aspects of the operation. The visual cues method was found to significantly improve safety in terms of peak collision force, whereas the head-mounted display additionally improves the performance significantly. Furthermore, positive scores of the subjective analysis indicate an increased user acceptance of both methods. This work provides a new study on the importance of visual feedback related to (interaction) forces and spatial information for haptic assistance and provides two methods to take advantage of its potential benefits in the teleoperation of robotic arms. ...
Journal article (2024) - Tjasa Kunavar, Marko Jamšek, Edwin Johnatan Avila-Mireles, Elmar Rueckert, Luka Peternel, Jan Babič
During the learning of a new sensorimotor task, individuals are usually provided with instructional stimuli and relevant information about the target task. The inclusion of haptic devices in the study of this kind of learning has greatly helped in the understanding of how an individual can improve or acquire new skills. However, the way in which the information and stimuli are delivered has not been extensively explored. We have designed a challenging task with nonintuitive visuomotor perturbation that allows us to apply and compare different motor strategies to study the teaching process and to avoid the interference of previous knowledge present in the naïve subjects. Three subject groups participated in our experiment, where the learning by repetition without assistance, learning by repetition with assistance, and task Segmentation Learning techniques were performed with a haptic robot. Our results show that all the groups were able to successfully complete the task and that the subjects’ performance during training and evaluation was not affected by modifying the teaching strategy. Nevertheless, our results indicate that the presented task design is useful for the study of sensorimotor teaching and that the presented metrics are suitable for exploring the evolution of the accuracy and precision during learning. ...
Conference paper (2024) - A. Díaz Rosales, Jose Rodriguez-Nogueira, Eloise Matheson, D.A. Abbink, L. Peternel
Teleoperation is a crucial technology enabling human operators to control robots remotely to perform tasks in hazardous and difficult-to-access environments. Tasks in such environments often involve complex physical interactions with tools and objects of various softness. To this end, teleimpedance enables the operators to adjust the robot impedance in real-time to simplify such interactions. While the existing teleimpedance approaches provide several interfaces to command the robot impedance, there are no interfaces to visualize both the commanded impedance and that of the objects to be interacted with. This paper presents a novel interface to provide visual feedback on the impedance of remote robots and objects. To do so, we use virtual stiffness ellipsoids and different modes that display the individual impedance of the robot and objects as well as combined post-contact impedance. The key advantage of visual feedback on the impedance compared to force feedback is that the operator can see the interaction characteristics before the contact occurs. This enables the operator to act proactively before contact rather than just reactively after the contact. This paper also proposes a new intuitive way to command the robot impedance using mixed reality, interacting with these ellipsoids and modifying them as needed. To demonstrate the key functionalities of the developed interface, we performed proof-of-concept experiments on teleoperated tasks. ...
Conference paper (2024) - F.M.C. Kraakman, L. Peternel
In this paper, we present a design and evaluation of a novel finger-operated teleimpedance interface used to command stiffness ellipsoids to the remote robot. The proposed interface provides a practical alternative to the state-of-the-art teleimpedance interfaces based on physiological signals that can be impractical in daily use. On the other hand, as opposed to existing practical interfaces that lack in terms of controlled degrees of freedom, the proposed interface enables control of 3D aspects of the ellipsoid. The remote robot stiffness ellipsoid is controlled with a single hand using the thumb, index, and middle fingers to operate two scroll wheels, a joystick, and a force sensor. These combinations of inputs can be mapped to control different aspects of the stiffness ellipsoid, i.e., orientation and shape/size. To investigate different modes of input mapping, we perform a human factors experiment to evaluate the performance and user acceptance of the proposed interface modes. The results of the experiments indicate that the participants can successfully operate the interface to complete 3D stiffness configuration alignment tasks in different modes. To further demonstrate the functionality of the proposed teleimpedance interface, we performed an additional experiment utilising a Force Dimension Sigma7 haptic device to control the motion of a KUKA LBR iiwa robotic arm while performing a complex physical interaction task. ...
In this work, we propose a method of capturing the patient’s discomfort during robotic shoulder physiotherapy, creating "discomfort maps". These maps depict the personalized distribution of discomfort that each patient perceived across their shoulder range of motion, facilitating both robotic devices and human therapists to account for patient-specific characteristics during the therapeutic process. Our system enables a patient to communicate and map discomfort in space and time during movement via a handheld push-button device, while interacting with a robotic physical therapy device capable of moving the patient and estimating their pose. We validated our method through human factors experiments simulating shoulder physiotherapy sessions with 10 healthy participants. To avoid the risk of injury to the participants and to allow for ground truth map information, we emulate perceived discomfort via an auditory signal. Our experimental apparatus enabled participants to reconstruct synthetic discomfort maps, demonstrating the feasibility of automatically capturing and storing patient discomfort during robotic physiotherapy. ...
In the original version of the book, on page xi, one of the authors listed for Chapter 2 is “R. Schnmehl”, which should be “R. Schmehl”. On page 21, the same correction needs to be made twice, in the listed authors at the top of the page and also in the footnotes. “Schnmehl” should become “Schmehl” in both the cases. This has now been rectified and the author’s name has been corrected. The correction to this book have been updated with the changes. ...
Conference paper (2024) - G. Siegemund, A. Díaz Rosales, Arne Glodde, Franz Dietrich, L. Peternel
This paper presents a method for semiautonomous teleimpedance where the control is shared between the human operator and the robot. The human commands the position of the teleoperated robotic arm end-effector while the robot autonomously adjusts the impedance depending on the object with which the end-effector interacts. We developed a vision system that calculates the appropriate robot stiffness based on the detected object geometry and material and object’s relation to the environment. This system uses an RGB-D camera near the robot’s end-effector to capture different perspectives of the scene. To validate the proposed method, we conducted experiments on a teleoperation system where a Force Dimension Sigma7 haptic device was used to operate a KUKA LBR iiwa robotics arm. At the same time, the Intel RealSense D455 depth camera provided the visual input. We examined two practical tasks: engaging with bolts on a plate and polishing a stripe. ...