D.A. Abbink
Please Note
70 records found
1
Legibility-Optimizing Trajectory Modulation for Mobile Robots
Effects on Human Inference of Robot Navigation Goals
From Moral Will toMoral Skill
Operationalizing Care-CenteredValue Sensitive Design in Robotics
Biomechanics-aware control for robot-assisted physiotherapy
A novel approach to treating shoulder injuries
In this context, the adoption of robotic devices offers opportunities to support manual manipulation of patients and provide sophisticated sensors to monitor them. Yet, despite advances in robot design and control, current systems remain unaware of the patient’s underlying biomechanics, and therefore cannot monitor or prevent harmful loading of healing tissues.
This thesis addresses such critical lack of knowledge by embedding state-of-the-art musculoskeletal models into the control of rehabilitation robots. Through the development of novel algorithms, it enables real-time estimation of deep muscle activity and tendon strain in the shoulder during physical human-robot interaction. By spanning from improved biomechanical simulations to their integration in robotic therapy execution, this work significantly advances the current state of the art to form a cohesive framework for biomechanics-aware robotic physiotherapy. ...
In this context, the adoption of robotic devices offers opportunities to support manual manipulation of patients and provide sophisticated sensors to monitor them. Yet, despite advances in robot design and control, current systems remain unaware of the patient’s underlying biomechanics, and therefore cannot monitor or prevent harmful loading of healing tissues.
This thesis addresses such critical lack of knowledge by embedding state-of-the-art musculoskeletal models into the control of rehabilitation robots. Through the development of novel algorithms, it enables real-time estimation of deep muscle activity and tendon strain in the shoulder during physical human-robot interaction. By spanning from improved biomechanical simulations to their integration in robotic therapy execution, this work significantly advances the current state of the art to form a cohesive framework for biomechanics-aware robotic physiotherapy.
Initiated with an interest in the growth of MR and robotic automation, the research examines how MR might bridge the gap between user satisfaction and the increasing robot implementation in automating tasks execution. While automation streamlines repetitive or labour-intensive tasks, it often removes aspects that contribute to user engagement and job satisfaction. MR, as an immersive and interactive tool, offers a promising solution to enabling users to control and collaborate with robots in a more intuitive and meaningful way.
The project is built around the research question: “How can a unilateral vision-based control system, implemented through a Mixed Reality headset, enable telemanipulation of a robotic arm for crafting purposes, without disrupting the user’s experience and performance?”
The study defines MR’s unique potential for HRI by leveraging vision-based hand-tracking to control a robotic arm, specifically in a crafting task. A prototype was developed to explore the interaction dynamics, using, due to technical constraints, virtual reality to simulate the MR interaction and the robot The primary goal was to design an interaction system that offers full control of spatial navigation and force application, allowing the direct and real-time adjustments essential to a crafting task.
The iterative design process involved testing multiple prototypes with users, identifying challenges and the experience in relation to craftsmanship. Key issues identified included difficulty in manipulating the robot precisely, due to the absence of haptic feedback, limited visual depth awareness, and a reliance on visual-only feedback, which, while informative, have a low robustness and clutters the user interface. These findings lead to the design of three revised prototypes with various constraint levels, one with full user autonomy, one with low constraints and a one highly constrained, to test the impact on user experience and task effectiveness and find the right balance between user control and ease of manipulation.
The results indicate that the low-constraint system provided the most positive user experience, finding the right balance between user control and guidance while maintaining the essential qualities of craftsmanship. Although MR offers a viable framework for enhancing user experience in HRI, the research highlights limitations in replicating the experience of real-world crafting. The study concludes by recommending that future MR applications should leverage MR’s unique qualities, rather than simply mimicking traditional crafting qualities, to foster novel, user-centred interactions.
...
Initiated with an interest in the growth of MR and robotic automation, the research examines how MR might bridge the gap between user satisfaction and the increasing robot implementation in automating tasks execution. While automation streamlines repetitive or labour-intensive tasks, it often removes aspects that contribute to user engagement and job satisfaction. MR, as an immersive and interactive tool, offers a promising solution to enabling users to control and collaborate with robots in a more intuitive and meaningful way.
The project is built around the research question: “How can a unilateral vision-based control system, implemented through a Mixed Reality headset, enable telemanipulation of a robotic arm for crafting purposes, without disrupting the user’s experience and performance?”
The study defines MR’s unique potential for HRI by leveraging vision-based hand-tracking to control a robotic arm, specifically in a crafting task. A prototype was developed to explore the interaction dynamics, using, due to technical constraints, virtual reality to simulate the MR interaction and the robot The primary goal was to design an interaction system that offers full control of spatial navigation and force application, allowing the direct and real-time adjustments essential to a crafting task.
The iterative design process involved testing multiple prototypes with users, identifying challenges and the experience in relation to craftsmanship. Key issues identified included difficulty in manipulating the robot precisely, due to the absence of haptic feedback, limited visual depth awareness, and a reliance on visual-only feedback, which, while informative, have a low robustness and clutters the user interface. These findings lead to the design of three revised prototypes with various constraint levels, one with full user autonomy, one with low constraints and a one highly constrained, to test the impact on user experience and task effectiveness and find the right balance between user control and ease of manipulation.
The results indicate that the low-constraint system provided the most positive user experience, finding the right balance between user control and guidance while maintaining the essential qualities of craftsmanship. Although MR offers a viable framework for enhancing user experience in HRI, the research highlights limitations in replicating the experience of real-world crafting. The study concludes by recommending that future MR applications should leverage MR’s unique qualities, rather than simply mimicking traditional crafting qualities, to foster novel, user-centred interactions.
Human-Machine Co-Learning
Anticipating, Identifying and Sharing Emergent Collaboration Patterns
Communication-Enabled Interactions in Highway Traffic
A joint driver model for merging
Human drivers communicate intent through vehicle kinematics during interactions, making joint decisions and exhibiting joint behaviors. However, current autonomous vehicle (AV) models often lack generalization to real-world behaviors and fail to capture dynamic interactions. AVs typically use models assuming human drivers only respond to AV behavior, leading to over-conservative and sometimes awkward interactions.
To enhance AV-human interactions, the thesis proposes a joint driver model that considers multi-level contributions of drivers. It critiques existing models, highlighting their limitations in capturing dynamic interactions. For instance, many models only consider single drivers and fail to address communication and continuous behavioral adaptation.
The CEI model framework explicitly accounts for driver communication and integrates deterministic future plans with probabilistic beliefs. This framework acknowledges that humans do not continuously optimize behavior but seek satisfactory solutions. The thesis presents a case study where the CEI model accurately describes merging scenarios, generating human-like gap-keeping behavior.
Further, the thesis explores naturalistic driving behaviors using the HighD dataset and develops visualization tools to validate driver models. It extracts and analyzes similar driving scenarios to understand variability in human responses, both operationally and tactically. Controlled experiments in simulators examine driver behaviors during merging conflicts, revealing insights into acceleration control and conflict resolution.
The empirical findings inspire improvements to the CEI model, incorporating intermittent piecewise-constant control observed in human drivers. This updated model accurately reproduces joint driver behaviors and outcomes from experimental scenarios, emphasizing the importance of individual contributions to joint safety margins.
In conclusion, the thesis contributes valuable insights into human lane-changing and merging interactions, proposing a robust model framework for AVs to understand and emulate human driver behaviors. While the study focuses on simplified scenarios, it lays the groundwork for extending the model to more complex real-world situations. The work represents a significant step toward enhancing autonomous vehicles' ability to interact safely and effectively with human drivers on the road. ...
Human drivers communicate intent through vehicle kinematics during interactions, making joint decisions and exhibiting joint behaviors. However, current autonomous vehicle (AV) models often lack generalization to real-world behaviors and fail to capture dynamic interactions. AVs typically use models assuming human drivers only respond to AV behavior, leading to over-conservative and sometimes awkward interactions.
To enhance AV-human interactions, the thesis proposes a joint driver model that considers multi-level contributions of drivers. It critiques existing models, highlighting their limitations in capturing dynamic interactions. For instance, many models only consider single drivers and fail to address communication and continuous behavioral adaptation.
The CEI model framework explicitly accounts for driver communication and integrates deterministic future plans with probabilistic beliefs. This framework acknowledges that humans do not continuously optimize behavior but seek satisfactory solutions. The thesis presents a case study where the CEI model accurately describes merging scenarios, generating human-like gap-keeping behavior.
Further, the thesis explores naturalistic driving behaviors using the HighD dataset and develops visualization tools to validate driver models. It extracts and analyzes similar driving scenarios to understand variability in human responses, both operationally and tactically. Controlled experiments in simulators examine driver behaviors during merging conflicts, revealing insights into acceleration control and conflict resolution.
The empirical findings inspire improvements to the CEI model, incorporating intermittent piecewise-constant control observed in human drivers. This updated model accurately reproduces joint driver behaviors and outcomes from experimental scenarios, emphasizing the importance of individual contributions to joint safety margins.
In conclusion, the thesis contributes valuable insights into human lane-changing and merging interactions, proposing a robust model framework for AVs to understand and emulate human driver behaviors. While the study focuses on simplified scenarios, it lays the groundwork for extending the model to more complex real-world situations. The work represents a significant step toward enhancing autonomous vehicles' ability to interact safely and effectively with human drivers on the road.
A method has been developed that enables a human-robot team to co-learn the handover of an object from the robot to the human. Five design requirements were composed to address the challenges of human-robot co-learning in physically embodied environments. The method is based on a Q-learning algorithm that was adapted and extended to meet these requirements. An experiment was conducted with six participants. For every human-robot team, each design requirement was qualitatively evaluated. Interdependent co-learning was identified in three of the six teams. The limitation of the design, and how this method can be improved further, was discussed. The method, presented in this paper, demonstrates how human-robot co-learning and interdependence can be enabled in physically embodied tasks. ...
A method has been developed that enables a human-robot team to co-learn the handover of an object from the robot to the human. Five design requirements were composed to address the challenges of human-robot co-learning in physically embodied environments. The method is based on a Q-learning algorithm that was adapted and extended to meet these requirements. An experiment was conducted with six participants. For every human-robot team, each design requirement was qualitatively evaluated. Interdependent co-learning was identified in three of the six teams. The limitation of the design, and how this method can be improved further, was discussed. The method, presented in this paper, demonstrates how human-robot co-learning and interdependence can be enabled in physically embodied tasks.
This thesis investigates the possibility and potential benefits of fatigue consideration in the job-shop scheduling problem for a partially automated order picking system. We present a new bi-objective mixed integer nonlinear programming problem formulation to represent system constraints and a predictive fatigue model while considering worker fatigue and productivity during schedule optimisation. To put the results of simulated optimisation in perspective, we experimentally validate the fatigue model predictions and fatigue mitigation capabilities of the scheduling approach using heart rate measurements and qualitative fatigue ratings. These experiments occur with employees in a real-life partially automated order picking system.
Our mathematical model can find solutions that the conventional single-objective optimisation approach cannot, allowing fractional energy expenditure distribution improvements more than 4x larger than the decrease in productivity they require in 53% of the considered virtual cases. This is a promising result for fatigue mitigation in operations only by altering operational decision-making. However, the validation experiments show that our predictive fatigue model has an average RMSE of 2.20 kcal/min in estimating energy expenditure rates compared to heart rate measurements while also showing a low correlation. When assessing 10 minute intervals, a time span that fits a scheduling scope, the estimations improve slightly (avg. deviation of -1.85 kcal/min, avg. correlation of 0.17) but still underestimate the measured values. The experiments also show no significant differences in experienced fatigue between existing schedules and those with fatigue mitigation measures applied.
We conclude that the current scheduling formulation is not yet fit for application with a predictive fatigue model. However, real-life operations can benefit from energy expenditure estimation via heart rate measurements and a different approach for implementation is proposed. Research opportunities lie in further fatigue model development and validation, extension to indirect fatigue effects and other human factors, and further development of the mathematical formulation. ...
This thesis investigates the possibility and potential benefits of fatigue consideration in the job-shop scheduling problem for a partially automated order picking system. We present a new bi-objective mixed integer nonlinear programming problem formulation to represent system constraints and a predictive fatigue model while considering worker fatigue and productivity during schedule optimisation. To put the results of simulated optimisation in perspective, we experimentally validate the fatigue model predictions and fatigue mitigation capabilities of the scheduling approach using heart rate measurements and qualitative fatigue ratings. These experiments occur with employees in a real-life partially automated order picking system.
Our mathematical model can find solutions that the conventional single-objective optimisation approach cannot, allowing fractional energy expenditure distribution improvements more than 4x larger than the decrease in productivity they require in 53% of the considered virtual cases. This is a promising result for fatigue mitigation in operations only by altering operational decision-making. However, the validation experiments show that our predictive fatigue model has an average RMSE of 2.20 kcal/min in estimating energy expenditure rates compared to heart rate measurements while also showing a low correlation. When assessing 10 minute intervals, a time span that fits a scheduling scope, the estimations improve slightly (avg. deviation of -1.85 kcal/min, avg. correlation of 0.17) but still underestimate the measured values. The experiments also show no significant differences in experienced fatigue between existing schedules and those with fatigue mitigation measures applied.
We conclude that the current scheduling formulation is not yet fit for application with a predictive fatigue model. However, real-life operations can benefit from energy expenditure estimation via heart rate measurements and a different approach for implementation is proposed. Research opportunities lie in further fatigue model development and validation, extension to indirect fatigue effects and other human factors, and further development of the mathematical formulation.
Enhancing Musculoskeletal Injury Rehabilitation and Prevention in Rural Areas
Exploring Vibrotactile Feedback Usage in Rural Areas
To evaluate the presumptive advantages, a theoretical analysis into the suggested hybrid actuation solution is conducted. The analysis comprises of numerous simulations using a model of the hybrid drivetrain's working principle. Additionally, the analysis is employed to investigate and spot any early-stage defects or undesirable behavior. Consequently, a 3D model was created in order to 3D print a functioning prototype in order to apply the theory in a physical version. The prototype is initially used to validate the simulation model and enforce the findings of the theoretical analysis. Force data is measured when using the prototype and in turn is fed into the simulation model to assess whether the output behaviour is consistent with the prototype. Both the model's competence and the prototype's predictability are assessed using the outcomes.
Finally, an experiment is conducted to both asses the mechanical performance and predictability as well as the participant's perception; 2 virtual environments and 3 actuation modes (motor, brake, and hybrid) were cross-examined and each repeated 6 times, for a total of 36 trials. This experiment ultimately assesses the prototypes validity and determines whether the theoretically assumed benefits are present in the prototype as well. Finally, the participant completes a questionnaire in the form of a 5-point Likert scale to determine if the experiment data complies with the users experience.
The theoretical analysis revealed both reliable behavior and potential problems, such as oscillations in feedback force due to rapid switching between actuator activity. However, the cause of this behaviour was identified and anticipated to be less pronounced during use of the prototype. The prototype was successfully designed and produced, and a comparison to the model revealed consistent findings and, thus, predictable behavior. However, the model occasionally miscalculated the velocity. The experiment also shows that the hybrid drivetrain approach is promising and outperforms the state of the art both in mechanical prowess as well as physical fidelity when providing haptic feedback; the motor fails to render rigid objects and the brake cannot render spring-like objects, while the hybrid approach can render both successfully.
The thesis demonstrates that additional investigation into the suggested drivetrain is supported, despite some obvious limitations. When compared to equally powerful DC motors, the hybrid technique, as employed in the prototype, is capable of portraying the largest variety of haptics, including hard walls, impacts, and stiffness while potentially conserving weight. ...
To evaluate the presumptive advantages, a theoretical analysis into the suggested hybrid actuation solution is conducted. The analysis comprises of numerous simulations using a model of the hybrid drivetrain's working principle. Additionally, the analysis is employed to investigate and spot any early-stage defects or undesirable behavior. Consequently, a 3D model was created in order to 3D print a functioning prototype in order to apply the theory in a physical version. The prototype is initially used to validate the simulation model and enforce the findings of the theoretical analysis. Force data is measured when using the prototype and in turn is fed into the simulation model to assess whether the output behaviour is consistent with the prototype. Both the model's competence and the prototype's predictability are assessed using the outcomes.
Finally, an experiment is conducted to both asses the mechanical performance and predictability as well as the participant's perception; 2 virtual environments and 3 actuation modes (motor, brake, and hybrid) were cross-examined and each repeated 6 times, for a total of 36 trials. This experiment ultimately assesses the prototypes validity and determines whether the theoretically assumed benefits are present in the prototype as well. Finally, the participant completes a questionnaire in the form of a 5-point Likert scale to determine if the experiment data complies with the users experience.
The theoretical analysis revealed both reliable behavior and potential problems, such as oscillations in feedback force due to rapid switching between actuator activity. However, the cause of this behaviour was identified and anticipated to be less pronounced during use of the prototype. The prototype was successfully designed and produced, and a comparison to the model revealed consistent findings and, thus, predictable behavior. However, the model occasionally miscalculated the velocity. The experiment also shows that the hybrid drivetrain approach is promising and outperforms the state of the art both in mechanical prowess as well as physical fidelity when providing haptic feedback; the motor fails to render rigid objects and the brake cannot render spring-like objects, while the hybrid approach can render both successfully.
The thesis demonstrates that additional investigation into the suggested drivetrain is supported, despite some obvious limitations. When compared to equally powerful DC motors, the hybrid technique, as employed in the prototype, is capable of portraying the largest variety of haptics, including hard walls, impacts, and stiffness while potentially conserving weight.
Human-centered robotisation at KLM Engines
A co-design tool for FRAIM & KLM Engines to explore and evaluate the possibilities & limitations of robotising the workflow
Firstly, through interviewing and stakeholder mapping, the essential stakeholders are identified. These stakeholders have high power or/and high interest in the company and thus contribute to the future of the organisation. Afterwards, the stakeholders are brought together for a co-creation session. In this session, the stakeholders discuss their individual needs and values to identify the contradictions and similarities. Next, a shared vision is created, representing the stakeholders’ view on ‘optimising’ the workflow.
A stakeholder which is often negatively affected by the implementation of robots, is the employee, their needs are often neglected. That’s why this project’s focus is mostly on this stakeholder. During context mapping sessions, the themes of meaningful work are identified. These themes represent the conditions of meaningful work for the employees at KLM Engines.
During the research phase, it became clear that the stakeholders are unaware of the contradictions and similarities of their values. Furthermore, the knowledge and skills needed to understand the possibilities and limitations of implementing robots and their consequences on the stakeholders are lacking.
In the design phase, a co-design is created to tackle this challenge. Co-design is chosen, because it is well-known to be used for bringing together different expertise and for sharing knowledge and skills. Besides, the concept of game design is used. Game design helps to represent a real-life situation in a fun and engaging way and helps the participants to explore different scenarios. Using game design helps to visualise the consequences of robots on the workflow well.
The co-design is tested and evaluated to assess if the co-design meets the design requirements and to discover how the tool can be improved. A final iteration is done to improve the concept.
Lastly, the project is concluded and reflected on. The limitations of the project are discussed and future research and design recommendations are proposed.
...
Firstly, through interviewing and stakeholder mapping, the essential stakeholders are identified. These stakeholders have high power or/and high interest in the company and thus contribute to the future of the organisation. Afterwards, the stakeholders are brought together for a co-creation session. In this session, the stakeholders discuss their individual needs and values to identify the contradictions and similarities. Next, a shared vision is created, representing the stakeholders’ view on ‘optimising’ the workflow.
A stakeholder which is often negatively affected by the implementation of robots, is the employee, their needs are often neglected. That’s why this project’s focus is mostly on this stakeholder. During context mapping sessions, the themes of meaningful work are identified. These themes represent the conditions of meaningful work for the employees at KLM Engines.
During the research phase, it became clear that the stakeholders are unaware of the contradictions and similarities of their values. Furthermore, the knowledge and skills needed to understand the possibilities and limitations of implementing robots and their consequences on the stakeholders are lacking.
In the design phase, a co-design is created to tackle this challenge. Co-design is chosen, because it is well-known to be used for bringing together different expertise and for sharing knowledge and skills. Besides, the concept of game design is used. Game design helps to represent a real-life situation in a fun and engaging way and helps the participants to explore different scenarios. Using game design helps to visualise the consequences of robots on the workflow well.
The co-design is tested and evaluated to assess if the co-design meets the design requirements and to discover how the tool can be improved. A final iteration is done to improve the concept.
Lastly, the project is concluded and reflected on. The limitations of the project are discussed and future research and design recommendations are proposed.
Orbital HMD: An Intuitive Visual Interface for Robot Teleoperation
With Robotic Welding as Use Case
Collision detection, isolation and identification
Implemented on a legged manipulator
I investigated how factors that play a role in gap acceptance decisions during overtaking (namely, size of the gap and the drivers’ velocity) influence the response time. I also studied the change of the drivers' velocity during the decision process. I proposed a novel method to measure the response time in drivers' overtaking decisions, and conducted an experiment to demonstrate the usefulness of the method. 25 participants were presented with multiple overtaking situations in a driving simulator experiment, with varying distance gaps. I analyzed how the probability of gap acceptance varied with the distance gap and participants’ velocity at the start of the overtaking situation using a generalized logistic mixed model. I also analyzed how the response time varied with the distance gap, participants’ velocity and the decision outcome using a linear mixed model. Lastly, I analyzed if the velocity changed between the start and the end of the decision process and whether there was a relation between the decision outcome and the distance gap and the velocity change, using a linear mixed model.
The probability of accepting a gap increased significantly with the distance gap and the velocity of the participant. The response times for rejected gaps were on average 0.7s longer than accepted gaps. The response time increased with the distance gap (42ms per 10m), but decreased with the velocity (-92ms per 1m/s). The velocity changed differently between the decision processes leading to either decisions, with an average difference of 4 m/s.
Using the proposed method, I found that the factors which influence the outcome of the decision, also influence the response time. The dependence of response time on the distance gap and participants’ velocity could be explained by the speed-accuracy tradeoff or the difficulty of the decision. Furthermore, it was shown that the drivers already adapt their velocity during the decision process, instead of after they have made their decision. So, I can conclude that using the proposed method for measuring response time can give insight in the way drivers make gap acceptance decisions during overtaking. My results provide basis for cognitive process models that can help further understand the results and are capable of predicting decision outcomes and response times. ...
I investigated how factors that play a role in gap acceptance decisions during overtaking (namely, size of the gap and the drivers’ velocity) influence the response time. I also studied the change of the drivers' velocity during the decision process. I proposed a novel method to measure the response time in drivers' overtaking decisions, and conducted an experiment to demonstrate the usefulness of the method. 25 participants were presented with multiple overtaking situations in a driving simulator experiment, with varying distance gaps. I analyzed how the probability of gap acceptance varied with the distance gap and participants’ velocity at the start of the overtaking situation using a generalized logistic mixed model. I also analyzed how the response time varied with the distance gap, participants’ velocity and the decision outcome using a linear mixed model. Lastly, I analyzed if the velocity changed between the start and the end of the decision process and whether there was a relation between the decision outcome and the distance gap and the velocity change, using a linear mixed model.
The probability of accepting a gap increased significantly with the distance gap and the velocity of the participant. The response times for rejected gaps were on average 0.7s longer than accepted gaps. The response time increased with the distance gap (42ms per 10m), but decreased with the velocity (-92ms per 1m/s). The velocity changed differently between the decision processes leading to either decisions, with an average difference of 4 m/s.
Using the proposed method, I found that the factors which influence the outcome of the decision, also influence the response time. The dependence of response time on the distance gap and participants’ velocity could be explained by the speed-accuracy tradeoff or the difficulty of the decision. Furthermore, it was shown that the drivers already adapt their velocity during the decision process, instead of after they have made their decision. So, I can conclude that using the proposed method for measuring response time can give insight in the way drivers make gap acceptance decisions during overtaking. My results provide basis for cognitive process models that can help further understand the results and are capable of predicting decision outcomes and response times.
Adaptive lane change assistance
Design and evaluation of a trial-by-trial adaptive lane change assistance system on a motion-based simulator