The psychological impact that robots have on workers in a physical human-robot collaboration is not well researched. This creates a risk of creating monotonous jobs for workers as more and more robots enter the workforce. To mitigate this risk, physical human-robot collaboration should be designed to have a positive psychological impact by facilitating a flow state. Flow is the experience of complete absorption into the current moment. Based on studies in other fields, a new method for designing a physical human-robot collaboration for flow has been developed. This new method was applied to an abstract blending task in which the robot assisted the human in blending. A controlled experiment was conducted to test the psychological impact that this design method had on participants. The findings provide evidence that designing for flow made the task more satisfying, as such indications were found that suggest participants were more motivated in their task, finding it more rewarding, causing them to do more of the task than required. However, no significant impact on flow was found because participants in the control group were equally focused as participants who had the task designed for flow. Nonetheless, the positive impact that designing the physical-human robot collaboration had on the motivation and satisfaction of the humans still created a positive psychological impact. This makes designing for flow a promising method for building towards a human-centered future of work. ... Read more

In dangerous environments, teleoperation is needed to enable humans to execute tasks remotely. To assist in these tasks, haptic teleoperation systems provide the human operator with the sense of touch of the telerobot. One way to provide this sense of touch is through high-frequency vibration feedback. State-of-the-art solutions generally rely on integrated hardware, which limits their application to specific telerobots and master devices.
The aim of this study is to develop a deployable high-frequency vibration feedback method through an add-on setup. In the presented system, both the vibration recording device and vibration feedback display run on a single microcontroller. Furthermore, all components are small in size and portable by the robotic or human hand.
Spectral analysis of the replicated vibrations shows that the presented system is capable of mimicking textures. To evaluate the effectiveness of the texture imitations, a human factors experiment is conducted. Twenty participants executed a texture identification task for two conditions: a manual condition with direct tactile feedback and a teleoperated condition with tactile feedback displayed by the presented system.
Results show that 75-85% of the textures were correctly identified in the teleoperated condition. These correctness rates are close to the results of the manual condition (96% correct) and outperform the chances of random guessing by a factor three. In the teleoperated condition, participants took on average 67% longer than in the manual feedback condition.
Based on these results, it is concluded that the presented add-on system enables humans to accurately feel high-frequency vibrations in teleoperation. ... Read more

Understanding human behavior in overtaking scenarios is crucial for enhancing road safety in mixed traffic with automated vehicles (AVs). Modeling plays a pivotal role in advancing our comprehension of human overtaking behavior in dynamically evolving scenarios. Currently, our understanding of overtaking behavior primarily revolves around straightforward interactions with human-driven vehicles (HDVs). To address this gap, we conducted a ``reverse" Wizard-of-Oz driving simulator experiment with 30 participants interacting with both oncoming AVs and HDVs, featuring time-varying dynamics. We hypothesized that the type of oncoming vehicle (AV or HDV) does not significantly influence gap acceptance during overtaking, while we anticipated an increase in gap acceptance when the oncoming vehicle briefly decelerates during interactions with the human ego-vehicle driver. Our findings reveal that participants did not significantly alter their overtaking behavior when interacting with oncoming AVs compared to HDVs. Surprisingly, brief decelerations in the oncoming vehicle's velocity did not significantly affect the decision-making processes of overtaking. Moreover, our results reinforced previous insights into the significance of the initial distance and time-to-arrival to the oncoming vehicle, and the ego-vehicle velocity on participants' overtaking behavior. We highlight the potential of simple drift-diffusion models (DDMs), a subset of cognitive models, in understanding human overtaking behavior in dynamically evolving scenarios involving oncoming AVs. Our proposed model accurately captures qualitative patterns in gap acceptance during these intricate overtaking scenarios, further advancing the ongoing development of safer interactions between human drivers and AVs during overtaking maneuvers. ... Read more

We propose a novel shared control interface that enables teleoperated teaching of both high-level decision-making skills and low-level impedance modulation skills using a single haptic device. In the proposed method, high-level teaching is achieved by repurposing the haptic device to remotely modify Behaviour Trees (BTs), allowing human operators to guide decision-making. Repurposing of the haptic device is achieved by exploiting its degrees of freedom for different functionalities. Low-level skill teaching involves an impedance command interface, that is used to command endpoint stiffness by manipulating a 3D virtual stiffness ellipsoid with the haptic device. Both teaching modes are connected: a newly demonstrated low-level skill appears in the BT at a user-specified index. Control is shared between the human and the autonomous system on a high- and low-level. At the higher level, the human can change the BT online, while ongoing execution of the low-level actions within behavior tree remains uninterrupted. During low-level teaching, shared control is implemented between the robotic motion skill and human-demonstrated stiffness. To provide a proof-of-concept and demonstrate the main features of the proposed interface, we performed several experiments in a teleoperation setup operating a remote shelf-stocker robot in a supermarket environment. A predefined BT encodes high-level decisions for a pick-and-place task. The impedance command interface is evaluated in a “peg-in-hole”-like task of placing a product on a cluttered shelf. Ultimately, the proposed interface can facilitate teleoperation-based Learning from Demonstration for the transfer of both high- and low-level skills in an integrated manner. ... Read more

It is impossible to imagine modern day interaction with technology without the use of touchscreens. It is a go-to interface to use for many applications, because of the high stimuli-response compatibility and adaptability of the graphical user interface. But the haptic feedback one would have with physical buttons and dials, is lost with the use of touchscreens. High potential to improve the interaction with high resolution haptic feedback is often ignored. In this paper, the use of a haptic pseudo-potential field rendering method on a friction modulated touchscreen is proposed. With this method, the user is assisted in moving towards a target by lowering the friction and impeded in moving away by increasing the friction coefficient. In a human factors experiment, this rendering method is compared to a position-based friction modulation method. Subjects are instructed to find a target path, based on the haptic feedback. The results show that the position-based rendering method has a higher hit-rate and lower movement times. This demonstrates that the pseudo-potential field method is difficult to perceive, however it is expected that advancements in rendering larger friction coefficient ranges or even active lateral force feedback will improve this rendering method. ... Read more

Our remarkable sense of touch provides us the feedback that is crucial for successfully manipulating a wide range of objects.
The unconscious synergy between touch and the precision grip is particularly astonishing.
During precision manipulation, humans constantly control their grip force to maintain a safety margin of approximately 25 percent above the minimum force required to prevent held objects from slipping.
The ability to accurately control this safety margin heavily relies on tactile feedback founded on sensed deformations of our fingertips.
Previous studies have demonstrated that, by using this feedback, humans even manage to maintain this safety margin independently of the weight or friction of a lifted object, and when the weight of a held object is perturbed.
However, it is still unknown whether the sense of touch can help us to maintain this safety margin when the friction of a statically held object is perturbed.
As previous methods could not deliver these friction perturbations, we demonstrated the viability of a new friction perturbation method that we employed to fill this knowledge gap.
Here we show that humans in fact do not adapt their grip force in response to an abrupt increase of friction, but do increase their grip force in response to an abrupt decrease of friction.
The asymmetry of these grip adaptations is consistent with current hypotheses on the limitations of our sense of friction.
Our results support the existence of the hypothesized inability of our sense of touch to directly sense an increase of friction.
These findings can help to enhance the haptic interaction between humans and machines, and may inspire the design of an artificial sense of touch that can greatly improve the manipulation dexterity of robotic grippers. ... Read more

Thermal feedback has been proven to enhance the user experience in human-machine interaction. However, state-of-the-art technology mainly focuses on static contact using either palm or fingertip, overlooking dynamic and multi-finger interactions. Underlying challenges include incompatible designs of the conventional interfaces for providing controllable salient thermal stimuli for such interactions and, thereby, lack of knowledge on human thermal perception for relevant conditions. Here we designed a new thermal display that can deliver distributed spatio temporal thermal patterns and investigated the influence of user exploration on the perception of these patterns. Twenty-three human participants interacted with the device using three exploration conditions (static-single finger, dynamic-single finger, and static-multi finger) and evaluated 15 temperature differences ranging from +1.5◦C to -7.5◦C. Our results showed that humans are significantly more sensitive to thermal stimuli when exploring via static single-finger contact than other tested conditions. Moreover, in the case of static-single finger interaction, we found larger thermal discrimination thresholds compared to the literature. Our findings offer new perspectives on providing salient and consistent thermal feedback for future tactile interfaces. ... Read more

The ever-emerging mobile market induced a blooming interest in stylus-based interactions. However, most state-of-the-art styli are passive or display only unimodal tactile feedback. Multimodal haptic devices that simultaneously stimulate our cutaneous and kinesthetic receptors to provide immersive and realistic sensations in a virtual environment during touchscreen interactions are highly desired. To this end, we developed FeelPen, a novel handheld multimodal haptic interface for touchscreens, incorporating various actuators in a smartly designed way. A voice-coil actuator, placed along the stylus tip, simulates object compliance by modifying its stroke force. Electrovibration, generated between the stylus tip and a capacitive screen, delivers roughness and stickiness cues. In addition, temperature feedback on the fingertip is provided by a miniature thermal module. We conducted characterization experiments to determine the physical characteristics and limitations of the device, followed by a psychophysical experiment, where the perceptual dimensions of the device were extracted using the semantic differential method on a set of artificial textures. Our results revealed four tactile dimensions, with the first two related to texture surface properties, and the third and fourth dimensions linked to material softness and coldness, respectively. FeelPen opens up new dimensions for future realistic texture rendering on touchscreens. ... Read more

Immersive virtual reality (IVR) is an artificially designed environment that can be used to produce realistic and engaging environments which are being used actively in the field of healthcare through training and rehabilitation. The use of IVR nowadays ranges from training surgical operations in a safe environment to neurorehabilitation. IVR implementation for rehabilitation is more task-specific, enhances patients' attention during training, and provides visual feedback. Avatar, a virtual extension of the user, can be used to interact with the virtual environment and aid in postural adjustments during rehabilitation. Although IVR training of upper limbs is often seen, the research is ongoing for lower limbs. Walking activities in the real-world post-stroke are essential for active participation in the community and to reduce potential mental illness. There is ongoing research on how to implement walking activities in VR.
This study aimed to explore the effect of visualizing different ground surfaces on gait patterns.
Twelve healthy young participants were recruited for the experiment. Two scenes with a different ground surfaces -- ice and concrete -- were designed. A male and a female avatar were animated and implemented in the scene. The participants were asked to walk eight times in both. Trackers located at the left and right foot and pelvis were used to obtain kinematic parameters such as stride and step length and gait speed. The participants were asked to answer an embodiment questionnaire, which consisted of questions about body ownership, sense of agency, and location, after each scene.
We found that the first kinematic values of stride and step lengths and gait speed were lower while walking over the virtual ice scene compared to concrete. Overall, the values of body ownership, and sense of agency were higher when compared to the control questions of body ownership and sense of agency, after each scene. The value of the sense of location after each scene was also higher. The present findings show that the participants embodied in both the scenes and the ground surface had a significant influence on their gait modification. Thus, implementing ground surfaces along with IVR in rehabilitation can benefit patients by helping them adapt their gait to the ground surface. ... Read more

More and more vehicles have multiple advanced driver-assistance systems (ADAS), that take over tasks from the human driver, thereby taking the driver out of the loop of control. This might create a discrepancy between the responsibility that the human driver feels and the responsibility that is attributed to them when something goes wrong. Previous studies into perceived responsibility were mostly conducted in traded control systems, in which either the vehicle or the driver was performing the task, and tasks were shifted between them. In haptic shared control systems the automation and human driver cooperate continuously. The Level of Haptic Authority (LoHA) determines how strong the controller enforces its guidance. We examine how this LoHA impacts the driver's own perceived outcome responsibility, as well as that attributed to the automation when the automation makes a mistake. We found that when authority is shifted towards the car, the human driver feels less responsible, and attributes more responsibility to the automation, but only to a certain degree. Our findings correspond with previous research and with our own hypothesis. They add a new perspective to the current literature, as this is the first research-paper to examine responsibility perception in haptic shared driving from the drivers perspective. More research in the human driver's experience is needed to better understand human behaviour whilst driving with driving automation systems. ... Read more

Human-robot interaction is a growing field that aims to research and develop communication channels between humans and robots to enhance comfort, safety, and productivity in healthcare, the household, and the industry. Researchers have considered ergonomy-related metrics to compose these channels for physical human-robot collaborative scenarios. We refer to these communication channels as arbitration methods. Several of these metrics, such as human arm manipulability and muscle fatigue, have taken their turns in the literature to set the base for arbitration methods reaching promising results. Human arm force manipulability represents the transmission between joint torques in the joint space and end-point force in the task space depending on the configuration of the joint angles. Muscle fatigue keeps track of the muscle activation and builds up depending on the muscle activation level and previous fatigue value. The first one has predictive value. The other has a reactive value.\par
Nevertheless, no work in the literature explores the power of combining both metrics into an arbitration method. Here we develop a multi-metric arbitration method that combines human arm force manipulability and muscle fatigue as input for a finite state machine (FSM) that translates the human multi-metric state to robot control level over a collaborative task. Although some modifications may be worth trying and evaluating to reach generalizability in physical human-robot collaborative tasks, the system reached satisfactory results. Moreover, as future steps, we should conduct human-factors research to compare the effect of the system on task performance. ... Read more

The olivocerebellar system plays a crucial role in control of movements of the human body in terms of coordination, precision and timing. Long-term plasticity is directly linked to motor learning and control. In this research, we developed a phenomenological model of the olivocerebellar system with balancing of long-term potentiation (LTP) and long-term depression (LTD) at the parallel fiber-Purkinje cell (PF-PC) synapse. By ranging the PF input over frequencies, we found that PCs can select frequencies in a highly non-linear manner. There is a sharp contrast in synaptic weight change between neighbouring frequencies, which is caused by the temporal spiking property of the inferior olive (IO) cell. This research found a novel signal processing capability of the PC. ... Read more

Variable steering systems have the ability to change the ratio between the steering wheel and the front wheels while driving. These adjustable steering systems have led to an improvement in traffic and road safety and decrease in driver’s workload. A previous study concludes that driver steering behaviour is significantly dependent on vehicle speed and road curvature (number and sharpness of bends). Interestingly, variable steering ratio systems often depend on speed but not on road curvature. Variable steering ratio dependent on road curvature possibly influences driving behaviour and might be desirable for safety and driver acceptance. The goal of this research is to investigate driver acceptance and driving behaviour for two separate steering ratios (1:12 and 1:40) and two different road profiles (i.e. specific curvatures straight highway and curvy country road) at a constant speed. We hypothesize that on a curvy country road low steering ratio (1:12) leads to higher safety margins and subjective ratings, whereas on a straight highway a high steering ratio (1:40) leads to higher safety margins and subjective ratings. Therefore we conducted a within-subject driving experiment (N=24) in a fixed-based passenger vehicle simulator at constant speed. The results show that on a country road a vehicle with a low steering ratio increases time-to-line- crossing (TLC) safety margins and increases self-reported subjective ratings compared to the high steering ratio setting. Likewise, on a highway, a vehicle with a high steering ratio leads to higher safety margins and comfort rating compared to a low steering ratio. Thereby it can be concluded that steering ratio variable to the road profile improves safety and acceptance. These results provide promising evidence to make steering systems adaptable to road profile (e.g. steer-by-wire and active rear wheel steering). ... Read more

Many users of body-powered upper extremity prostheses experience difficulties using their device and a large group abandons usage altogether. Shoulder control via Bowden cable is widely used because of the intuitive use and low cost, but requires large shoulder movements and high operating forces that, according to literature, often exceed the upper limit of around 20N that would allow for fatigue-free prolonged use. Implementing by-wire control reduces friction forces due to shorter cables, but it also allows for prosthesis control to be treated as a telemanipulation problem: workspace extension methods could prove effective in further reducing these movements and forces. To test if these methods are indeed applicable, three different modes of control - proportional gain control, non-linear variable gain control, and velocity control - were implemented in an ideal virtual environment. Performance was measured at both ends of the range of motion using an experiment based on Fitts' translational tapping task. It was hypothesised that variable gain control would improve speed during gross positioning and accuracy during fine positioning, improving overall performance, while velocity control would perform worse. The results show that the hypothesis holds, as well as improved controller performance when using a variable gain. It can be concluded that variable gain control in shoulder actuated prostheses can be beneficial and it would be worth exploring in real-life applications. ... Read more

The use of social robots increased in the past few years. Current technology, however, lacks in deploying a single robot for different applications without the help of a human being. Current solutions are time-consuming, labour intensive and hard to generalize. Being aware of its surroundings, in terms of environment and context, the robot can select the appropriate application that the situation needs. We propose a multi-modal, knowledge-based hybrid scene classification method for applying awareness to the robot. As scene we refer to the combination of the environment and the context of the surroundings; a study on how to describe a scene has been done through knowledge-engineering methods that comprehend an anonymous online questionnaire and observations. The method inputs features of the type of objects, audio, and human detection and understanding; and outputs the probabilities of the possible social roles for the robot (Receptionist, Tutor and Waiter). The classification is based on a hybrid approach and trained and validated on a real-time multi-modal data-set collected by a mobile robot. The training experiment aimed to collect the data-set, to select the features that describe different roles and to calculate their weights. The validation experiments aimed to measure the performance and the generalization of the method. Results show that the robot was able to successfully classify the Receptionist role with an accuracy of 83.4%; the Tutor role with 82.7%; and finally, the Waiter role with 55.9%. On average, the method generalizes for 74% of unseen data. ... Read more

Tele-impedance augments classical teleoperation by enabling the human operator to actively command remote robot stiffness. Hereby, an essential strategy used by humans to successfully interact with the unstructured environment complements remote robot-environment interaction. However, literature lacks awareness of benefits and disbenefits of currently used stiffness command interfaces (SCIs) in bilateral tele-impedance. In this paper, we introduce a term called the coupling effect. The coupling effect pertains to the coupling between human initiated commanded stiffness and force-feedback from the master robot. It is hypothesized that, whenever the operator’s commanded stiffness and the master device are coupled, like in muscle activity based SCIs, force-feedback can invoke changes in the commanded stiffness due to human reflexes. Although the coupling effect takes away some degree of the operator’s control over the commanded stiffness, these involuntary changes can be either beneficial (e.g. during position tracking) or detrimental (e.g. during force tracking) to the task performance on the remote robot side. In an experimental study 16 participants perform position and force tracking tasks by using both an EMG based coupled type and an external device based decoupled type of SCI. Our results demonstrate a benefit of the coupling effect by showing lower absolute position error during the unexpected force perturbation when a coupled SCI was is in the position task. The coupling effect does not affect reference stiffness tracking in the force task during the process of establishing contact compared to maintaining reference force when a coupled SCI is used. The decoupled SCI is beneficial for tracking reference force in the force task and tracking reference stiffness for both tasks. We conclude that—when using bilateral tele-impedance—one should be aware of the coupling effect, which is beneficial for rejecting a disturbance in a position tracking task but detrimental for establishing contact in a force tracking task. ... Read more

Objective. Dexterous control of myoelectric upper limb prosthesis is still limited by the capabilities of the modern human-machine interfaces. The first goal of the current work was to develop a system that supplements the academic myoelectric state-of-the-art interface during the interaction with objects (e.g., grasping, manipulation) with the goal of increasing the overall performances and robustness of the prosthetic device. Additionally, the current study aims to define guidelines for a larger-scale experiment to be performed in the immediate future. Approach. I developed algorithms, which provide context- and user-awareness to the system by fusing multimodal sensory input data, and a control scheme that employs such context-awareness to estimate the user’s grasp intentions to automatically preshape the prosthesis for grasping in real time. The control scheme was compared against the major academic state-of-the-art myoelectric control scheme (i.e., pattern recognition) in two able-bodied subjects. The experimental tests consisted of grasping, reorienting, and relocating sets of common objects using a multi-degree-of-freedom prosthesis with two grip types and two degrees-of-freedom actuated wrist. Main Results. The proposed semi-autonomous system was able to function in realistic and time-varying cluttered environments. The obtained results illustrate better and more consistent performances (i.e., lower task completion time and standard deviation) of the developed control scheme with respect to the state-of-the-art counterpart. Improvements in control robustness during object manipulation (i.e., lower number of object drops) have also been obtained. The current study helped in defining guidelines for the future larger-scale experiment: more than one experimental session, data logging and subjective measurements recording. Significance. The proposed system improves multiple aspects involved in the control of myoelectric multi-degree-of-freedom upper limb prostheses. The guidelines defined in this work, are essential for evaluating, during the future larger-scale study, the impact of the proposed system on users’ experience (e.g., workload and ease of use).
... Read more

By monitoring head movement and orientation in space, the vestibular system can evoke appropriate muscle responses in order to maintain standing balance. The present study investigates whether vestibular-evoked muscle responses are dependent on sensory cues of gravity by examining these responses across varying load and gravity conditions. Standing subjects were exposed to a stochastic electrical vestibular stimulus (EVS, ±5 mA, 0-25 Hz) that induced a vestibular error signal, while vertical loading forces or vestibular signals of gravity were independently modified. A backboard structure limited subjects’ whole-body rotation to the sagittal plane which corresponded with the EVS-evoked sway responses in anteroposterior direction, as the subject’s head was rotated in yaw. Vestibular-evoked muscle responses were greatest when sensory cues of gravity matched the expected terrestrial force of gravity, and decreased when these cues were modified. The reduction was largest when both load- and vestibular-related cues of gravity were different from normal. Our results indicate that the vestibular drive for standing balance control is attenuated when sensory cues of gravity are not congruent to normal (i.e. terrestrial) expectations of standing balance and that the degree of attenuation is dependent upon the cumulative incongruency that arises from multiple sensory cues. ... Read more

The elicitation of user engagement is one of the current challenges of human-robot interaction, alongside with the identification of appropriate metrics to evaluate users' experience. Several studies focused on strategies aimed at maintaining engagement throughout an interaction, but limited research has been done on how to initiate it by drawing users' attention. The use of social cues in nonverbal human-human communication has been identified as a reliable source of information to determine if a person is engaged. These social cues can be used not only to understand more about human behavior, but also to design robot behaviors that can successfully draw the attention of humans. In this project we look to investigate what are effective techniques to draw attention and elicit initial engagement with a social robot at the entrance of a building. The robot proposed to display these behaviors is the humanoid robot Pepper, from SoftBank Robotics, as it has been specifically designed for human-robot interaction. Initially, the on-board functionalities of the robot are going to be tested. Secondarily, state-of-the-art techniques are going to extend those functionalities to improve Pepper's interactive skills. Eventually, robot behaviors are going to be designed and displayed to participants during an experiment. We aim at understanding the reactions of people to identify the most effective drawing attention behaviors and examine if the encounter with the robot is affected by a novelty effect. Different metrics are proposed to measure both these phenomena in our results. Our system is developed in Python with features extracted from Pepper's Naoqi framework. ... Read more

Identifying the time-varying, adapting human operator online could enable adaptive human-machine support systems and attention monitoring for human-in-the-loop vehicle control tasks. A validated, online identification method, including adaptation detection is missing, however.

In this MSc thesis project, online human operator identification using low-order ARX models with recursive least squares estimation was implemented. Operator delay estimation was attempted in simulations using the parallel recursive ARX method. Two online adaptation detection algorithms were evaluated on experimental data from eight subjects who performed a compensatory tracking task with time-varying controlled element. Participants were instructed to explicitly indicate when they detected this change with a button push, as comparison with the performance of the detection methods.

The experiment validated the online identification method, but time-varying delay estimation was not sufficiently accurate. The best detection method was based on recognizing the deviation of the human error rate response gain from a priori measured non-adapted behavior, and had a detection accuracy of 57%. A more practical method based on the parameter moving average was also developed and tested, but had a lower accuracy of 43%. For both methods, the lag in detection was found to be equivalent to the human operator detection lags measured in the experiment. Thus, the developed methods open the door for further development of identification-driven adaptive operator support systems in manual vehicle control tasks.
... Read more