Background: Immersive virtual reality (IVR) is increasingly used for task training. Adjusting sensory and cognitive complexity to individual capacity is important for effective training, yet it remains unclear how varying visual, auditory, and cognitive demands influence behavioral responses, physiological activity, and task performance during goal-directed activities in IVR. Objective: This study aimed to examine how 2 levels (low and high) of visual, auditory, and mental demands affect self-reported mental demand and effort (MDE), task performance, heart rate, gaze behavior, and head movements during a virtual grocery shopping task. Methods: A within-subjects study was conducted with 22 unimpaired university students (11 women, 11 men; aged 23-27 years) recruited via convenience sampling at TU Delft, the Netherlands. Participants completed a grocery shopping task (collecting products from shelves during 122-second trials) under 7 conditions: baseline, visual low/high (background characters), auditory low/high (background sounds), and mental low/high (0-back/2-back tasks), in random order. Outcome measures included MDE (0-100), task performance (products placed in the cart), mean heart rate (from electrocardiography), head-stillness duration, and gaze duration on areas of interest (via integrated eye-tracking). Two-tailed paired-samples t tests with Bonferroni correction (α=.008) were supplemented by linear mixed-effects models. Results: The mental high condition produced the largest effects: MDE increased relative to baseline (mean difference [M_diff] 52.73, 95% CI 44.01-61.44; t₂₁=12.58; P<.001), performance decreased (M_diff –8.00 products, 95% CI –9.57 to –6.43; t₂₁=–10.58; P<.001), head-stillness duration increased (M_diff 13.54 s, 95% CI 9.94-17.14; t₂₁=7.83; P<.001), and gaze on the shopping list increased (M_diff 9.89 s, 95% CI 5.36-14.43; t₂₁=4.54; P<.001). Auditory high demands increased MDE (M_diff 17.05, 95% CI 9.15-24.94; t₂₁=4.49; P<.001) without significantly affecting performance. Visual high demands redistributed gaze away from the main shelves (M_diff –6.11 s, 95% CI –8.70 to –3.52; t₂₁=–4.91; P<.001) without reducing performance. Heart rate increased under mental low demand (M_diff 4.02 bpm, 95% CI 1.69-6.34; t₂₁=3.59; P=.002); the mental high effect did not reach corrected significance in the paired-samples test (P=.02), possibly due to reduced physical activity in this condition. Under high cognitive load, participants spent more time checking the shopping list and showed greater head-stillness, consistent with task-serialization and memory-offloading strategies. Conclusions: Cognitive demands strongly impacted workload, performance, and behavior, while sensory demands affected attention distribution and perceived workload, with no significant performance decrements at the Bonferroni-corrected threshold. By comparing 2 demand levels against a no-demand baseline and incorporating gaze and head-movement measures not previously examined, this study extends prior work in this domain. The findings suggest IVR environments can incorporate sensory richness without large performance reductions in unimpaired young adults, and that behavioral strategies under cognitive load may support real-world workload monitoring and adaptive task adjustment, pending clinical validation.

Advances in haptic technologies enable rich, multi-channel haptic rendering of interactions with virtual objects during virtual reality training. However, it remains an open question whether multi-channel haptic rendering (kinesthetic and tactile) provides superior motor learning and transfer when training dynamic tasks compared to simpler, single-channel sensory information. We investigated how 40 participants learned to invert and balance a virtual pendulum after training under four haptic rendering conditions: congruent kinesthetic and tactile rendering, kinesthetic rendering alone, tactile rendering alone, and no haptic rendering. Kinesthetic information was delivered through a delta robot, and tactile information through a two-dimensional skin-stretch device at the finger pads. Participant performance was measured in catch trials during training, in short- and long-term retention trials, and with a transfer task with a shorter pendulum. Participants from all four training conditions demonstrated the ability to improve and transfer their skills. However, we observed poorer performance during catch trials when training with reduced or absent haptic rendering compared to training with congruent kinesthetic and tactile rendering. The advantage of congruent haptic rendering over conditions lacking kinesthetic rendering was maintained during short-term retention, whereas no significant performance differences were observed between conditions in long-term retention and the transfer task. These results suggest that congruent haptic rendering benefits the task's early learning by supporting the generation of internal models of the task dynamics, with kinesthetic rendering playing a major role. Overall, our findings highlight the potential benefits of multi-channel haptic rendering to accelerate virtual reality training.

Introduction Immersive virtual reality applications are increasingly popular in entertainment, education, and professional training. While many aim for maximal realism, simplifying the virtual environment may offer benefits such as reducing mental workload and improving focus on core tasks. However, the impact of different types of demand on users’ mental workload remains unclear. Objective This study explored the impact of visual, auditory, and cognitive demands on users’ mental workload during a daily living activity in immersive virtual reality. Methods Twenty-four participants used a head-mounted display for a virtual shopping task, i.e., picking ten listed products from a shelf, under different conditions: visual demands (moving characters), auditory demands (background noise), cognitive demands (simultaneous arithmetic task), and a combination of all three. Mental workload measures included heart rate, pupil diameter, and self-reported mental demand & effort. Results The cognitively demanding secondary task induced the largest mental workload, significantly exceeding that of auditory and visual demands. For example, on a scale of 1 (low) to 10 (high), self-reported mental demand & effort was 4.40 for the moving characters, 5.00 for the background noise, 6.67 for the arithmetic task, and 7.17 for the combined condition. Biosignal differences were consistent within participants but were masked by high inter-individual variability. Conclusions In virtual shopping tasks, reducing enforced cognitive demands may be more effective for decreasing mental workload than reducing non-task-relevant visual or auditory demands.

Force perturbations during gait training can increase movement variability, which may support motor exploration and learning. However, when such perturbations are delivered through a robotic exoskeleton, they can also reduce perceived exoskeleton transparency, potentially hindering user acceptance. We tested whether visualizing continuous upper leg-level perturbations in immersive virtual reality (VR) could preserve their variability-enhancing effect while mitigating the cost to perceived transparency in a pelvis-centered walking task. Thirty healthy adults walked on a treadmill while wearing a robotic exoskeleton and performed a ball-in-cup task, requiring continuous mediolateral control of the pelvis. Participants trained under one of three conditions: Control (no perturbations), Perturbation (continuous noise-like multisine forces applied at the thighs), or Perturbation + Visual (same forces with a real-time, body-referenced force-beam visualization). Step-width variability was evaluated during Training. Task performance, intrinsic motivation, and perceived transparency were assessed across Baseline, Training, Retention, and a faster-speed Transfer test (120% of preferred speed). Both perturbation conditions significantly increased step-width variability during Training relative to Control, with no detectable difference between Perturbation and Perturbation + Visual. Task performance improved from Baseline to Retention and Transfer across all conditions, with no significant differences across conditions. Motivation did not differ between conditions either. Critically, perceived transparency decreased only in the non-visualized Perturbation condition and remained stable in both Control and Perturbation + Visual. Our results show that continuous leg-level perturbations reliably enrich lateral gait variability and that simple visual force cues can prevent a perceived transparency cost without compromising the variability manipulation. Future work should test adaptive dosing, multi-session training, and clinical cohorts with impaired lateral stability.

Arm weight support (AWS) from upper-limb exoskeletons aids training by reducing fatigue and increasing the movement workspace, especially in people with acquired brain injury (ABI). Simultaneously, task-specific training could be enabled via haptic rendering, simulating the interaction forces with tangible virtual objects. However, how AWS affects somatosensory perception, such as the perception of the weight of haptically rendered virtual objects, is unclear. We therefore conducted a psychophysics experiment with 40 participants investigating the effect of partial AWS on virtual object weight perception during dynamic lifting. Participants performed the lifting task under two conditions: with and without AWS, where AWS compensated for 50% of their arm weight. An upperlimb exoskeleton equipped with a haptic hand module provided both AWS and haptic rendering of virtual objects. The virtual objects and a representation of the participant's arm and hand were visualized using immersive virtual reality.Weight perception was assessed using just-noticeable differences (JNDs). To evaluate the experimental setup, participants' estimated elbow torques were analyzed. Additionally, subjective measures of task load, motivation, and user experience with the exoskeleton were collected to explore potential secondary factors influencing weight perception. We found that partial AWS effectively reduced elbow load during dynamic lifting compared to no-AWS. The JND was not significantly different between conditions, suggesting that AWS does not hamper weight discrimination. However, participants with a larger self-reported sense of control over their movement and those who reported that the robot followed their movements more closely were associated with better weight perception. Our findings suggest that including haptic rendering into exoskeletons that provide partial AWS could be a viable solution to provide task-specific training.

Neuroscience evidence suggests that personalized, task-specific, high-intensity training is essential for maximizing recovery after acquired brain injury. Robotic devices combined with immersive virtual reality (VR) games, visualized through head-mounted displays (HMDs), can support such intensive training within naturalistic virtual environments with audio-visual stimuli tailored to individual needs. However, the impact of these auditory and visual demands on cognitive load remains an open question. To address this, we conducted an experiment with 22 healthy participants to explore how varying levels of visual, auditory, and cognitive demands affect users’ cognitive load and performance during a shopping task in immersive VR. We found that mental demand had the most significant impact on increasing cognitive load and hampering task performance. Visual demands, although affecting gaze behavior, did not significantly affect cognitive load or performance. Auditory demands showed small effects on cognitive load.

Background: Robotic devices have shown promise in supporting motor (re)learning. However, there is a limited understanding of how personality traits influence the effectiveness of robot-aided training strategies. Methods: We conducted a motor learning experiment with 40 unimpaired participants who trained to control a virtual pendulum using a robotic haptic device. Before the experiment, we assessed personality traits including the perceived control over life events (Locus of Control), the tendency to turn challenges into engaging activities (Transform of Challenge), and other subscales from Autotelic and Hexad gaming style questionnaires. Participants were divided into two groups, one receiving haptic guidance during training and a second one without assistance. Short- and long-term retention was assessed, and relationships between personality traits, performance metrics, and human-robot interaction metrics were analyzed. Results: Participants with high Transform of Challenge or external Locus of Control characteristics who received physical guidance during training reduced the human-robot interaction forces to a lesser extent compared to the ones who did not receive guidance. Additionally, participants with a high Free Spirit gaming style showed greater sensitivity to how their perception of the guidance affected their performance during the retention phases. Conclusion: Our findings suggest that autotelic personality, Locus of Control, and gaming style modulate motor learning outcomes during robotic-assisted training, affecting both performance and human-robot interaction metrics. This highlights the potential of integrating personality-based adaptations in robot-aided rehabilitation protocols to enhance performance and motor (re)learning. Future works should explore the relationship between personality traits and psychological states (e.g., perceived difficulty, attention) across diverse tasks and guidance methods in clinical populations.

BACKGROUND: Head-mounted displays can be used to offer personalized immersive virtual reality (IVR) training for patients who have suffered an Acquired Brain Injury (ABI) by tailoring the complexity of visual and auditory stimuli to the patient's cognitive capabilities. However, it is still an open question how these virtual environments should be designed. METHODS: We used a human-centered design approach to help define the characteristics of suitable virtual training environments for ABI patients. We conducted (i) observations, (ii) interviews with eleven neurorehabilitation experts, and (iii) an online questionnaire with 24 neurorehabilitation experts to examine how therapists modify current training environments to promote patients' recovery in conventional sensorimotor neurorehabilitation settings. Finally, (iv) we involved eight neurorehabilitation experts in a participatory design workshop to co-create examples of IVR training environments. RESULTS: Five phases of the recovery process (Screening, Planning, Training, Reflecting, and Discharging) and six key themes describing the characteristics of suitable (physical) training environments (Specific, Meaningful, Versatile, Educational, Safe, and Supportive) were identified. The experts agreed that modulating the number of elements (e.g., objects, people) or distractions (e.g., background noise) in the physical training environment enables therapists to provide their patients with suitable conditions to execute functional tasks. Additionally, the experts highlighted the importance of developing IVR training environments that are meaningful and realistic. CONCLUSIONS: Through consultations with neurorehabilitation experts, we gained insights into how therapists adjust physical training environments to promote the execution of functional sensorimotor tasks in patients with diverse cognitive capabilities. Their recommendations on how to modulate and make IVR environments meaningful may contribute to increased motivation and skill transfer. Future studies on IVR-based neurorehabilitation should involve patients themselves.

Robotic devices, in combination with virtual reality games, have the potential to increase therapy dosage while enhancing patient’s motivation. Yet, current robotic interventions suffer from poor usability, over-reliance on the availability of trained therapists, and the inability to provide meaningful somatosensory information despite its importance for relearning skillful movements. To address this gap, we co-created two novel haptic rehabilitation robots for in-clinic and in-home rehabilitation capable of high-fidelity haptic rendering during functional reach and grasp training in motivating virtual games together with rehabilitation experts. We evaluated the usability of our solutions with therapists and patients following a mixed-methods approach, gathering quantitative and qualitative data from questionnaires and semi-structured interviews. The results showed good usability and high enjoyment, with the fidelity of virtual object interactions highly praised. Some mechanical design improvements, mainly with regard to comfort, were also identified. Our devices offer naturalistic sensations during training, paving the way for more holistic sensorimotor neurorehabilitation.

Motor augmentation (MA) is an emerging field at the intersection of engineering, robotics, and neuroscience, aiming to enhance human capabilities through the integration of extra limbs. This concept leverages the body’s physiological redundancies, including those within the nervous system. This study examined motor imagery (MI) involving a virtual extra arm, focusing on differentiating its neural patterns from those of biological limbs. Thirty participants performed unimanual reaching MI tasks before (Pre) and after (Post) a conditioning phase in a virtual environment, during which half of the participants received tactile feedback on the movement of the extra arm. Electroencephalographic (EEG) recordings revealed distinct event-related desynchronization (ERD) in α and β rhythms between the extra and biological limbs. Additionally, a Riemannian decoder successfully classified MI for the left, right, and extra virtual arm, providing further evidence of distinct neural patterns. While the conditioning played a role in the ERD’s neural signatures, we did not find the same effects on the decoding. We believe that more complex movements, other sensory encoding modalities, or longer conditioning periods would likely strengthen the connection between tactile feedback and neural control.

Gait variability, the subtle fluctuations in walking patterns, is crucial for adaptation and motor learning. While existing methods to increase gait variability often rely on force-based perturbations, these can reduce motivation. This study explored if a subtle visual feedback distortion (VFD), applied to a first-person avatar's foot position in an immersive virtual reality environment, could increase gait variability without such a drawback. Twenty healthy adults walked on a treadmill wearing a head-mounted display and motion trackers, performing a stepping task under two conditions: with and without VFD. The VFD introduced a continuously changing, noise-like offset to the displayed foot positions, designed to be minimally noticeable. We quantified gait variability through the standard deviation of step width and step length and collected self-report measures on embodiment, motivation, and simulator sickness. We found that VFD significantly increased step width variability by about 15%, indicating enhanced lateral adaptability. In contrast, step length variability remained unchanged. Participants adjusted their foot placement in the opposite direction of the visual distortion, supporting the idea that proprioceptive recalibration underpinned the observed changes. Notably, this increase in variability occurred without any significant effects on embodiment, motivation, or simulator sickness. These findings suggest that subtle VFD can enhance gait variability - potentially facilitating motor learning and adaptability - while preserving user experience and motivation. Future research should determine whether such VFD-based interventions yield lasting functional improvements and investigate their applicability in rehabilitation contexts, potentially offering a noninvasive, user-friendly approach to promoting healthy gait dynamics.

Haptic rendering of weight plays an essential role in naturalistic object interaction in virtual environments. While kinesthetic devices have traditionally been used for this aim by applying forces on the limbs, tactile interfaces acting on the skin have recently offered potential solutions to enhance or substitute kinesthetic ones. Here, we aim to provide an in-depth overview and comparison of existing tactile weight rendering approaches. We categorized these approaches based on their type of stimulation into asymmetric vibration and skin stretch, further divided according to the working mechanism of the devices. Then, we compared these approaches using various criteria, including physical, mechanical, and perceptual characteristics of the reported devices. We found that asymmetric vibration devices have the smallest form factor, while skin stretch devices relying on the motion of flat surfaces, belts, or tactors present numerous mechanical and perceptual advantages for scenarios requiring more accurate weight rendering. Finally, we discussed the selection of the proposed categorization of devices together with the limitations and opportunities for future research. We hope this study guides the development and use of tactile interfaces to achieve a more naturalistic object interaction and manipulation in virtual environments.

Robotic rehabilitation systems may benefit from haptic rendering to provide sensorimotor training to patients with acquired brain injuries. Haptic rendering usually involves modulating stiffness and viscosity to simulate real-world hand-object interactions. Yet, the effect of rendering different viscosities on brain activity remains mainly unexplored. To fill this gap, we ran an experiment with twelve unimpaired participants who were asked to grasp and release virtual liquid dispensers whose stiffness and viscosity were rendered using a haptic hand rehabilitation robot. All liquid dispensers had identical wall stiffness but contained liquids of three different viscosities. We also incorporated control conditions without viscosity and stiffness rendering, involving both passive and active grasping movements. Electroencephalography data were recorded during the experiment. We found stronger ipsilateral somatosensory mu and beta event-related desynchronization during movements with viscosity and stiffness rendering compared to the control conditions, while different viscosity levels did not result in significant variations. Furthermore, no significant electroencephalography activity differences were found between control conditions. These findings indicate that while viscosity and stiffness rendering strengthens brain activity, modulating viscosity levels does not significantly affect this response. This insight may contribute to the design of rehabilitation games by informing the choice of viscosity rendering parameters.

Robotic rehabilitation can deliver high-dose gait therapy and improve motor function after a stroke. However, for many devices, high costs and lengthy setup times limit clinical adoption. Thus, we designed, built, and evaluated the Passive Mechanical Add-on for Treadmill Exercise (P-MATE), a low-cost passive end-effector add-on for treadmills that couples the movement of the paretic and non-paretic legs via a reciprocating system of elastic cables and pulleys. Two human-device mechanical interfaces were designed to attach the elastic cables to the user. The P-MATE and two interface prototypes were tested with a physical therapist and eight unimpaired participants. Biomechanical data, including kinematics and interaction forces, were collected alongside standardized questionnaires to assess usability and user experience. Both interfaces were quick and easy to attach, though user experience differed, highlighting the need for personalization. We also identified areas for future improvement, including pretension adjustments, tendon derailing prevention, and understanding long-term impacts on user gait. Our preliminary findings underline the potential of the P-MATE to provide effective, accessible, and sustainable stroke gait rehabilitation.

A large proportion of stroke survivors suffer from sensory loss, negatively impacting their independence, quality of life, and neurorehabilitation prognosis. Despite the high prevalence of somatosensory impairments, our understanding of somatosensory interventions such as sensory electrical stimulation (SES) in neurorehabilitation is limited. We aimed to study the effectiveness of SES combined with a sensory discrimination task in a well-controlled virtual environment in healthy participants, setting a foundation for its potential application in stroke rehabilitation. We employed electroencephalography (EEG) to gain a better understanding of the underlying neural mechanisms and dynamics associated with sensory training and SES. We conducted a single-session experiment with 26 healthy participants who explored a set of three visually identical virtual textures—haptically rendered by a robotic device and that differed in their spatial period—while physically guided by the robot to identify the odd texture. The experiment consisted of three phases: pre-intervention, intervention, and post-intervention. Half the participants received subthreshold whole-hand SES during the intervention, while the other half received sham stimulation. We evaluated changes in task performance—assessed by the probability of correct responses—before and after intervention and between groups. We also evaluated differences in the exploration behavior, e.g., scanning speed. EEG was employed to examine the effects of the intervention on brain activity, particularly in the alpha frequency band (8–13 Hz) associated with sensory processing. We found that participants in the SES group improved their task performance after intervention and their scanning speed during and after intervention, while the sham group did not improve their task performance. However, the differences in task performance improvements between groups only approached significance. Furthermore, we found that alpha power was sensitive to the effects of SES; participants in the stimulation group exhibited enhanced brain signals associated with improved touch sensitivity likely due to the effects of SES on the central nervous system, while the increase in alpha power for the sham group was less pronounced. Our findings suggest that SES enhances texture discrimination after training and has a positive effect on sensory-related brain areas. Further research involving brain-injured patients is needed to confirm the potential benefit of our solution in neurorehabilitation.

Purpose: Virtual Reality (VR) has proven to be an effective tool for motor (re)learning. Furthermore, with the current commercialization of low-cost head-mounted displays (HMDs), immersive virtual reality (IVR) has become a viable rehabilitation tool. Nonetheless, it is still an open question how immersive virtual environments should be designed to enhance motor learning, especially to support the learning of complex motor tasks. An example of such a complex task is triggering steps while wearing lower-limb exoskeletons as it requires the learning of several sub-tasks, e.g., shifting the weight from one leg to the other, keeping the trunk upright, and initiating steps. This study aims to find the necessary elements in VR to promote motor learning of complex virtual gait tasks. Methods: In this study, we developed an HMD-IVR-based system for training to control wearable lower-limb exoskeletons for people with sensorimotor disorders. The system simulates a virtual walking task of an avatar resembling the sub-tasks needed to trigger steps with an exoskeleton. We ran an experiment with forty healthy participants to investigate the effects of first- (1PP) vs. third-person perspective (3PP) and the provision (or not) of concurrent visual feedback of participants’ movements on the walking performance – namely number of steps, trunk inclination, and stride length –, as well as the effects on embodiment, usability, cybersickness, and perceived workload. Results: We found that all participants learned to execute the virtual walking task. However, no clear interaction of perspective and visual feedback improved the learning of all sub-tasks concurrently. Instead, the key seems to lie in selecting the appropriate perspective and visual feedback for each sub-task. Notably, participants embodied the avatar across all training modalities with low cybersickness levels. Still, participants’ cognitive load remained high, leading to marginally acceptable usability scores. Conclusions: Our findings suggest that to maximize learning, users should train sub-tasks sequentially using the most suitable combination of person’s perspective and visual feedback for each sub-task.

We evaluated the impact of Model Predictive Control (MPC) robotic-assisted versus unassisted training on motor learning of a complex bicycle steering task. Ten participants were divided into two groups, alternating between MPC-assisted and unassisted training to ride a steer-by-wire bicycle on a treadmill to collect virtual stars. At Baseline, Mid-Training, and Post-Training, motor skills were assessed by the average and standard deviation (SD) of distance to stars, while performance was measured by the mean absolute and SD of the steering rate. We found significant improvements in task skill and steering performance, with notable benefits observed in the performance of the group initially trained unassisted. Our findings suggest that starting the training unassisted could stimulate an internal focus (concentrating on one's own body movements) and intrinsic skill perception. This foundation may then form a basis for later integration of MPC assistance to refine further the gained motor skills. Such a sequential training approach may benefit motor skill acquisition of complex dynamics tasks. Further research is necessary to validate and apply these findings to enhance training methods.

Object properties perceived through the tactile sense, such as weight, friction, and slip, greatly influence motor control during manipulation tasks. However, the provision of tactile information during robotic training in neurorehabilitation has not been well explored. Therefore, we designed and evaluated a tactile interface based on a two-degrees-of-freedom moving platform mounted on a hand rehabilitation robot that provides skin stretch at four fingertips, from the index through the little finger. To accurately control the rendered forces, we included a custom magnetic-based force sensor to control the tactile interface in a closed loop. The technical evaluation showed that our custom force sensor achieved measurable shear forces of ± 8 N with accuracies of 95.2-98.4 % influenced by hysteresis, viscoelastic creep, and torsional deformation. The tactile interface accurately rendered forces with a step response steady-state accuracy of 97.5-99.4% and a frequency response in the range of most activities of daily living. Our sensor showed the highest measurement-range-to-size ratio and comparable accuracy to sensors of its kind. These characteristics enabled the closed-loop force control of the tactile interface for precise rendering of multi-finger two-dimensional skin stretch. The proposed system is a first step towards more realistic and rich haptic feedback during robotic sensorimotor rehabilitation, potentially improving therapy outcomes.