In clinical practice, objective and quantitative assessment of motor dysfunction is required for monitoring disease progression over time and evaluating response to therapeutic interventions. Thereby, clinicians typically want their patients to make movements to their full physical potential. Augmented reality (AR) games using 3D hand and body tracking that engage patients, could motivate them to perform repetitive tasks to the limit of their physical capabilities in a safe environment. This paper reports on different AR games developed for objective upper extremity motor dysfunction assessment of Parkinson's Disease (PD) patients and stroke patients. Quantitative and qualitative evaluations of various user studies involving 23 PD patients, 22 stroke patients and 39 healthy persons are discussed to make design recommendations for designing engaging AR games for objective assessment of upper extremity motor dysfunction. ... Read more

In clinical practice, upper extremity motor impairments are commonly assessed with disease-specific, subjectively scored and low-resolution rating scales that often do not consider the variations in tasks and environment that are essential aspects of daily life. Augmented reality (AR) systems with contactless tracking of the hand and upper body offer opportunities for objective quantification of motor (dys)function in a challenging, engaging and patient-tailored environment. In this study, we explore the potential of AR for evaluating 1) speed and goal-directedness of movements within the individually determined interaction space, 2) adaptation of hand opening to objects of different sizes, and 3) obstacle avoidance in healthy individuals (N = 10) and two highly prevalent neurological conditions (N = 10 patients with Parkinson’s Disease and N = 10 stroke patients). We successfully implemented three AR games to evaluate these key aspects of motor function. As expected, PD patients moved slower than controls and needed more time for task completion. No differences were observed between stroke patients and controls, perhaps because motor impairments in this patient group were relatively mild. Importantly, usability of our AR system was good and considerably improved compared to our previous study due to more natural and patient-tailored interaction. Although our findings testify to the potential of AR for assessing motor impairments in patients with neurological conditions and provide starting points for further improvement, there are still many steps to be taken towards application in clinical practice. ... Read more

Advances in technology offer new opportunities for a better understanding of how different disorders affect motor function. In this paper, we explore the potential of an augmented reality (AR) game implemented using free hand and body tracking to develop a uniform, cost-effective and objective methods for evaluation of upper extremity motor dysfunction in different patient groups. We conducted a study with 20 patients (10 Parkinson’s Disease patients and 10 stroke patients) who performed hand/arm movement tasks in four different conditions in AR and one condition in real world. Despite usability issues mainly due to non-robust hand tracking, the patients were moderately engaged while playing the AR game. Our findings show that moving virtual objects was less targeted, took more time and was associated with larger trunk displacement and a lower variability of elbow angle and upper arm angle than moving real objects. No significant correlations were observed between characteristics of movements in AR and movements in the real world. Still, our findings suggest that the AR game may be suitable for assessing the hand and arm function of mildly affected patients if usability can be further improved. ... Read more

For a better understanding of how different diseases (e.g. neurovascular diseases, neurodegenerative diseases, and musculoskeletal pain conditions) affect human motor function, a uniform, standardized and objective evaluation is a desirable goal expressed within the clinical community. We explore the capabilities of an augmented reality (AR) game that uses free hand interaction to facilitate an objective assessment of the upper extremity motor dysfunction. First, the design process of the game and the system architecture are described. Second, a study about usability of the AR framework and game engagement is presented based on an experiment we conducted with five patients and ten healthy people. Lastly, a short analysis of the accuracy of the hand data when participants performed “fingers tapping” gesture is done. The results of the study show that even though users experienced the system as physically and mentally demanding, it was engaging enough to make them complete the game. The study also shows that hand data captured is accurate enough to allow a high degree (95%) of pinching gesture recognition. ... Read more

Advances in technology offer new opportunities for a better understanding of how different disorders affect motor function. Our aim is to explore the potential of augmented reality (AR) using free hand and body tracking to develop engaging games for a uniform, cost-effective and objective evaluation of upper extremity motor dysfunction in different patient groups. Based on the insights from a study with 20 patients (10 Parkinson’s Disease patients and 10 stroke patients) who performed hand/arm movement tasks in AR, we created a set of different augmented reality games for upper extremity motor dysfunction assessment. ... Read more

n the clinical community there is a need for assessment tools that allow for objective, quantitative and valid measures of motor dysfunction. In this paper, we report on the design and evaluation of a serious game that engages patients with Parkinson's disease in upper extremity (hand/arm) movements. The game employs augmented reality to show virtual movement targets, i.e. candies falling from a conveyor belt, and a haptic game controller to catch the candies, that is able to acquire quantitative data about the patient's movement. This paper first describes the design process of the game and the system components. Secondly, we present results of our small quantitative evaluation study (N11, age: 26–60, healthy persons) regarding the usability of the system, the task load and user experience of the game. Our findings show that the system has a relatively good usability and the game is engaging, but there is still need for technical improvement with regard to tracking the controller in 3D space. ... Read more

In many fields of activity, working in teams is necessary for completing tasks in a proper manner and often requires visual context-related information to be exchanged between team members. In such a collaborative environment, awareness of other people’s activity is an important feature of shared-workspace collaboration. We have developed an augmented reality framework for virtual colocation that supports visual communication between two people who are in different physical locations. We address these people as the remote user, who uses a laptop and the local user, who wears a head-mounted display with an RGB camera. The remote user can assist the local user in solving a spatial problem, by providing instructions in form of virtual objects in the view of the local user. For annotating the shared workspace, we use the state-of-the-art algorithm for localization and mapping without markers that provides “anchors” in the 3D space for placing virtual content. In this paper, we report on a user study that explores on how automatic audio and visual notifications about the remote user’s activities affect the local user’s workspace awareness. We used an existing game to research virtual colocation, addressing a spatial challenge on increasing levels of task complexity. The results of the user study show that participants clearly preferred visual notifications over audio or no notifications, no matter the level of the difficulty of the task. ... Read more

For a better understanding of how different disorders affect motor function, a uniform, standardized and objective evaluation is a desirable goal for the clinical community. We explore the potential of Augmented Reality (AR) combined with serious gaming and free hand tracking to facilitate objective, cost-effective and patient-friendly methods for evaluation of upper extremity motor dysfunction in different patient groups. In this paper, we describe the design process of the game and the system architecture of the AR framework to meet these requirements. Furthermore, we report our findings from two pilot studies we conducted with healthy people aged over 50. First, we present a usability study (n = 5) on three different modalities of visual feedback for natural hand interaction with AR objects (i. e., no augmented hand, partial augmented hand and a full augmented hand model). The results show that a virtual representation of the fingertips or hand improves the usability of natural hand interaction. Secondly, a study about game engagement is presented. The results of this experiment (n = 8) show that there might be potential for engagement, but usability needs to be improved before it can emerge. ... Read more

For most professional tasks nowadays, it is necessary to work in teams. Such collaboration often requires the exchange of visual context-related information among the team mem- bers. For so-called shared workspace collaboration, awareness of other people's activities is of utmost importance. We have developed an augmented reality (AR) framework in or- der to support visual communication between a team of two people who are virtually co-located. We address these people as the remote user, who uses a laptop and the local user, who wears a head-mounted display (HMD) with an RGB camera. The remote user can support the local user in solving a spatial problem by providing instructions as virtual objects in the view of the local user. For placing virtual objects in the shared workspace, we use a state-of- The- Art algorithm for localization and mapping without markers. In this paper, we report on a user study that explores on how automatic audio and visual notifications about the remote user's activities affect the collaboration. The results show that in our current implementation, visual notifications are preferred over audio or no notifications independent from the level of dificulty of the task. ... Read more

For operational units in the security domain that work together in teams, it is important to quickly and adequately exchange context-related information to ensure well-working collaboration. Currently, most information exchange is based on oral communication. This paper reports on different scenarios from the security domain in which augmented reality (AR) techniques are used to support such information exchange. The scenarios have been designed with a User Centred Design approach, in order to make the scenarios as realistic as possible. To support these scenarios, an AR system has been developed and evaluated in two rounds. In the first round, the usability and feasibility of the AR support has been evaluated with experts from different operational units in the security domain. The second evaluation round then focussed on the effect of AR on collaboration and situational awareness within the expert teams. With regard to the usability and feasibility of AR, the evaluation shows that the scenarios are well defined and the AR system can successfully support information exchange in teams operating in the security domain. The second evaluation round showed that AR can especially improve the situational awareness of remote colleagues not physically present at a scene. ... Read more

NEO ... Read more