Background and aim Regular physical activity preserves functional independence in older adults, yet care-home residents often miss out because personalized supervision is scarce. Autonomous, technology-supported exercise platforms could deliver such guidance without additional staff time—but only if sessions are automatically monitored for safety and quality. We therefore designed a deep learning (DL) system that (a) recognizes individual exercise types and (b) estimates joint angle trajectories from a standard video recording. These outputs are used to compute objective exercise performance metrics (EPMs) such as duration, repetition count, motion variability, and range of motion.

Methods Seven care-home residents (aged between 65–94 years) performed six common rehabilitation exercises in front of a single camera while wearing 17 inertial sensors (Xsens MVN Awinda) that provided ground-truth joint angles. Two-dimensional skeleton poses estimated from the video were fed into a temporal convolutional neural network to recognize the exercises and estimate three-dimensional joint angles. We evaluated exercise segmentation with F1@50 and angle regression with mean per-joint angular error (MPJAE) across nine trunk and lower-limb joints, using leave-one-subject-out cross-validation. Pearson correlations assessed agreement between estimated and ground-truth EPMs.

Results The DL model achieved an F1@50 of 0.92 (± 0.04) for exercise recognition and an MPJAE of 7.7° (± 0.91) for joint angle estimation. The estimated EPMs aligned closely with ground truth, achieving correlation scores of 0.93 (95% CI [0.90, 0.95]) for duration, 0.86 (95% CI [0.80, 0.90]) for repetition count, and between 0.3 and 0.9 for motion variability and range of motion across exercises.

Conclusion The DL algorithm reliably estimates key exercise outcomes from a single video stream. This video-based monitoring pipeline could enable unsupervised, technology-supported exercise assessment in residential care homes while safeguarding session quality and safety. Future work will validate the approach in larger and more diverse cohorts.

Background: Home-based neurophysiological monitoring is improving the assessment and management of neurological conditions such as epilepsy. Technologies such as electroencephalography (EEG), electromyography (EMG), and accelerometry are increasingly integrated into wearable systems for at-home use. Due to an increasing amount of data from long-term monitoring, machine learning algorithms assist in automated data analysis. However, ensuring device accuracy, signal quality, and user compliance remains crucial for clinical useability. Objective: This chapter explores advances and challenges in at-home neurophysiological monitoring, with a primary focus on EEG systems and their applications. Content: The discussion highlights the technological advances and the challenges associated with at-home monitoring. The focus will be on EEG systems, as well as a discussion of EMG in epilepsy. Next, we will provide an overview of the clinical applications for home-based monitoring of epilepsy and sleep disorders. Lastly, we will briefly discuss emerging topics within home-based monitoring in movement disorders and neurodegenerative disorders. Conclusion: Future advancements are expected with new generations of wearable systems capable of providing long-term monitoring with minimal maintenance. Beyond epilepsy and sleep disorders, home-based technologies are also being investigated in other neurological diseases including movement disorders and neurodegenerative diseases showing the expanding scope of home-based technologies in neurology.

Freezing of gait (FOG) is a debilitating symptom of Parkinson's disease (PD), characterized by an absence or reduction in forward movement of the legs despite the intention to walk. Detecting FOG during free-living conditions presents significant challenges, particularly when using only inertial measurement unit (IMU) data, as it must be distinguished from voluntary stopping events that also feature reduced forward movement. Influences from stress and anxiety, measurable through galvanic skin response (GSR) and electrocardiogram (ECG), may assist in distinguishing FOG from normal gait and stopping. However, no study has investigated the fusion of IMU, GSR, and ECG for FOG detection. Therefore, this study introduced two methods: a two-step approach that first identified reduced forward movement segments using a Transformer-based model with IMU data, followed by an XGBoost model classifying these segments as FOG or stopping using IMU, GSR, and ECG features; and an end-to-end approach employing a multi-stage temporal convolutional network to directly classify FOG and stopping segments from IMU, GSR, and ECG data. Results showed that the two-step approach with all data modalities achieved an average F1 score of 0.728 and F1@50 of 0.725, while the end-to-end approach scored 0.771 and 0.759, respectively. However, no significant difference was found compared to using only IMU data in both approaches (p-values: 0.466 to 0.887). In conclusion, adding physiological data did not provide a statistically significant benefit in distinguishing between FOG and stopping. The limitations may be specific to GSR and ECG data, and may not generalize to other physiological modalities.

Video annotation is the gold-standard method to assess Freezing of Gait (FOG) in Parkinsonian disorders, but it is time-consuming. Deep learning (DL)-based assessment of FOG using inertial measurement units ameliorates these problems but poses challenges. Particularly, the large heterogeneity between patients and assessment methods potentially affects detection performance between independent cohorts. To evaluate heterogeneity effects, we developed a DL model on a local cohort (85 participants; 2043 trials) and validated it across six external cohorts (256 participants; 1058 trials). Model-expert agreement on the percentage-of-time-frozen was strong locally (ICC=0.886 [0.79,0.90]) but reduced in external cohorts (ICC=0.562±0.141). Fine-tuning the DL model with just 50 minutes of external cohort data improved the ICC to 0.732±0.138, falling within the borderline of the inter-rater agreement (ICC=0.73-0.99). Therefore, while unified standards are still being developed, we propose an expert-in-the-loop workflow as an effective intermediary and present a proof-of-concept web-based platform for fine-tuning and expert review (aidfog.be).

The ability to identify and temporally segment fine-grained actions in motion capture sequences is crucial for applications in human movement analysis. Motion capture is typically performed with optical or inertial measurement systems, which encode human movement as a time series of human joint locations and orientations or their higher-order representations. State-of-the-art action segmentation approaches use multiple stages of temporal convolutions. The main idea is to generate an initial prediction with several layers of temporal convolutions and refine these predictions over multiple stages, also with temporal convolutions. Although these approaches capture long-term temporal patterns, the initial predictions do not adequately consider the spatial hierarchy among the human joints. To address this limitation, we recently introduced multi-stage spatial-temporal graph convolutional neural networks (MS-GCN). Our framework replaces the initial stage of temporal convolutions with spatial graph convolutions and dilated temporal convolutions, which better exploit the spatial configuration of the joints and their long-term temporal dynamics. Our framework was compared to four strong baselines on five tasks. Experimental results demonstrate that our framework is a strong baseline for skeleton-based action segmentation.

— Freezing of gait (FOG) is an episodic and highly disabling symptom of Parkinson’s disease (PD). Although described as a single phenomenon, FOG is heterogeneous and can express as different manifestations, such as trembling in place or complete akinesia. We aimed to analyze the efficacy of deep learning (DL) trained on inertial measurement unit data to classify FOG into both manifestations. We adapted and compared four state-of-the-art FOG detection algorithms for this task and investigated the advantages of incorporating a refinement model to address oversegmentation errors. We evaluated the model’s performance in distinguishing between trembling and akinesia, as well as other forms of movement cessation (e.g., stopping and sitting), against gold-standard video annotations. Experiments were conducted on a dataset of eighteen PD patients completing a FOG-provoking protocol in a gait laboratory. Results showed our model achieved an F1 score of 0.78 and segment F1@50 of 0.75 in detecting FOG manifestations. Assessment of FOG severity was strong for trembling (ICC=0.86, [0.66,0.95]) and moderately strong for akinesia (ICC=0.78, [0.51,0.91]). Importantly, our model successfully differentiated FOG from other forms of movement cessation during 360-degree turning-in-place tasks. In conclusion, our study demonstrates that DL can accurately assess different types of FOG manifestations, warranting further investigation in larger and more diverse verification cohorts.

Background: Freezing of gait (FOG) is an episodic and highly disabling symptom of Parkinson’s Disease (PD). Traditionally, FOG assessment relies on time-consuming visual inspection of camera footage. Therefore, previous studies have proposed portable and automated solutions to annotate FOG. However, automated FOG assessment is challenging due to gait variability caused by medication effects and varying FOG-provoking tasks. Moreover, whether automated approaches can differentiate FOG from typical everyday movements, such as volitional stops, remains to be determined. To address these questions, we evaluated an automated FOG assessment model with deep learning (DL) based on inertial measurement units (IMUs). We assessed its performance trained on all standardized FOG-provoking tasks and medication states, as well as on specific tasks and medication states. Furthermore, we examined the effect of adding stopping periods on FOG detection performance. Methods: Twelve PD patients with self-reported FOG (mean age 69.33 ± 6.02 years) completed a FOG-provoking protocol, including timed-up-and-go and 360-degree turning-in-place tasks in On/Off dopaminergic medication states with/without volitional stopping. IMUs were attached to the pelvis and both sides of the tibia and talus. A temporal convolutional network (TCN) was used to detect FOG episodes. FOG severity was quantified by the percentage of time frozen (%TF) and the number of freezing episodes (#FOG). The agreement between the model-generated outcomes and the gold standard experts’ video annotation was assessed by the intra-class correlation coefficient (ICC). Results: For FOG assessment in trials without stopping, the agreement of our model was strong (ICC (%TF) = 0.92 [0.68, 0.98]; ICC(#FOG) = 0.95 [0.72, 0.99]). Models trained on a specific FOG-provoking task could not generalize to unseen tasks, while models trained on a specific medication state could generalize to unseen states. For assessment in trials with stopping, the agreement of our model was moderately strong (ICC (%TF) = 0.95 [0.73, 0.99]; ICC (#FOG) = 0.79 [0.46, 0.94]), but only when stopping was included in the training data. Conclusion: A TCN trained on IMU signals allows valid FOG assessment in trials with/without stops containing different medication states and FOG-provoking tasks. These results are encouraging and enable future work investigating automated FOG assessment during everyday life.

Skeleton-based Temporal Action Segmentation involves the dense action classification of variable-length skeleton sequences. Current approaches primarily apply graph-based networks to extract framewise, whole-body-level motion representations, and use one-hot encoded labels for model optimization. However, whole-body motion representations do not capture fine-grained part-level motion representations and the one-hot encoded labels neglect the intrinsic semantic relationships within the language-based action definitions. To address these limitations, we propose a novel method named Language-assisted Human Part Motion Representation Learning (LPL), which contains a Disentangled Part Motion Encoder (DPE) to extract dual-level (i.e., part and whole-body) motion representations and a Language-assisted Distribution Alignment (LDA) strategy for optimizing spatial relations within representations. Specifically, after part-aware skeleton encoding via DPE, LDA generates dual-level action descriptions to construct a textual embedding space with the help of a large-scale language model. Then, LDA motivates the alignment of the embedding space between text descriptions and motions. This alignment allows LDA not only to enhance intra-class compactness but also to transfer the language-encoded semantic correlations among actions to skeleton-based motion learning. Moreover, we propose a simple yet efficient Semantic Offset Adapter to smooth the cross-domain misalignment. Our experiments indicate that LPL achieves state-of-the-art performance across various datasets (e.g., +4.4\% Accuracy, +5.6\% F1 on the PKU-MMD dataset). Moreover, LDA is compatible with existing methods and improves their performance (e.g., +4.8\% Accuracy, +4.3\% F1 on the LARa dataset) without additional inference costs.

To gain insights into the impact of upper limb (UL) dysfunctions after breast cancer treatment, this study aimed to develop a temporal convolutional neural network (TCN) to detect functional daily UL use in breast cancer survivors using data from a wrist-worn accelerometer. A pre-existing dataset of 10 breast cancer survivors was used that contained raw 3-axis acceleration data and simultaneously recorded video data, captured during four daily life activities. The input of our TCN consists of a 3-axis acceleration sequence with a receptive field of 243 samples. The 4 ResNet TCN blocks perform dilated temporal convolutions with a kernel of size 3 and a dilation rate that increases by a factor of 3 after each iteration. Outcomes of interest were functional UL use (minutes) and percentage UL use. We found strong agreement between the video and predicted data for functional UL use (ICC = 0.975) and moderately strong agreement for %UL use (ICC = 0.794). The TCN model overestimated the functional UL use by 0.71 min and 3.06%. Model performance showed good accuracy, f1, and AUPRC scores (0.875, 0.909, 0.954, respectively). In conclusion, using wrist-worn accelerometer data, the TCN model effectively identified functional UL use in daily life among breast cancer survivors.

Purpose Physical activity through exercise helps to delay, prevent, or reverse functional decline in older adults (Bean et al., 2004). Recently, various technologies, including assistive robot coaches, have been developed to engage older adults in physical activities (Avioz-Sarig et al., 2021). Nevertheless, there is a lack of understanding regarding the specific needs, preferences, and potential barriers faced by older adults and their caregivers when interacting with this technology (Fasola and Mataric, 2012). This study is part of the interdisciplinary AI@WZC project, aimed at developing an AI-driven robotic exercise coach for older adults living in a residential care home. We aimed to acquire insights from residents and their caregivers to inform the design of this robotic coach. Method We organized two semi-structured focus groups with respectively five and four care home residents (age: 70–90 years old; gender: session 1 with three females and two males, session 2 with two females and two males; MMSE: 18-30) and two caregivers (one physical therapist, one nurse), adhering to the participatory design principles of the PERCEPT methodology (Bourazeri and Stumpf, 2018). During the first workshop, we gained insights into their technology use, physical challenges, and daily activities. This discussion aided the participants to co-create personas for a care home resident, a physical therapist, and a nurse. In the second workshop, these personas served as a basis to facilitate further discussions. Here, we asked participants to rank their preferred physical exercises related to training specific body parts (e.g., ankle, shoulder, neck), and inquired about their preferences related to functionalities the robot should have, with emphasis on the information that the robot should provide to them through its interface. Results and Discussion Participants have varying physical abilities and preferences regarding physical exercises they engage in. For example, care home residents use different mobility aids (e.g., walker, wheelchair), which impacts which exercises are suitable and safe. Therefore, it is necessary that caregivers can individually assess care home residents’ training progress with the robot, and make adjustments to ensure the exercises fit their individual needs. Flexibility is also important regarding the planning of the robotic coach’ visits. For example, the residents often engage in group or individual exercise sessions with a physical therapist. As such, participants voiced that training with the robot should not be scheduled too close to a session with a physical therapist. Furthermore, participants want to know in advance when an exercise session with the robot is planned, and want to have the freedom to cancel or reschedule such a session when it does not suit them, e.g., in case of unexpected family visitors, appointment with their doctor, or other activities. The design needs to take into account their limited experience with technology, while avoiding burdening caregivers with additional work load to manage the robot’s schedule. Regarding the interface of the robot, participants indicated that feedback primarily needs to be straightforward and functional, e.g., the robot needs to inform them about their progress during the training, letting them know how long it takes before they completed the exercise. Participants also expressed the wish not be interrupted during their training with the robot, meaning that non-functional distractions (e.g., visual embellishments, abundant feedback or encouragement) should be kept to a minimum. Summarizing these results, we identified two key user requirements based on the data collected during the two sessions. (1) The robotic coach needs to serve as an assessment tool, allowing caregivers to monitor functional decline and ensure the safety of exercise regimens. (2) It needs to enable personalization of exercise regimens according to individual preferences and physical limitations of care home residents, therefore allowing them to maintain agency regarding their exercise routines. At the same time, interaction between the robot and residents should be minimal, with a focus on functional feedback during exercises. In conclusion, this approach and the findings from this study can inspire and guide future researchers in developing assistive physical exercise technologies for use in residential care homes.

Freezing of gait (FOG) is a common and severe symptom of Parkinson's disease (PD). Due to the complex underlying pathophysiology, FOG is difficult to assess, hampering further insight into this phenomenon. Inertial measurement units (IMUs) may enable FOG assessment during everyday life, but lack of standardization, e.g., the number and position of the IMUs, complicates an objective comparison of automatic FOG assessment algorithms. We propose a multi-stage temporal dilated convolutional model to automatically assess FOG based on IMU data. We collected simultaneous optical motion capture (MoCap) and IMU data of ten people with PD and FOG. We devised a simulation pipeline, i.e., generating IMU data from MoCap data, to objectively compare our approach to two state-of-The-Art FOG assessment models. The comparison was performed for five simulated IMU configurations, ranging from 1 to 7 IMUs. The results show that our approach outperforms the two state-of-The-Art methods on most of the simulated IMU configurations. The complete lower-body IMU setup of 7 IMUs (pelvis and both sides of the talus, tibia, and femur) enables the best FOG detection performance. Lastly, we show that our model trained by incorporating simulated IMU data enabled significantly improved FOG detection performance than our model trained only with real IMU data. In doing so, we demonstrate that retrospective MoCap datasets can be re-used to train expressive IMU-based FOG assessment models, reducing the required amount of dedicated and labor-intensive IMU data collection experiments.

Freezing of gait (FOG) is a common and severe symptom of Parkinson’s disease (PD). Due to the complex underlying pathophysiology, FOG is difficult to assess, hampering further insight into this phenomenon. Inertial measurement units (IMUs) may enable FOG assessment during everyday life, but lack of standardization, e.g., the number and position of the IMUs, complicates an objective comparison of automatic FOG assessment algorithms. We propose a multi-stage temporal dilated convolutional model to automatically assess FOG based on IMU data. We collected simultaneous optical motion capture (MoCap) and IMU data of ten people with PD and FOG. We devised a simulation pipeline, i.e., generating IMU data from MoCap data, to objectively compare our approach to two state-of-the-art FOG assessment models. The comparison was performed for five simulated IMU configurations, ranging from 1 to 7 IMUs. The results show that our approach outperforms the two state-of-the-art methods on most of the simulated IMU configurations. The complete lower-body IMU setup of 7 IMUs (pelvis and both sides of the talus, tibia, and femur) enables the best FOG detection performance. Lastly, we show that our model trained by incorporating simulated IMU data enabled significantly improved FOG detection performance than our model trained only with real IMU data. In doing so, we demonstrate that retrospective MoCap datasets can be re-used to train expressive IMU-based FOG assessment models, reducing the required amount of dedicated and labor-intensive IMU data collection experiments.

Background: Freezing of gait (FOG) is a common and debilitating gait impairment in Parkinson’s disease. Further insight into this phenomenon is hampered by the difficulty to objectively assess FOG. To meet this clinical need, this paper proposes an automated motion-capture-based FOG assessment method driven by a novel deep neural network. Methods: Automated FOG assessment can be formulated as an action segmentation problem, where temporal models are tasked to recognize and temporally localize the FOG segments in untrimmed motion capture trials. This paper takes a closer look at the performance of state-of-the-art action segmentation models when tasked to automatically assess FOG. Furthermore, a novel deep neural network architecture is proposed that aims to better capture the spatial and temporal dependencies than the state-of-the-art baselines. The proposed network, termed multi-stage spatial-temporal graph convolutional network (MS-GCN), combines the spatial-temporal graph convolutional network (ST-GCN) and the multi-stage temporal convolutional network (MS-TCN). The ST-GCN captures the hierarchical spatial-temporal motion among the joints inherent to motion capture, while the multi-stage component reduces over-segmentation errors by refining the predictions over multiple stages. The proposed model was validated on a dataset of fourteen freezers, fourteen non-freezers, and fourteen healthy control subjects. Results: The experiments indicate that the proposed model outperforms four state-of-the-art baselines. Moreover, FOG outcomes derived from MS-GCN predictions had an excellent (r = 0.93 [0.87, 0.97]) and moderately strong (r = 0.75 [0.55, 0.87]) linear relationship with FOG outcomes derived from manual annotations. Conclusions: The proposed MS-GCN may provide an automated and objective alternative to labor-intensive clinician-based FOG assessment. Future work is now possible that aims to assess the generalization of MS-GCN to a larger and more varied verification cohort.

Rheumatic heart disease (RHD) is one of the most common causes of cardiovascular complications in developing countries. It is a heart valve disease that typically affects children. Impaired heart valves stop functioning properly, resulting in a turbulent blood flow within the heart known as a murmur. This murmur can be detected by cardiac auscultation. However, the specificity and sensitivity of manual auscultation were reported to be low. The other alternative is echocardiography, which is costly and requires a highly qualified physician. Given the disease’s current high prevalence rate (the latest reported rate in the study area (Ethiopia) was 5.65%), there is a pressing need for early detection of the disease through mass screening programs. This paper proposes an automated RHD screening approach using machine learning that can be used by non-medically trained persons outside of a clinical setting. Heart sound data was collected from 124 persons with RHD (PwRHD) and 46 healthy controls (HC) in Ethiopia with an additional 81 HC records from an open-access dataset. Thirty-one distinct features were extracted to correctly represent RHD. A support vector machine (SVM) classifier was evaluated using two nested cross-validation approaches to quantitatively assess the generalization of the system to previously unseen subjects. For regular nested 10-fold cross-validation, an f1-score of 96.0 ± 0.9%, recall 95.8 ± 1.5%, precision 96.2 ± 0.6% and a specificity of 96.0 ± 0.6% were achieved. In the imbalanced nested cross-validation at a prevalence rate of 5%, it achieved an f1-score of 72.2 ± 0.8%, recall 92.3 ± 0.4%, precision 59.2 ± 3.6%, and a specificity of 94.8 ± 0.6%. In screening tasks where the prevalence of the disease is small, recall is more important than precision. The findings are encouraging, and the proposed screening tool can be inexpensive, easy to deploy, and has an excellent detection rate. As a result, it has the potential for mass screening and early detection of RHD in developing countries.

Background

Although deep neural networks (DNNs) are showing state of the art performance in clinical gait analysis, they are considered to be black-box algorithms. In other words, there is a lack of direct understanding of a DNN’s ability to identify relevant features, hindering clinical acceptance. Interpretability methods have been developed to ameliorate this concern by providing a way to explain DNN predictions.
Methods

This paper proposes the use of an interpretability method to explain DNN decisions for classifying the movement that precedes freezing of gait (FOG), one of the most debilitating symptoms of Parkinson’s disease (PD). The proposed two-stage pipeline consists of (1) a convolutional neural network (CNN) to model the reduction of movement present before a FOG episode, and (2) layer-wise relevance propagation (LRP) to visualize the underlying features that the CNN perceives as important to model the pathology. The CNN was trained with the sagittal plane kinematics from a motion capture dataset of fourteen PD patients with FOG. The robustness of the model predictions and learned features was further assessed on fourteen PD patients without FOG and fourteen age-matched healthy controls.
Results

The CNN proved highly accurate in modelling the movement that precedes FOG, with 86.8% of the strides being correctly identified. However, the CNN model was unable to model the movement for one of the seven patients that froze during the protocol. The LRP interpretability case study shows that (1) the kinematic features perceived as most relevant by the CNN are the reduced peak knee flexion and the fixed ankle dorsiflexion during the swing phase, (2) very little relevance for FOG is observed in the PD patients without FOG and the healthy control subjects, and (3) the poor predictive performance of one subject is attributed to the patient’s unique and severely flexed gait signature.
Conclusions

The proposed pipeline can aid clinicians in explaining DNN decisions in clinical gait analysis and aid machine learning practitioners in assessing the generalization of their models by ensuring that the predictions are based on meaningful kinematic features.

Background
Manual annotation of initial contact (IC) and end contact (EC) is a time consuming process. There are currently no robust techniques available to automate this process for Parkinson's disease (PD) patients with freezing of gait (FOG).

Objective
To determine the validity of a data-driven approach for automated gait event detection.

Methods
15 freezers were asked to complete several straight-line and 360 degree turning trials in a 3D gait laboratory during the off-period of their medication cycle. Trials that contained a freezing episode were indicated as freezing trials (FOG) and trials without a freezing episode were termed as functional gait (FG). Furthermore, the highly varied gait data between onset and termination of a FOG episode was excluded. A Temporal Convolutional Neural network (TCN) was trained end-to-end with lower extremity kinematics. A Bland-Altman analysis was performed to evaluate the agreement between the results of the proposed model and the manual annotations.

Results
For FOG-trials, F1 scores of 0.995 and 0.992 were obtained for IC and EC, respectively. For FG-trials, F1 scores of 0.997 and 0.999 were obtained for IC and EC, respectively. The Bland-Altman plots indicated excellent timing agreement, with on average 39% and 47% of the model predictions occurring within 10 ms from the manual annotations for FOG-trials and FG-trials, respectively.

Significance
These results indicate that our data-driven approach for detecting gait events in PD patients with FOG is sufficiently accurate and reliable for clinical applications.