Towards Data-Driven Precision Neurorehabilitation
Validation and Implementation of a Multivariable Prediction Model for Prognosis of Functional Independence in Young Adults with Acquired Brain Injury
E.H.M. Lie (TU Delft - Mechanical Engineering)
A.H.A. Stienen – Graduation committee member (TU Delft - Biomechatronics & Human-Machine Control)
Dr. M. Königs – Mentor (Amsterdam UMC)
Prof.dr. W.C. Peul – Graduation committee member (Leiden University Medical Center)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Introduction
Acquired Brain Injury (ABI) presents a significant public health challenge due to the diverse recovery trajectories resulting from its heterogeneous nature. Prediction models, derived from structured data collection, offer a more personalised approach to neurorehabilitation. However, a substantial gap remains between the development of these models and their successful implementation in clinical practice. This study addresses this gap by focusing on two key components: externally validating prediction models for functional independence in young adults with ABI, and creating a user-friendly interface to support their application in clinical settings. These efforts represent a crucial step toward advancing precision neurorehabilitation, enabling data-driven, individualised care tailored to the unique needs of ABI patients.
Design and Methods
Previously, three multivariable prediction models were developed to predict the prognosis of severe ABI in young adults aged 16 to 35 admitted to the Daan Theeuwes Centre, demonstrating promising performance. These models focused on functional independence, measured by the Barthel Index (BI) at admission, three months later, and the change in independence during this period. This study focused on the implementation of these prediction models by external validation of these models, and the development of a web-based tool to facilitate their implementation in clinical practice. Data for the external validation cohort were sourced from the Measurement Feedback System (MFS). Highly incomplete variables were excluded, and missing data were handled using Predictive Mean Matching (PMM). Model performance was assessed using Coefficient of Determination (R2), Root Mean Square (RMSE), and Mean Absolute Error (MAE), alongside calibration and correlation analyses. Additionally, results were assessed against the 95% Prediction Interval (PI) of the development cohort. A web-based tool was developed simultaneously to facilitate the practical application of these models in clinical practice, informed by clinician feedback and literature insights.
Results
The validation cohort (n = 21) showed minimal discrepancies compared to the development cohort (n = 100), but external valida- tion revealed reduced predictive accuracy. The ”Level of Independence at Admission,” ”Level of Independence at Three Months Post-Admission” and ”Change in Independence over Three Months” models had notable drops in R2 from 65.7% to 42.8%, 59.3% to 29.7%, and 76.3% to 35.9%, respectively. All models fell outside the 95% Confidence Interval (CI) of R2 for the development cohort and showed increased RMSE and MAE values. Calibration showed overestimation of lower BI scores and underestimation of higher scores, with a substantial proportion of predictions falling outside the 95% PI of the development cohort. Correlation analysis indicated that longer hospital stays and Post-Traumatic Amnesia (PTA) were linked to higher prediction errors, while higher BI scores at admission and focal injuries in Traumatic Brain Injury (TBI) were associated with lower errors. The web-based tool included a page for applying the models, one for visualising recovery trajectories via an interactive flow diagram, and another for accessing detailed model information.
Discussion
The implementation of prediction models involves several key phases, beginning with structured data collection, model devel- opment, and evaluation. Once the model demonstrates sufficient performance, it can be implemented into clinical practice. To maintain its relevance, continuous monitoring and updating are essential. In this study, we focused on two main components— external validation of the prediction models and the practical implementation through the development of a user-friendly tool. By addressing these, we have taken significant steps towards implementing prediction models into clinical practice. These components underscore the importance of structured data collection, rigorous validation, and practical application to ensure the models’ effectiveness in real-world settings. By implementing prediction models, we aim to employ a data-driven approach that brings us closer to precision neurorehabilitation.