User-centered co-development of Artificial Intelligence applications

Abstract

Vertebral fractures are the most common osteoporotic fractures, with a prevalence of 12-20% in Europe, making them a major health problem because of the associated morbidity, mortality and costs. Without adequate treatment, vertebral fractures are often followed by subsequent fractures, leading to further invalidation and deterioration of health. Therefore, early detection of vertebral fractures is important so preventative treatment can be initiated.

Vertebral fracture assessment (VFA) using Dual-Energy X-ray Absorptiometry (DXA) equipment is an imaging technique in which a lateral image of the spine is made. With vertebral morphometry, vertebral heights are measured and fractures are identified when vertebral height is lower than expected. However, vertebral morphometry is time and labor-intensive, and is subject to inter-operator variability. Automating VFA using Artificial Intelligence (AI) could help to overcome these limitations. We employed a co-development approach, aiming to create an AI-based tool to automatically perform vertebral morphometry on VFA images to identify vertebral fractures.

Firstly, we conducted a literature review to investigate the current use of AI in quantitative DXA imaging in a broad sense (Chapter 2). Besides VFA, other quantitative parameters describing bone macrogeometry and microgeometry can be extracted from DXA images. Incorporating these risk factors into multivariate prediction models could improve the identification of those at risk of fracture and help in clinical decision making. Although still in development, AI has been successfully applied in aid of fracture risk assessment, showing promising results.

In Chapter 3, the results of our reader study are described, evaluating VFA with manual vertebral morphometry as it is currently performed. This study served as a baseline measurement to quantify the effort needed to perform manual VFA and assess the potential value of automating VFA. The average annotation time per VFA image was 259 seconds. Although the intraclass correlation for vertebral height measurements between different readers was high, inter-observer agreement for fracture classification was only poor to moderate.

Together with our industry partner, we developed an AI-based software tool to perform vertebral morphometry. This tool is still in development, and we evaluated its current performance and potential impact in the study described in Chapter 4. Although its current standalone performance is suboptimal and shows room for improvement, this initial investigation showed that automated VFA has the potential to significantly reduce the required reader time.

Finally, in Chapter 5 we reflect on our user-centered co-development process and the next steps to bring our AI tool to market. We believe that collaboration between academic healthcare institutions and industry are essential for successful development of AI products. Validation of these products throughout the development process and actively involving intended users should be done. In the near future, vertebral fracture assessment can be supported by our AI-based application, potentially leading to lower annotation times and improving clinical workflow. However, further improvements have to be made and independent validation is required before market access.