Burying the Lead
Adjusting Goals to Manage Functional Limitations of AI Tools in Healthcare
Jacqueline Kernahan (TU Delft - Information and Communication Technology)
Richard Bartels ( University Medical Centre Utrecht)
Mark de Reuver (TU Delft - Information and Communication Technology)
Daniel Oberski (Universiteit Utrecht)
Roel Dobbe (TU Delft - Information and Communication Technology)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Artificial intelligence based tools are being developed for decision support in healthcare, however, they are frequently found to lack the required functionality to achieve the clinical goals for which they were built. This results in wasted time, money and resources for hospitals attempting to implement and operate such tools. To determine how functionality issues can be resolved prior to tool implementation, it is necessary to understand why such tools are being designed and then built. Our research focuses on clinical decision support tools with functionality issues arising from target variable invalidity. In this paper, we analyze published articles which present clinical decision support tool designs related to clinical goals. These tools use machine learning models trained on electronic health record data. We find that design decisions driven by data availability can introduce construct invalidity in clinical decision support tool designs, leading to an inability of the tool to address the clinical goal. We observe that alternative goals to the main clinical goal are used to justify continued development. We show that functional limitations of the tool related to the clinical goal can be obscured by imprecise terminology in the model’s stated functionality. Finally, we highlight the need for reconsidered approaches to dataset creation, defining success criteria, and the reporting and transparency of research outcomes as they relate to clinical goals.