Clearing the Air: An Exploration of Pulmonologists' Needs and Intents in XAI Solutions for Respiratory Medicine

Abstract

Despite the low adoption rates of artificial intelligence (AI) in respiratory medicine, its potential to improve patient outcomes is substantial. To facilitate the integration of AI systems into the clinical setting, it is essential to prioritise the development of explainable AI (XAI) solutions that improve the understanding of the AI predictions. These XAI solutions empower clinicians to collaborate effectively with AI systems, thereby enhancing the overall outcomes for patients in respiratory medicine. Unfortunately, the lack of user-centric studies in this domain has made it challenging to identify the specific aspects of explainability that are most effective in improving the adoption of AI in the real-world environment. To address this gap, we conducted a mixed-methods study of clinicians in respiratory medicine to identify the most relevant and crucial aspects of XAI solutions. Our study focused on understanding how XAI can be effectively translated into clinical practice by leveraging the expertise of doctors in the field. Because of the lack of knowledge about XAI concepts among pulmonologists a different approach is taken to regular user-centric XAI research and no direct examples of state-of-the-art XAI solutions are used. Rather the expertise of doctors is used to make them implicitly identify their needs and intents. Our findings reveal that the successful adoption of XAI solutions in respiratory medicine requires tailored solutions that address communication barriers, promote patient-centric care, and overcome AI adoption challenges. The study highlights the significance of task-specific visualisations, comprehensive explanations, preferred granularity, and the ability to mimic human judgement in successful XAI solutions. Trust and collaboration between clinicians and AI systems are essential for effective adoption, wherein AI is perceived as a colleague rather than a replacement. This ensures that clinicians can easily understand and work with the model predictions, ultimately leading to improved patient outcomes. By aligning XAI design with the needs and intents of pulmonologists, we established the importance of Co-designing solutions with domain experts and embedding XAI within clinical workflows emerged as key strategies. Our research underscores the imperative of transparency, extended validation, and continuous alignment of AI technologies with medical values. By following these principles, XAI solutions can be developed to enhance the diagnosis and treatment of respiratory illnesses, ultimately improving patient outcomes in respiratory medicine.