Epistemic Justice as a Condition for Meaningful Human Control Over Medical AI

Abstract

AI technologies are increasingly deployed in medical care and decision-making, and efforts geared toward conceptualizing how human control over AI systems can be meaningful, i.e., sufficient to preserve the relevant human agency and responsibility, are mounting. However, a suitable conceptualization of Meaningful Human Control (MHC) explicitly tailored to AI-mediated clinical practice is still underdeveloped. This paper addresses this research gap in two ways. First, it applies the framework of Meaningful Human Control as reason-responsiveness to the medical field. Second, it shows that considerations of epistemic (in)justice ought to be included in efforts toward securing MHC in medical care. MHC demands that the moral reasons of relevant agents be made available to the socio-technical system in which the AI operates. However, this requirement can be compromised by epistemic injustices, i.e., when patients’ and clinicians’ epistemic offerings to the medical discourse are unduly limited. The paper argues that epistemic justice is an important enabler for MHC, and, when properly understood, MHC is a crucial element in a strategy to promote a more just medical AI. Since epistemic injustice depends on power asymmetries and systemic inequalities, achieving epistemic justice and MHC over medical AI requires addressing power and justice issues in the development and use of (new) medical AI.