AI technologies are increasingly deployed in medical care and decision-making, and efforts geared toward conceptualizing how human control over AI systems can be meaningful, i.e., sufficient to preserve the relevant human agency and responsibility, are mounting. However, a suitable conceptualization of Meaningful Human Control (MHC) explicitly tailored to AI-mediated clinical practice is still underdeveloped. This paper addresses this research gap in two ways. First, it applies the framework of Meaningful Human Control as reason-responsiveness to the medical field. Second, it shows that considerations of epistemic (in)justice ought to be included in efforts toward securing MHC in medical care. MHC demands that the moral reasons of relevant agents be made available to the socio-technical system in which the AI operates. However, this requirement can be compromised by epistemic injustices, i.e., when patients’ and clinicians’ epistemic offerings to the medical discourse are unduly limited. The paper argues that epistemic justice is an important enabler for MHC, and, when properly understood, MHC is a crucial element in a strategy to promote a more just medical AI. Since epistemic injustice depends on power asymmetries and systemic inequalities, achieving epistemic justice and MHC over medical AI requires addressing power and justice issues in the development and use of (new) medical AI.

Achieving trustworthy AI is increasingly considered an essential desideratum to integrate AI systems into sensitive societal fields, such as criminal justice, finance, medicine, and healthcare, among others. For this reason, it is important to spell out clearly its characteristics, merits, and shortcomings. This article is the first survey in the specialized literature that maps out the philosophical landscape surrounding trust and trustworthiness in AI. To achieve our goals, we proceed as follows. We start by discussing philosophical positions on trust and trustworthiness, focusing on interpersonal accounts of trust. This allows us to explain why trust, in its most general terms, is to be understood as reliance plus some “extra factor”. We then turn to the first part of the definition provided, i.e., reliance, and analyze two opposing approaches to establishing AI systems’ reliability. On the one hand, we consider transparency and, on the other, computational reliabilism. Subsequently, we focus on debates revolving around the “extra factor”. To this end, we consider viewpoints that most actively resist the possibility and desirability of trusting AI systems before turning to the analysis of the most prominent advocates of it. Finally, we take up the main conclusions of the previous sections and briefly point at issues that remain open and need further attention.

The advances in machine learning (ML)-based systems in medicine give rise to pressing epistemological and ethical questions. Clinical decisions are increasingly taken in highly digitised work environments, which we call artificial epistemic niches. By considering the case of ML systems in life-critical healthcare settings, we investigate (1) when users’ reliance on these systems can be characterised as epistemic dependence and (2) how this dependence turns into what we refer to as harmful epistemic dependence of clinical professionals on medical ML. The latter occurs when the impossibility of critically assessing the soundness of a system’s output in situ implies a moral obligation to comply with its recommendation since a failure to do so constitutes a moral risk that cannot be justified then and there. We analyse the epistemic and moral consequences of harmful epistemic dependence on the status of medical professionals. We conclude by assessing how a suitable design of the epistemic niche can address the problem.

In this paper, we discuss epistemic and ethical concerns brought about by machine learning (ML) systems implemented in medicine. We begin by fleshing out the logic underlying a common approach in the specialized literature (which we call the informativeness account). We maintain that the informativeness account limits its analysis to the impact of epistemological issues on ethical concerns without assessing the bearings that ethical features have on the epistemological evaluation of ML systems. We argue that according to this methodological approach, epistemological issues are instrumental to and autonomous of ethical considerations. This means that the informativeness account considers epistemological evaluation uninfluenced and unregulated by an ethical counterpart. Using an example that does not square well into the informativeness account, we argue for ethical assessments that have a substantial influence on the epistemological assessment of ML and that such influence should not be understood as merely informative but rather regulatory. Drawing on the case analyzed, we claim that within the theoretical framework of the informativeness approach, forms of epistemic injustice—especially epistemic objectification—remain unaddressed. Our analysis should motivate further research investigating the regulatory role that ethical elements play in the epistemology of ML.

The principle of trust has been placed at the centre as an attitude for engaging with clinical machine learning systems. However, the notions of trust and distrust remain fiercely debated in the philosophical and ethical literature. In this article, we proceed on a structural level ex negativo as we aim to analyse the concept of “institutional distrustworthiness” to achieve a proper diagnosis of how we should not engage with medical machine learning. First, we begin with several examples that hint at the emergence of a climate of distrust in the context of medical machine learning. Second, we introduce the concept of institutional trustworthiness based on an expansion of Hawley’s commitment account. Third, we argue that institutional opacity can undermine the trustworthiness of medical institutions and can lead to new forms of testimonial injustices. Finally, we focus on possible building blocks for repairing institutional distrustworthiness.