In a recent commentary, Aboodi (2025) has criticized our (Buijsman et al., 2025) concern with inauthentic value shifts [IVS] that can occur through human-AI interactions. We presented emerging evidence that such interactions can lead to unperceived changes in values, which can lead to an IVS in one’s practical identity. Such an alienation from one’s identity is, in our opinion, a problem that needs to be accounted for in the design of AI systems. Since AI systems such as LLMs tend to be closely aligned to WEIRD values (see Atari et al., 2023 for empirical evidence that LLMs align with US cultural values), we additionally positioned this as a special concern for non-WEIRD cultures. Aboodi (2025) argues that (1) while inauthentic value shifts may be an issue if they are due to manipulation, they are not an independent problem and (2) the very concern with inauthenticity and autonomy is unique to WEIRD cultures. [...]

Do we need explanations of AI outputs in order to use AI systems (in high-risk settings)? This question has been actively debated recently, with one group denying that explanations are needed as long as the AI system is sufficiently accurate. What matters, according to them, is that outcomes improve. The other group argues that we have procedural reasons, centered around autonomy and self-advocacy, which trump outcome-based arguments to the contrary. I here present a set of arguments to show that outcome-based arguments should in fact also favor explainability for many of the current systems, as challenges with human oversight and accountability often lead to worse overall outcomes even if a more accurate AI system is integrated. Critics of explainability overlooked the fact that AI operates within a broader socio-technical system, and its accuracy alone tells us little of the final outcomes. In addition, I consolidate the procedural arguments and present a view of the upshot of these arguments. On this, we should avoid applications of AI that largely replace decision-making (relegating humans to the position of checking outputs). We can, however, use AI in other roles even for high-risk decision making while conforming to all of the requirements set by both outcome-based and procedural arguments. What matters, in the end, is the ability to explain decisions, and with the right role for AI that is possible even when supported by opaque systems.

We would like to have a wide range of explanations for the behaviour of machine learning systems. However, how should we understand these explanations? Typically, attempts to clarify what an explanations for questions such as ‘why am I getting this output for these inputs?’ have been approached from the philosophy of science, through an analogy with scientific (and often causal) explanations. I show that ML systems are best thought of as noncausal, specifically mathematical objects. We should therefore interpret these explanations differently, through analogy with mathematical explanations. I show that this still allows us to use much of the same theoretical apparatus, and argue that the asymmetry of many of the standard ML explanations can be accounted for in virtue of the link these systems have with concrete implementations.

This article examines the ethical and moral implications of automation bias in high-stakes decision-making contexts. Drawing on empirical studies, we distinguish between weak automation bias, where users follow system’s automated cues (or its silence) without consulting readily accessible evidence that contradicts them, and strong automation bias, where users follow such cues (or their absence) even when they are aware of such evidence. While weak automation bias, in our view, resembles automation-based complacency and is plausibly associated with negligence on the part of the human operator, strong automation bias reveals an excessive and unwarranted transfer of trust from operators to automated systems which results in epistemic deference of the former to the prompts of the latter. We argue that what is ethically and morally troubling about this form of deference, is that it interferes with the exercise of the operators’ autonomous agency as well as with their duty to exercise human judgment in high-stakes decision-making contexts. To mitigate these effects, we discuss two design-based tools introducing epistemic friction - Reflection Machines (RMs) and defeaters - which ultimately aim at cultivating critical trust in the interaction between human operators and decision-support systems.

Integrating AI systems into workflows risks undermining the competence of the people supported by them, specifically due to a loss of meta-cognitive competence. We discuss a recent suggestion to mitigate this through better uncertainty quantification. While this is certainly a step in the right direction, there is a question whether users are sufficiently supported to engage in critical reflection with literacy and tools alone. We therefore suggest that socio-technical system design focused on the role of AI systems is crucial to preserving autonomy, even when supported by uncertainty quantification.

AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy—the capacity for self-governed action within defined realms of skill or expertise—remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns—including careful role specification, implementation of defeater mechanisms, and support for reflective practice—that can help maintain domain-specific autonomy while leveraging AI capabilities. This framework provides concrete guidance for developing AI systems that enhance rather than diminish human agency within specialized domains of action.

Recently, several large tech companies have pushed the notion of AI assistants into the public debate. These envisioned agents are intended to far outshine current systems, as they are intended to be able to manage our affairs as if they are personal assistants. In turn, this ought to give users a leg up, as one prominent tech exec has put it. However, it remains to be seen how these Personal AI Assistants (PAIAs) are implemented, and critical reflection on how and whether they can be implemented in a responsible way is needed. Currently, such agents are undertheorized and this may cause us to misunderstand their value and capacity. In this paper, we explore and critique the potential for responsible implementation by considering some design requirements based on the notion of meaningful human control. If we desire to have control over such assistants, then we need to be able to do so meaningfully and effectively. In looking at the design requirements, we run into the issue that their broad and differing capacities make any kind of design requirements hard because there are simply no standards to which we can measure PAIAs. Furthermore, it seems that the implementation of these assistants will be a matter of trade-offs both in capacities and in values, which will likely lead to enhancement for some rather than an improvement for all.

Artificial Intelligence (AI) in healthcare holds transformative potential but faces critical challenges in ethical accountability and systemic inequities. Biases in AI models, such as lower diagnosis rates for Black women or gender stereotyping in Large Language Models, highlight the urgent need to address historical and structural inequalities in data and development processes. Disparities in clinical trials and datasets, often skewed toward high-income, English-speaking regions, amplify these issues. Moreover, the underrepresentation of marginalized groups among AI developers and researchers exacerbates these challenges. To ensure equitable AI, diverse data collection, federated data-sharing frameworks, and bias-correction techniques are essential. Structural initiatives, such as fairness audits, transparent AI model development processes, and early registration of clinical AI models, alongside inclusive global collaborations like TRAIN-Europe and CHAI, can drive responsible AI adoption. Prioritizing diversity in datasets and among developers and researchers, as well as implementing transparent governance will foster AI systems that uphold ethical principles and deliver equitable healthcare outcomes globally.

What variables should be used to get explanations (of AI systems) that are easily interpretable? The challenge to find the right degree of abstraction in explanations, also called the ‘variables problem’, has been actively discussed in the philosophy of science. The challenge is striking the right balance between specificity and generality. Concepts such as proportionality and exhaustivity are investigated and discussed. We propose a new and formal definition based on Kolmogorov complexity and argue that this corresponds to our intuitions about the right level of abstraction. First, we require that variables are uniform, so that they cannot be decomposed into less abstract variables without increasing the Kolmogorov complexity. Next, uniform variables are optimal for an explanation if they can compose the explanation without increasing its Kolmogorov complexity. For this, the concepts K-decomposability and K-composability of sets are defined. Explanations of a certain instance should encompass a maximal set of instances without being K-decomposable. Although Kolmogorov complexity is uncomputable and depends on the choice of programming language, we show that it can be used effectively to evaluate and reason about explanations, such as in the evaluation of XAI methods.

The advances in machine learning (ML)-based systems in medicine give rise to pressing epistemological and ethical questions. Clinical decisions are increasingly taken in highly digitised work environments, which we call artificial epistemic niches. By considering the case of ML systems in life-critical healthcare settings, we investigate (1) when users’ reliance on these systems can be characterised as epistemic dependence and (2) how this dependence turns into what we refer to as harmful epistemic dependence of clinical professionals on medical ML. The latter occurs when the impossibility of critically assessing the soundness of a system’s output in situ implies a moral obligation to comply with its recommendation since a failure to do so constitutes a moral risk that cannot be justified then and there. We analyse the epistemic and moral consequences of harmful epistemic dependence on the status of medical professionals. We conclude by assessing how a suitable design of the epistemic niche can address the problem.

Objective
As hospitals increasingly adopt artificial intelligence (AI) to manage rising patient volumes, workforce shortages and healthcare costs, concerns about ethical implementation have become prominent. This systematic review aims to assess how hospital-focused AI literature addresses the WHO’s six ethical AI principles—autonomy; well-being and safety; transparency and explainability; responsibility and accountability; inclusiveness and equity; and responsiveness and sustainability.

Methods and analysis
A systematic review (PROSPERO registration: CRD42022347871) was conducted by searching Embase, MEDLINE ALL, Web of Science and the Cochrane Central Register of Controlled Trials from inception to December 2023, supplemented by Google Scholar. English-language studies describing AI (machine learning, deep learning, predictive analytics) relevant to inpatient settings and referencing at least one WHO principle were included. Two reviewers independently screened titles, abstracts and full texts, extracting data on publication year, country, study design, AI type, technology readiness level and ethical considerations. Discrepancies were resolved by consensus.

Results
Of 4770 unique records, 673 were included. Most (83%) originated from high-income countries, with publication volume rising sharply after 2021. Of these, 558 (83%) addressed at least one WHO principle in depth, most frequently inclusiveness and equity (49%), transparency and explainability (45%) and autonomy (42%). Well-being and safety (26%) and responsibility and accountability (29%) were less frequently covered, while responsiveness and sustainability (6%) was rarely explored. Among 44 studies developing AI applications with technology readiness levels 1–6, ethical principles were acknowledged but rarely operationalised.

Conclusion
Hospital-based AI research demonstrates increasing attention to ethical principles but lacks comprehensive application, particularly regarding sustainability. High-income countries dominate this discourse, underscoring the need for broader global engagement. To achieve equitable, safe and sustainable AI in clinical practice, clearer operational guidance and more inclusive collaboration is warranted.

Importance: Artificial intelligence (AI) presents transformative opportunities to address the increasing challenges faced by health care systems globally. Particularly, in data-rich environments, such as intensive care units (ICUs), AI could assist in enhancing clinical decision-making, streamline workflows, and improve patient outcomes. Despite these promising applications, the practical implementation of AI in clinical settings remains limited. Objective: To systematically evaluate AI system operationalization in the ICU, focusing on the AI field's progress over time, technical maturity, and risk of bias. Evidence Review: In this systematic review, 5 databases (Embase, MEDLINE ALL, Web of Science Core Collection, Cochrane Central Register of Controlled Trials, and Google Scholar) were searched for studies published from July 28, 2020, to June 10, 2024. Eligible studies evaluated AI applications designed for use within ICUs for adults (aged ≥16 years) and used data collected during ICU stays. Two reviewers independently screened titles and abstracts, with a third reviewer resolving disagreements. Data extraction included AI application aims, dataset origins, technology readiness level (TRL) categorization, and the use of reporting standards. Risk of bias was assessed using the PROBAST (Prediction Model Study Risk of Bias Assessment Tool). Findings: Of 17 401 screened records, 1263 studies met the inclusion criteria. A total of 936 studies (74% of all studies) were classified as TRL 4 or below, indicating early-stage development or initial validation. Among these, 447 (37%) used internal datasets, 562 (46%) used MIMIC (Medical Information Mart for Intensive Care) datasets (I-IV), and 78 (6%) used the open-source eICU Collaborative Research Database. External validation (TRL 5) was achieved by 24% of studies. Only 25 (2%) progressed to clinical integration (TRL≥6), with no studies reaching full implementation (TRL 9). Although approximately half of generative AI models reached a higher TRL (14 [47%] with TRL 5), none reached clinical integration. Additionally, only 207 studies (16%) referenced reporting standards, with adherence modestly increasing from 14% in 2021 to 23% in 2024. High risk of bias was identified in 581 of 1103 studies (53%), primarily due to methodologic shortcomings in the analysis domain. Conclusions and Relevance: Despite substantial growth in AI research within intensive care medicine in recent years, the transition from development to clinical implementation still remains limited and has made little progress over time. A paradigm shift is urgently required in the medical literature-one that moves beyond retrospective validation toward the operationalization and prospective testing of AI for tangible clinical impact.

ChatGPT is a powerful language model from OpenAI that is arguably able to comprehend and generate text. ChatGPT is expected to greatly impact society, research, and education. An essential step to understand ChatGPT’s expected impact is to study its domain-specific answering capabilities. Here, we perform a systematic empirical assessment of its abilities to answer questions across the natural science and engineering domains. We collected 594 questions on natural science and engineering topics from 198 faculty members across five faculties at Delft University of Technology. After collecting the answers from ChatGPT, the participants assessed the quality of the answers using a systematic scheme. Our results show that the answers from ChatGPT are, on average, perceived as “mostly correct”. Two major trends are that the rating of the ChatGPT answers significantly decreases (i) as the educational level of the question increases and (ii) as we evaluate skills beyond scientific knowledge, e.g., critical attitude.

Concepts are an important construct in semantics, based on which humans understand the world with various levels of abstraction. With the recent advances in explainable artificial intelligence (XAI), concept-level explanations are receiving an increasing amount of attention from the broad research community. However, laypeople may find such explanations difficult to digest due to the potential knowledge gap and the concomitant cognitive load. Inspired by prior work that has explored analogies and sensemaking, we argue that augmenting concept-level explanations with analogical inference information from commonsense knowledge can be a potential solution to tackle this issue. To investigate the validity of our proposition, we first designed an effective analogy-based explanation generation method and collected 600 analogy-based explanations from 100 crowd workers. Next, we proposed a set of structured dimensions for the qualitative assessment of such explanations, and conducted an empirical evaluation of the generated analogies with experts. Our findings revealed significant positive correlations between the qualitative dimensions of analogies and the perceived helpfulness of analogy-based explanations, suggesting the effectiveness of the dimensions. To understand the practical utility and the effectiveness of analogybased explanations in assisting human decision-making, we conducted a follow-up empirical study (N = 280) on a skin cancer detection task with non-expert humans and an imperfect AI system. Thus, we designed a between-subjects study spanning five different experimental conditions with varying types of explanations. The results of our study confirmed that a knowledge gap can prevent participants from understanding concept-level explanations. Consequently, when only the target domain of our designed analogy-based explanation was provided (in a specific experimental condition), participants demonstrated relatively more appropriate reliance on the AI system. In contrast to our expectations, we found that analogies were not effective in fostering appropriate reliance. We carried out a qualitative analysis of the open-ended responses from participants in the study regarding their perceived usefulness of explanations and analogies. Our findings suggest that human intuition and the perceived plausibility of analogies may have played a role in affecting user reliance on the AI system. We also found that the understanding of commonsense explanations varied with the varying experience of the recipient user, which points out the need for further work on personalization when leveraging commonsense explanations. In summary, although we did not find quantitative support for our hypotheses around the benefits of using analogies, we found considerable qualitative evidence suggesting the potential of high-quality analogies in aiding non-expert users in their decision making with AI-assistance. These insights can inform the design of future methods for the generation and use of effective analogy-based explanations.

Machine learning techniques are driving — or soon will be driving — much of scientific research and discovery. Can they function as models similar to more traditional modeling techniques in scientific contexts? Or might they replace models altogether if they deliver sufficient predictive accuracy? These questions cut across at least two types of applications of machine learning models. First, machine learning models are used to study the brain, where neural networks might represent aspects of neural activity. The principal question here is: Can the uses of neural networks provide scientific explanations and models for neuroscience? Second, machine learning models are applied in science more broadly, where representational links are less clear. What are the epistemic implications of machine learning in those areas? Can they replace more traditional scientific models of phenomena, or is mere predictive accuracy sufficient? These two strands are finally brought together to create an overview of the (epistemic) role machine learning can play in scientific modeling.

This chapter explores the principles and frameworks of human-centered artificial intelligence (AI), specifically focusing on user modeling, adaptation, and personalization. It introduces a four-dimensional framework comprising paradigms, actors, values, and levels of realization that should be considered in the design of human-centered AI systems. This framework highlights a perspective-taking approach with four lenses of technology-centric, user-centric, human-centric, and future-centric perspectives. Ethical considerations, transparency, fairness, and accountability, among other aspects, are highlighted as values when developing and deploying AI systems. The chapter further discusses the corresponding human values for each of these concepts. Opportunities and challenges in human-centered AI are examined, including the need for interdisciplinary collaboration and the complexity of addressing diverse perspectives. Human-centered AI provides valuable insights for designing AI systems that prioritize human needs, values, and experiences while considering ethical and societal implications.

With the widespread use of artificial intelligence, it becomes crucial to provide information about these systems and how they are used. Governments aim to disclose their use of algorithms to establish legitimacy and the EU AI Act mandates forms of transparency for all high-risk and limited-risk systems. Yet, what should the standards for transparency be? What information is needed to show to a wide public that a certain system can be used legitimately and responsibly? I argue that process-based approaches fail to satisfy, as knowledge about the development process is insufficient to predict the properties of the resulting system. Current outcome-based approaches [Mitchell et al., 2019; Loi et al., 2021] are also criticized for a lack of attention to the broader socio-technical system and failure to account for empirical results that show that people care about more than just the outcomes of a process [as reported by Meyerson et al. (Procedural justice and relational theory: Empirical, philosophical, and legal perspectives, Taylor & Francis, 2021)]. Instead, I propose value-based transparency, on which the information we need to provide is what values have been considered in the design and how successful these have been realized in the final system. This can handle the objections to other frameworks, matches with current best practices on the design of responsible AI and provides the public with information on the crucial aspects of a system’s design.

Relevancy is a prevalent term in value alignment. We either need to keep track of the relevant moral reasons, we need to embed the relevant values, or we need to learn from the relevant behaviour. What relevancy entails in particular cases, however, is often ill-defined. The reasons for this are obvious, it is hard to define relevancy in a way that is both general and concrete enough to give direction towards a specific implementation. In this paper, we describe the inherent difficulty that comes along with defining what is relevant to a particular situation. Simply due to design and the way an AI system functions, we need to state or learn particular goals and circumstances under which that goal is completed. However, because of both the changing nature of the world and the varied wielders and users of such implements, misalignment occurs, especially after a longer amount of time. We propose a way to counteract this by putting contestability front and centre throughout the lifecycle of an AI system, as it can provide insight into what is actually relevant at a particular instance. This allows designers to update the applications in such a manner that they can account for oversight during design.

Process reliabilist accounts claim that a belief is justified when it is the result of a reliable belief-forming process. Yet over what range of possible token processes is this reliability calculated? I argue against the idea that all possible token processes (in the actual world, or some other subset of possible worlds) are to be considered using the case of a user acquiring beliefs based on the output of an AI system, which is typically reliable for a substantial local range but unreliable when all possible inputs are considered. I show that existing solutions to the generality problem imply that these cases cannot be solved by a more fine-grained typing of the belief-forming process. Instead, I suggest that reliability is evaluated over a range restricted by the content of the actual belief and by the similarity of the input to the actual input.