Circular Image

S.E. Carter

info

Please Note

3 records found

More than Manipulation

Journal article (2026) - Stefan Buijsman, Sarah E. Carter, Juan Pablo Bermúdez
In a recent commentary, Aboodi (2025) has criticized our (Buijsman et al., 2025) concern with inauthentic value shifts [IVS] that can occur through human-AI interactions. We presented emerging evidence that such interactions can lead to unperceived changes in values, which can lead to an IVS in one’s practical identity. Such an alienation from one’s identity is, in our opinion, a problem that needs to be accounted for in the design of AI systems. Since AI systems such as LLMs tend to be closely aligned to WEIRD values (see Atari et al., 2023 for empirical evidence that LLMs align with US cultural values), we additionally positioned this as a special concern for non-WEIRD cultures. Aboodi (2025) argues that (1) while inauthentic value shifts may be an issue if they are due to manipulation, they are not an independent problem and (2) the very concern with inauthenticity and autonomy is unique to WEIRD cultures. [...] ...
Journal article (2025) - Stefan Buijsman, Sarah E. Carter, Juan Pablo Bermúdez
Integrating AI systems into workflows risks undermining the competence of the people supported by them, specifically due to a loss of meta-cognitive competence. We discuss a recent suggestion to mitigate this through better uncertainty quantification. While this is certainly a step in the right direction, there is a question whether users are sufficiently supported to engage in critical reflection with literacy and tools alone. We therefore suggest that socio-technical system design focused on the role of AI systems is crucial to preserving autonomy, even when supported by uncertainty quantification. ...

Preserving Human Autonomy in AI Decision-Support

Journal article (2025) - Stefan Buijsman, Sarah E. Carter, Juan Pablo Bermúdez
AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy—the capacity for self-governed action within defined realms of skill or expertise—remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns—including careful role specification, implementation of defeater mechanisms, and support for reflective practice—that can help maintain domain-specific autonomy while leveraging AI capabilities. This framework provides concrete guidance for developing AI systems that enhance rather than diminish human agency within specialized domains of action. ...