Print Email Facebook Twitter Integrity-based Explanations for Fostering Appropriate Trust in AI Agents Title Integrity-based Explanations for Fostering Appropriate Trust in AI Agents Author Mehrotra, S. (TU Delft Interactive Intelligence) Centeio Jorge, C. (TU Delft Interactive Intelligence) Jonker, C.M. (TU Delft Interactive Intelligence) Tielman, M.L. (TU Delft Interactive Intelligence) Date 2024 Abstract Appropriate trust is an important component of the interaction between people and AI systems, in that ‘inappropriate’ trust can cause disuse, misuse or abuse of AI. To foster appropriate trust in AI, we need to understand how AI systems can elicit appropriate levels of trust from their users. Out of the aspects that influence trust, this paper focuses on the effect of showing integrity. In particular, this paper presents a study of how different integrity-based explanations made by an AI agent affect the appropriateness of trust of a human in that agent. To explore this, (1) we provide a formal definition to measure appropriate trust, (2) present a between-subject user study with 160 participants who collaborated with an AI agent in such a task. In the study, the AI agent assisted its human partner in estimating calories on a food plate by expressing its integrity through explanations focusing on either honesty, transparency or fairness. Our results show that (a) an agent who displays its integrity by being explicit about potential biases in data or algorithms achieved appropriate trust more often compared to being honest about capability or transparent about the decision-making process, and (b) subjective trust builds up and recovers better with honesty-like integrity explanations. Our results contribute to the design of agent-based AI systems that guide humans to appropriately trust them, a formal method to measure appropriate trust, and how to support humans in calibrating their trust in AI. Subject Intelligent AgentsHCITrustHonestyAppropriate TrustArtificial AgentsIntegrityExplanationsFairnessTransparency To reference this document use: http://resolver.tudelft.nl/uuid:369651f0-4fe9-48da-9723-cb663c876a00 DOI https://doi.org/10.1145/3610578 ISSN 2160-6455 Source ACM Transactions on Interactive Intelligent Systems, 14 (1) Part of collection Institutional Repository Document type journal article Rights © 2024 S. Mehrotra, C. Centeio Jorge, C.M. Jonker, M.L. Tielman Files PDF 3610578.pdf 3.51 MB Close viewer /islandora/object/uuid:369651f0-4fe9-48da-9723-cb663c876a00/datastream/OBJ/view