M.L. Tielman
Please Note
61 records found
1
"what's on your mind?"
Understanding the Development of Multidimensional Trust in Social Robots
Social AI for a Healthier Lifestyle
Four Competencies to Manage and Prevent Chronic Diseases
Advancing Human-Machine Teaming
Definitions, Challenges, Future Directions
Agent Allocation of Moral Decisions in Human-Agent Teams
Raise Human Involvement and Explain Potential Consequences
"even explanations will not help in trusting [this] fundamentally biased system"
A Predictive Policing Case-Study
In today's society, where Artificial Intelligence (AI) has gained a vital role, concerns regarding user's trust have garnered significant attention. The use of AI systems in high-risk domains have often led users to either under-trust it, potentially causing inadequate reliance or over-trust it, resulting in over-compliance. Therefore, users must maintain an appropriate level of trust. Past research has indicated that explanations provided by AI systems can enhance user understanding of when to trust or not trust the system. However, the utility of presentation of different explanations forms still remains to be explored especially in high-risk domains. Therefore, this study explores the impact of different explanation types (text, visual, and hybrid) and user expertise (retired police officers and lay users) on establishing appropriate trust in AI-based predictive policing. While we observed that the hybrid form of explanations increased the subjective trust in AI for expert users, it did not led to better decision-making. Furthermore, no form of explanations helped build appropriate trust. The findings of our study emphasize the importance of re-evaluating the use of explanations to build [appropriate] trust in AI based systems especially when the system's use is questionable. Finally, we synthesize potential challenges and policy recommendations based on our results to design for appropriate trust in high-risk based AI-based systems.
Appropriate trust is an important component of the interaction between people and AI systems, in that "inappropriate"trust can cause disuse, misuse, or abuse of AI. To foster appropriate trust in AI, we need to understand how AI systems can elicit appropriate levels of trust from their users. Out of the aspects that influence trust, this article focuses on the effect of showing integrity. In particular, this article presents a study of how different integrity-based explanations made by an AI agent affect the appropriateness of trust of a human in that agent. To explore this, (1) we provide a formal definition to measure appropriate trust, (2) present a between-subject user study with 160 participants who collaborated with an AI agent in such a task. In the study, the AI agent assisted its human partner in estimating calories on a food plate by expressing its integrity through explanations focusing on either honesty, transparency, or fairness. Our results show that (a) an agent who displays its integrity by being explicit about potential biases in data or algorithms achieved appropriate trust more often compared to being honest about capability or transparent about the decision-making process, and (b) subjective trust builds up and recovers better with honesty-like integrity explanations. Our results contribute to the design of agent-based AI systems that guide humans to appropriately trust them, a formal method to measure appropriate trust, and how to support humans in calibrating their trust in AI.
Child helplines offer a safe and private space for children to share their thoughts and feelings with volunteers. However, training these volunteers to help can be both expensive and time-consuming. In this demo, we present Lilobot, a conversational agent designed to train volunteers for child helplines. Lilobot’s reasoning is based on the Belief-Desire-Intention (BDI) model, which simulates, for example, a bullied child who contacts the helpline through text. Users engage with Lilobot in a role-play format, taking on the volunteer’s role. Through this system, volunteers can practice applying the Five Phase Model, a conversational strategy helplines use. The training tool includes a trainer interface for monitoring and modifying Lilobot’s interactions. Trainers can also create new conversational scenarios through an authoring tool. An initial evaluation led to enhancements in Lilobot’s knowledge base and intent recognition, addressing the main issues encountered by participants. The components used to implement the system were Java Spring for the BDI model and the authoring tool, Rasa for Natural Language Understanding, PostgreSQL for the database, and Vue.js for the front-end. This tool aims to provide volunteers with consistent, interactive training, enhancing their counselling skills in a controlled environment.