Much of the research on eXplainable Artificial Intelligence (XAI) has centered on providing transparency of machine learning models. More recently, the focus on human-centered approaches to XAI has increased. Yet, there is a lack of practical methods and examples on the integration of human factors into the development processes of AI-generated explanations that humans prove to uptake for better performance. This paper presents a case study of an application of a human-centered design approach for AI-generated explanations. The approach consists of three components: Domain analysis to define the concept & context of explanations, Requirements elicitation & assessment to derive the use cases & explanation requirements, and the consequential Multi-modal interaction design & evaluation to create a library of design patterns for explanations. In a case study, we adopt the DoReMi-approach to design explanations for a Clinical Decision Support System (CDSS) for child health. In the requirements elicitation & assessment, a user study with experienced paediatricians uncovered what explanations the CDSS should provide. In the interaction design & evaluation, a second user study tested the consequential interaction design patterns. This case study provided a first set of user requirements and design patterns for an explainable decision support system in medical diagnosis, showing how to involve expert end users in the development process and how to develop, more or less, generic solutions for general design problems in XAI.@en

The rapid advancement of technology empowered by artificial intelligence is believed to intensify the collaboration between humans and AI as team partners. Successful collaboration requires partners to learn about each other and about the task. This human-AI co-learning can be achieved by presenting situations that enable partners to share knowledge and experiences. In this paper we describe the development and implementation of a task context and procedures for studying co-learning. More specifically, we designed specific sequences of interactions that aim to initiate and facilitate the co-learning process. The effects of these interventions on learning were evaluated in an experiment, using a simplified virtual urban-search-and-rescue task for a human-robot team. The human participants performed a victim rescue- and evacuation mission in collaboration with a wizard-of-Oz (i.e., a confederate of the experimenter who executed the robot-behavior consistent with an ontology-based AI-model). The designed interaction sequences, formulated as Learning Design Patterns (LDPs), were intended to bring about co-learning. Results show that LDPs support the humans understanding and awareness of their robot partner and of the teamwork. No effects were found on collaboration fluency, nor on team performance. Results are used to discuss the importance of co-learning, the challenges of designing human-AI team tasks for research into this phenomenon, and the conditions under which co-learning is likely to be successful. The study contributes to our understanding of how humans learn with and from AI-partners, and our propositions for designing intentional learning (LDPs) provide directions for applications in future human-AI teams.@en

Decision support systems (DSS) have improved significantly but are more complex due to recent advances in Artificial Intelligence. Current XAI methods generate explanations on model behaviour to facilitate a user's understanding, which incites trust in the DSS. However, little focus has been on the development of methods that establish and convey a system's confidence in the advice that it provides. This paper presents a framework for Interpretable Confidence Measures (ICMs). We investigate what properties of a confidence measure are desirable and why, and how an ICM is interpreted by users. In several data sets and user experiments, we evaluate these ideas. The presented framework defines four properties: 1) accuracy or soundness, 2) transparency, 3) explainability and 4) predictability. These characteristics are realized by a case-based reasoning approach to confidence estimation. Example ICMs are proposed for -and evaluated on- multiple data sets. In addition, ICM was evaluated by performing two user experiments. The results show that ICM can be as accurate as other confidence measures, while behaving in a more predictable manner. Also, ICM's underlying idea of case-based reasoning enables generating explanations about the computation of the confidence value, and facilitates user's understandability of the algorithm.@en

The increasing use of ever-smarter AI-technology is changing the way individuals and teams learn and perform their tasks. In hybrid teams, people collaborate with artificially intelligent partners. To utilize the different strengths and weaknesses of human and artificial intelligence, a hybrid team should be designed upon the principles that foster successful human-machine learning and cooperation. The implementation of the identified principles sets a number of challenges. Machine agents should, just like humans, have mental models that contain information about the task context, their own role (self-awareness), and the role of others (theory of mind). Furthermore, agents should be able to express and clarify their mental states to partners. In this paper we identify six challenges for humans and machines to collaborate in an adaptive, dynamic and personalized fashion. Implications for research are discussed.@en