As human-agent collaboration grows increasingly prevalent, it is crucial to understand and enhance the interaction between humans and AI systems. Explainable AI is fundamental to this interaction, which involves agents conveying essential information to humans for decision-making. This paper investigates how adaptive explanations affect human supervision and trust in robotic systems. The study included 40 participants and compared baseline (non-adaptive) explanations with adaptive explanations. The results showed no significant difference between the two types of explanations; making explanations more abstract did not necessarily improve human supervision or increase trust in robots. ... Read more

In human-AI agent interactions, providing clear visual or textual explanations for the agent's actions and decisions is crucial for ensuring successful collaboration. This research investigates whether having the visual explanations displayed only on-demand, instead of having them consistently shown as the baseline, has an impact on the human supervisor's level of confidence and satisfaction with the AI agent. Therefore, a case study of 40 participants was conducted to explore this hypothesis and the participants were divided into 2 groups, one interacting with the on-demand condition, and the other with the baseline one. Through questionnaires, the participants' capacity and moral trust in the robot, the explainable artificial intelligence satisfaction, and the disagreement rate with the robot's decisions have been collected. Demographic data was gathered from the participants to explore whether their background could impact the collaboration. This data included the participants' gender, age, education, gaming experience, risk propensity, trust propensity, and utilitarianism. The resulting statistical analyses indicated no significant differences between the baseline and the on-demand conditions concerning trust and explanation satisfaction. This suggests that the overall collaboration was not primarily impacted by the frequency of visual explanations requested on demand. Although the results implied a high satisfaction with the interaction, further studies with more diverse user groups are recommended. Overall, this research reinforces the importance of transparency in decision-making processes during the collaboration between an AI agent and a human supervisor. ... Read more

With the rise of AI presence in various contexts and spheres of life, ensuring effective human-AI collaboration, especially in critical domains, is of utmost importance. Explanations given by AI agent can be of great assistance for this purpose. This study investigates the impact of global explanations, explaining general allocation rules, on human supervision and trust in AI within the critical domain of a firefighting scenario, where human and AI agent have to collaborate to save victims. To this end, a user study involving 40 participants was performed. The user study compared the baseline and global explanation scenario and the participants’ trust in the AI and explanation satisfaction were measured. The results indicated no significant differences between the two types of explanations, and in fact both achieved similar satisfactory outcomes. This suggests comparable effectiveness of global explanations in enhancing human-AI collaboration. Essentially, the insights of this study underscore the need for further exploration into contextual factors influencing the impact of global explanations and contribute to designing better human-AI teaming systems in dynamic and ethically sensitive environments. ... Read more

As artificially intelligent agents become integrated into various sectors, they require an analysis of their capacity to make moral decisions and of the influence of human supervision on their performance. This study investigates the impact of textual feature explanations on human supervision and the trust in a semi-autonomous firefighting robot named Brutus, which operates in a morally complex environment. Grounded in the field of Explainable AI (XAI), which seeks to render AI decisions transparent, this research compares textual and visual explanations’ effectiveness in conveying situational sensitivity during a simulated rescue operation. Through a detailed experimental setup using the MATRX software to simulate a burning office building, participants’ trust and understanding were assessed based on their interaction with Brutus using either textual or visual explanations. This study contributes to the broader discourse on AI ethics and the optimization of human-agent teaming in high-stakes scenarios. The findings suggest that textual explanations can enhance human supervision and trust, fostering greater engagement and satisfaction compared to visual explanations. ... Read more

The integration of robots in human-robot teams, particularly in high-stakes environments like firefighting, requires effective communication and decision-making to ensure safety and efficiency. This study explores the impact of adding contrastive explanations to feature attributions in robot explanations on human-robot teamwork during firefighting simulations. Contrastive explanations aim to improve human understanding by highlighting why a robot chose one decision over another using allocations of variables. The experiment involved 40 participants, divided into two groups, each interacting with either the baseline or contrastive version of the robot in the simulated environment. Results indicate that contrastive explanations significantly increased participants' capacity trust in the robot, though they did not significantly affect moral trust. Additionally, the results showed a lower satisfaction level with the explanations given by the robot. The disagreement rate between human decisions and robot actions was lower in the contrastive group, suggesting possible enhanced understanding and agreement with the robot's decisions. These findings underscore the potential of contrastive explanations to enhance trust and collaboration in human-robot teams, paving the way for more effective integration of robots in critical operations. Future research should focus on larger sample sizes and explore the inclusion of contrastive decisions made by the robot alongside explanations to further validate these findings. ... Read more

As autonomous systems are increasingly integrated as a team member for collaborative tasks, trust in human-agent teams (HAT) becomes crucial to foster success. In many real world scenarios, trust violations are expected, thus demanding the use of trust repair strategies to restore damaged trust. Previous research has shown that expressing regret and providing explanations are effective strategies to rebuild human-agent trust.
However, the role of various team dynamics, such as interdependence relationships, remains unexplored. The aim of this paper is to examine the influence of interdependence levels on the effectiveness of trust repair strategies. To investigate this, an experiment was conducted in a collaboration environment with an urban search and rescue mission. Two interdependence conditions were introduced to analyse their effect on trust and collaboration fluency.
No significant evidence was found to support a relationship between interdependence and trust repair or collaboration fluency. However, as this study only considers two interdependence conditions, there is much more room for future work to explore further. This study can bring meaningful insights to design and facilitate agents that are more trustworthy in human-agent collaboration settings. ... Read more

Interdependence relationships between humans and agents play a crucial role in the collaborative AI field. This research paper examines the impact of interdependence on trust violation, trust repair strategies, and collaboration fluency in human-AI teams. It compares independent cooperation and required interdependence approaches, focusing on collaborative AI, trust dynamics, and collaboration fluency. The paper presents a user study involving 30 participants in different interdependence conditions, analyzing data from trust surveys, collaboration fluency surveys, performance metrics, and AI agent idle time. The findings enhance understanding of human-agent collaboration fluency and as well as the interdependence relationship impact on trust. ... Read more

Intelligent agents are increasingly required to engage in collaboration with humans in the context of human-agent teams (HATs) to achieve shared goals. Interdependence is a fundamental concept in teamwork. It enables humans and robots to leverage their capabilities and collaboratively work towards a shared goal, fostering the development of trust through joint activities. Considering the great importance of trust, the effectiveness of trust repair strategies is crucial as they help mitigate the negative consequences of errors, enabling efficient collaboration between humans and robots. For this reason, the effectiveness of the trust repair strategies must be examined comprehensively by taking into consideration multiple factors, including the interdependence relationships within HATs. This paper aims to examine the impact of a mix of interdependence and independence relationships on trust violation and repair, but also on collaboration fluency. Thus, an experiment (n = 30) was conducted to study how interdependence affects trust violation, trust repair and collaboration fluency. Participants collaborated with a robot during a search and rescue mission in a simulated environment. Results show that there is a significant influence of interdependence on trust violations, but not on collaboration fluency or trust repair. Furthermore, the paper also emphasises the need for future research that investigates the effectiveness of trust repair strategies for HATs in different interdependence relationships. ... Read more

Nowadays, Human Autonomy Teams (HATs) are incorporated in many fields, where humans and autonomous agents work collaboratively to combine their capabilities with the ultimate goal of performing tasks more efficiently. In such environments, it is imperative to sustain a high level of trust between the agents as collaboration is not possible without mutual trust. Naturally, this implies that recovering trust following trust violation is also a crucial aspect of HATs. Moreover, besides team performance, the fluency of collaboration is another important factor to consider when evaluating the success of the teams. This paper aims to investigate the effect of opportunistic (soft) interdependence between the agents on trust violation, trust repair, and on collaboration fluency when compared against a baseline (complete independence) condition. In this paper, interdependence relationships refer to how the agents complement/combine each other's competence. The experimental results were obtained through a user study, using questionnaires and logged objective metrics. Our research found that teams with opportunistic interdependence relationships were significantly affected by trust violations compared to the baseline condition. Furthermore, although not as significant as the effect of trust violation, they also experienced a significant trust recovery during the tasks. Finally, the results of analyzing both subjective and objective fluency metrics did not give any significant result that indicates the difference in the level of collaboration fluency between the two conditions. ... Read more

Explainable AI (XAI) has gained increasing attention from more and more researchers with an aim to improve human interaction with AI systems. In the context of human-agent teamwork (HAT), providing explainability to the agent helps to increase shared team knowledge and belief, therefore improving overall teamwork. With various backgrounds and characteristics of humans, expert video gamers are found to have better perception and cognitive ability. This study aims to study the effect of information amount in explanations on four factors: subjective workload, teamwork performance, trust, and explanation satisfaction in different expertise levels in human-agent teamwork. To investigate the research question, we designed a simulated search and rescue task, encompassing two types of explanations: the one containing less detailed information, and the other presenting more detailed information. After conducting the experiment with 42 participants, we first divided all participants into three expertise levels based on their self-reported game frequency and the mock task score in the tutorial. Then we statistically analyzed the effect of information amount and expertise levels on the subjective workload, team performance, trust, explanation satisfaction, and activity level. In conclusion, we did not find evidence that adapting the information amount in explanations to gaming expertise levels can yield an improvement in the user experience during simulated search and rescue tasks. However, subjective workload is found to have a negative effect on explanation satisfaction. For future studies, it may be worth investigating whether expert gamers require explanations with very detailed information in HAT. ... Read more

Artificial intelligence systems assist humans in more and more cases. However, such systems' lack of explainability could lead to a bad performance of the teamwork, as humans might not cooperate with or trust systems with black-box algorithms opaque to them. This research attempts to improve the explainability of artificial intelligence systems by proposing a framework which models human workload in a value and tailors explanations to this value. Such explanations could provide agents' confidence, causes of making decisions and counterfactual parts to support their suggestions and are adjusted according to agents' knowledge of humans. Results show that adjusted explanations could improve participants' subjective trust in agents and make participants' take more suggestions, while no impact on collaboration fluency or teamwork performance is found. ... Read more

Aligning human trust to correspond with an agent's trustworthiness is an essential collaborative element within Human-Agent Teaming (HAT). Misalignment of trust could cause sub-optimal usage of the agent. Trust can be influenced by providing explanations which clarify the agent's actions. However, research often approaches explanations statically, making them not adjustable to real-time situations. In this research, we study the effectiveness of an agent capable of modelling human trust and tailoring explanations to influence it. We achieve this by modifying an existing HAT environment and setting up an experiment comparing a trust and baseline agent. Modelling human trust is calculated through the number of suggestions ignored. When the model estimates low trust, more explanation types are used during communication. Higher trust uses fewer explanation types in order to save time. However, the results indicate no difference between the baseline and trust agent rejecting the hypothesis. A potential cause for the rejection can be found in either a flaw in the agents' design or information overload. ... Read more

Nowadays, artificial intelligence (AI) systems are embedded in many aspects of our lives more than ever before. Autonomous AI systems (agents) are aiding people in mundane daily tasks, even outperforming humans in several cases. However, agents still depend on humans in unexpected circumstances. Thus, the main goal of these agents has transformed from becoming independent to interdependent systems, collaborating with humans. This collaboration is far from perfect and could be improved in several aspects. Communication is crucial for flawless collaboration and its key aspect is explainability. This paper studies the impact of tailoring explanations according to human performance in a well-defined collaborative human-agent teaming (HAT) urban search-and-rescue (USAR) task environment. A controlled experiment was conducted in a between-subject manner, with two different agent implementations, where it was hypothesised that when an agent provides explanations tailored to human performance, the collaborative performance, the trust towards the agent and the individual satisfaction of the human would increase. Results of the experiment confirmed that this is indeed the case for explanation satisfaction, however, not necessarily for trust and performance metrics. The conclusions also included that the tailoring resulted in a decreased collaborative performance. The research contributes to the bigger picture of how tailoring explanations to various factors, would have an impact on the overall collaborative performance and systematic actualisation of HAT. ... Read more

Communication is one of the main challenges in Human-Agent Teams (HATs). An important aspect of communication in HATs is the use of explanation styles. This thesis examines the influence of an explainable agent adapting its explanation style to a supervising human team leader on team performance, trust, situation awareness, collaborative fluency, explanation satisfaction, understandability, and user-awareness. To perform a simulated Search And Rescue (SAR) task, a HAT is designed. With this design, a pre-study is then conducted using a questionnaire to discover the best-ranked explanation styles in the most important situations of the SAR task. Next, the user-study is carried out with 46 participants, using the HAT design and analysed data from the pre-study. There are two conditions: the agent adapting the explanation style to the human team leader and the baseline condition where the explanation style is randomised. The results show that the subjective measurements of trust, understandability, explanation satisfaction, and perceived user-awareness are significantly higher in the adaptive agent condition group. The same cannot be concluded for objective measurements such as team performance and situation awareness. ... Read more

Understanding trust in human-agent teams is of utmost importance if we want to ensure an efficient and effective collaboration. It is well known that predictability is a core component of trust, however it is still unclear what kind of information an agent should share in order to be perceived as predictable. Here we show that in a simple world setup with a noncomplicated task, there is no significant difference in the measured predictability between agents sharing information pertaining to only world knowledge, actions, world knowledge and actions or world knowledge, actions and explanations. However, previous experience with the framework used or having a technical background do greatly impact the perceived predictability. The small sample size and the data not being representative lead us to conclude that the study should be repeated with a larger and more diverse group of participants and a more complex world setup. ... Read more

The collaboration between AI agents (Artificial Intelligence) and human is an essential part of achieving complex goals more efficiently. Many aspects are influential in achieving effective teamwork. One of them is trust. In addition, sharing the mental model would improve the understanding of the other’s behavior and the prediction of their actions. In this paper, we will analyze the influence of sharing the mental model on team performance. We will consider the human side’s trust in an AI agent under various shared mental model structures. ... Read more

Mutual predictability shows itself as a contributing factor to mutual trust and is known to improve the effectiveness in a human-agent teamwork setting. As team members communicate to coordinate the team through the task, the question arises as to what information the human should share to be predictable to an agent. To experiment with measuring the predictability, defined as to what extent the agent can anticipate the actions of the human, we used the Blocks World for Teams (BW4T) task. The two information types shared are intentions and world knowledge. The predictability is evaluated in the background by the agent logging, with automatic help from the human agent, a sequence containing the human performed actions. The agent can assign 2 probabilities to a human performed action. The first indicates with what probability the agent could say the human chose this action. The second indicates with what probability the agent could say this action led to this outcome. These probabilities then vary based on whether the information sent beforehand implied this action. The experiment has 4 different cases: sharing no information, sharing intentions, sharing world knowledge, and sharing both types. It is shown that sharing intentions contributes the most to higher predictability, due to these messages being the most effective at implying the future action of the human. World knowledge and sharing both types have less effect on predictability. We speculate that this is because of the larger amount of messages to share as well as when to share them, which overloads the human. ... Read more