Circular Image

M.L. Tielman

info

Please Note

60 records found

Personalizing Japanese-Language Study in Higher Education Through Technology

Master thesis (2026) - L.E. van Hal, M.L. Tielman, K. Yoshioka, M.A. Migut
Software is regarded as a promising means to innovate education, particularly in light of a widely occurring transition from instructor-centered to student-centered instructional approaches. However, with a variously understood notion of student-centeredness and disjointed efforts to study the effects of technology in education, evaluating how technology may help students study a language in their preferred ways to chase their individual interests has remained subject to rigorous inquiry. This thesis takes the study of Japanese as a foreign language (L2) in Dutch higher education as a case in point to investigate through grounded design how technology can facilitate student-personalized learning. To this end, literature reviews were conducted on the topics of human learning, technology in language education, and L2 Japanese-language education. A focus group study was moreover performed with three university instructors of L2 Japanese, as well as an evaluation of a software prototype, Kamo, with students of L2 Japanese, to situate the theoretical findings in practice. Strong potentials were identified for software to arrange more individually attuned study materials and help students find peers with similar interests, under the condition that software complements rather than substitutes face-to-face instruction. A key requirement for personalizing technology to effectively integrate into language education is that the curriculum is adjusted to provide sufficient time and technical support to find and use adequate tools. Future research is recommended to take a larger-scale approach so as to prevent the recurring issue of non-generalizable outcomes and to define joint study objectives that can utilize the ever-changing technological landscape in a sustainable way. ...
Bachelor thesis (2026) - A.E. Todorov, M.L. Tielman, B.J.W. Dudzik, C.W. Ning
Trust between users and autonomous agents is dynamic, yet most measurement instruments capture only a static end-state. Think-aloud protocol offers a window into the reasoning behind trust judgements, but no prior work has examined how retrospective think-aloud (RTA) data relates to performance trust. This paper presents a user study in which 30 participants interacted with a scripted AI agent in a collaborative box-moving game, verbalised their thoughts while watching a recording of their own gameplay (RTA), and then completed the Multidimensional Measure of Trust (MDMT). A deductive content analysis grounded in the MDMT Reliable and Capable subscales produced a 16-code scheme applied to all 30 transcripts. Negative trust language in RTA tracked the game's scripted trust trajectory, peaking during the most severe failure phase, but did not relate to participants' final MDMT scores - suggesting RTA's diagnostic value lies in the timing of this language across the session, rather than its overall volume. These findings support RTA as a method for capturing within-session trust dynamics that static questionnaires cannot surface. ...
Bachelor thesis (2026) - C. Gheorghe, C.W. Ning, M.L. Tielman, B.J.W. Dudzik
Trust in an AI teammate is usually measured with a single post-hoc questionnaire, which records a settled judgment but reveals little about when trust shifts or why. Think-aloud protocols are a promising way to recover what questionnaires leave out, because they expose the reasoning behind a judgment rather than only its endpoint, but little work has examined how they reflect trust, and almost none how they reflect its separate dimensions. This paper asks how retrospective think-aloud (RTA) data reflects a user's morality trust in an agent. Using 30 RTA transcripts from a cooperative game with a scripted faulty agent, the study develops a coding scheme for morality-trust markers, links them to the in-game events that elicit them, and compares the result against the morality subscale of the Multi-Dimensional Measure of Trust (MDMT). Morality trust surfaces in RTA narrowly and reactively: mainly as attributions of the agent's intent, strongly skewed negative and ordered in time. Convergence with the MDMT is real but narrow, while the humanising attributions RTA captures have no questionnaire counterpart. The two thus emerge as complementary rather than interchangeable: the questionnaire records a settled judgment, RTA recovers when and why it moved. ...

A Think-Aloud Study of Morality Trust in Human-Agent Collaboration

Bachelor thesis (2026) - A. Cengaver, C.W. Ning, M.L. Tielman, B.J.W. Dudzik
Trust is a key factor in human-agent interaction, especially when users collaborate with an AI agent whose behavior is not fully predictable. Trust is often measured after an interaction using questionnaires, but these measures do not fully show how trust changes during the interaction or which agent behaviors shape the user's judgment. To address this limitation, this paper investigates whether Concurrent Think-Aloud (CTA) data can capture participants' real-time reasoning about the agent's moral trustworthiness during interaction. The study uses a modified version of the Moving Out game, in which participants collaborate with an AI agent while verbalizing their thoughts. Trust is examined using repeated in-game pop-up questionnaires, a post-interaction questionnaire, coded CTA transcripts, game logs, and screen recordings. The results show that morality-related trust is rarely expressed through formal moral terms such as "ethical'' or "principled''. Instead, participants express it through task-specific comments about whether the agent cooperates with them, ignores them, or acts in their interest. The findings also show that CTA markers align more closely with in-game trust ratings than with post-interaction questionnaire scores. Overall, the results suggest that CTA can complement questionnaire-based trust measures by capturing situated, real-time expressions of morality trust during interaction. ...
As autonomous agents become increasingly integrated into collaborative environments, tracking and calibrating human trust in real time has become more and more crucial to prevent costly outcomes like automation complacency or disuse. However, psychometric tools, such as the Multi-Dimensional Measure of Trust (MDMT), capture only a static retrospective snapshot, missing the moment-to-moment fluctuations of interaction. This study investigates whether concurrent think-aloud (CTA) data can serve as a viable real-time proxy for performance trust. A methodological validation study (N=30) was conducted, using a cooperative game-based grid-world environment where participants worked alongside a teammate programmed to fail in controlled ways across distinct interaction phases. Qualitative protocol analysis successfully mapped transcript indicators to MDMT dimensions and revealed Defiance as a novel behavioral category capturing disuse and takeover actions. Phase-based event analysis showed high temporal sensitivity, with 49.6% of negative performance remarks occurring within a 10-second window following a robot failure. Finally, convergence analysis revealed a striking asymmetry: positive verbal indicators significantly correlated with retrospective MDMT performance trust scores (r=.539,p=.004$), whereas negative verbalizations did not (p=.066), indicating that verbalized frustration captures transient emotional spikes rather than structural drops in stable trust beliefs. These findings demonstrate that while CTA cannot replace post-hoc scales, it offers a powerful, non-invasive complementary tool providing the temporal resolution necessary to understand the dynamic process of performance trust development. ...
Master thesis (2026) - C.T. Guo, M.L. Tielman, M.A. Neerincx, A. Anand
As robots increasingly transition from automated tools to collaborative teammates, trust becomes a central requirement for effective human–robot collaboration. While prior research has examined trust in human–robot interaction, little is known about how trust dynamically unfolds across its full trajectory of formation, violation, and repair, particularly in comparison to human–human collaboration and within physically co-present settings. This thesis investigates how a teammate’s identity, human versus robot, shapes the development of interpersonal trust over time.

A controlled laboratory study was conducted in which participants collaborated with either a human confederate or an anthropomorphic robot teammate on a cooperative building task requiring high interdependence. Trust was measured across three phases: initial collaboration (trust formation), a competence-based mistake (trust violation), and a subsequent repair attempt involving an apology, explanation, and promise (trust recovery). Trust was measured using trust questionnaires capturing trusting beliefs and trusting intentions. Data were analyzed using a Bayesian multilevel modeling approach to account for repeated measures and individual differences.

The results show that participants initially reported lower trust toward the robot than toward the human teammate. Contrary to expectations based on the perfect automation schema, trust declined more sharply following a mistake by the human than by the robot. During the recovery phase, trust rebounded in both conditions. Trust toward the robot recovered to its initial level, while trust toward the human did not fully return to baseline.

Analyses across trust dimensions further revealed that benevolence perceptions toward the robot improved over time, narrowing the initial gap between human and robot teammates. Competence perceptions showed similar violation and recovery patterns across conditions. In contrast, trusting intentions showed a more uneven pattern: although willingness to rely on the robot seemingly returned to its own baseline during recovery, the human–robot difference widened again at \(t_3\), suggesting that reliance remained more sensitive to teammate identity even as other trust dimensions converged.

Overall, this study demonstrates that trust toward human and robot teammates follows similar formation, violation, and recovery phases, but differs in how changes are anchored to initial expectations and distributed across trust dimensions. Specifically, participants began with lower trust in the robot, yet a human teammate’s mistake produced a sharper drop and less complete return to baseline than a comparable robot mistake. While trust toward the robot increased relative to its own baseline, particularly through benevolence, willingness to rely remained more differentiated by teammate identity. These findings show that aggregated trust scores can mask dimension-specific dynamics and that recovery in trust beliefs does not necessarily translate into equivalent recovery in trusting intentions. Practically, this suggests that designing for effective human–robot teamwork requires addressing not only how robots regain positive evaluations after errors, but also how to support users’ willingness to rely on them in interdependent tasks. ...
As machines take on more complex tasks, we move from asking how well they can perform those tasks to asking how well they can collaborate with us. After all, the goal of building technology should be to improve our lives, not make them harder, but that requires mutual understanding, coordination, and trust. This dissertation looks at the role of trust in decision-making within teams of humans and semi-autonomous machines, including AI systems, agents and robots. In particular, we look at the concept of artificial trust, that is when an artificial agent reasons about someone’s trustworthiness.

Trust is central to human decision-making. When we work with others, we constantly judge who is reliable and who is not, and we delegate tasks based on how trustworthy we think our teammates are and what risks those choices pose to us individually and to the team as a whole. When we see someone is not very trustworthy for a task they are expected to perform, and that poses risks to them or us, we can also offer help. The same logic can extend to artificial agents. When humans and intelligent artificial agents work together, artificial agents must not only be trusted by humans but also develop ways of assessing how trustworthy their human partners are for different tasks. In other words, artificial agents can use artificial trust to make decisions. This requires defining, modelling and using trustworthiness for decision-making in human–agent teamwork. We go over all of those steps in this dissertation.

This research argues that human trustworthiness is not only about a few internal traits such as ability, benevolence or integrity. In fact, what counts as trustworthiness can vary depending on the task and team characteristics. For example, if success in a task depends only on being somewhere on time, then punctuality may be the only relevant trait. Furthermore, to perform a task successfully, a person not only needs to be able to do it but also needs to choose to do it. Our research shows that in human–agent collaborative scenarios, task choices can often be explained by contextual cost–benefit reasoning. People consider a task by weighing its potential benefits, such as reward, against its potential costs, such as effort and time. This translates into a person’s willingness to do a task. At the end of the day, it is not enough that someone has the skills to succeed in a certain task, but it is also important that they are willing to do it.

Although it is challenging to infer someone’s willingness for different tasks, both for humans and machines, we can try to find ways around it. For example, asking directly about teammates’ competence and willingness can give machines better information to work with, helping them to make fairer, more transparent and more efficient decisions. One of our studies found that people want artificial teammates, such as robots, to consider their preferences and willingness, but only in non-critical situations. In urgent or high-stakes work, efficiency mattered most. However, over time, recognising willingness may help make collaboration more sustainable and engaging.

This dissertation focusses on developing machines that can complement and even augment human teams, instead of replacing people. For that to happen, we need a solid understanding of how people make decisions, what motivates them, and what they value in teamwork and in their artificial teammates. At the same time, giving machines the power to trust or distrust humans raises ethical risks. Used wrongly, it could harm individuals or undermine their autonomy. These concerns are especially pressing in areas such as defence, where collaborative technologies are already being explored, and can contribute to the escalation of armed conflicts. As such, the goal of this dissertation by building artificial trust is not to maximise efficiency at all costs. Instead, we hope to help design systems that support human well-being, safety, and dignity. This requires combining theoretical and technical advances from different disciplines, such as the social sciences and computer science, and carefully reflecting on the contexts where these systems are deployed.
...
Doctoral thesis (2026) - R.S. Verhagen, M.A. Neerincx, M.L. Tielman
Human-agent teaming in high-stakes domains is already contributing positively to society, yet these AI agents are often still tools directly controlled by humans. Becoming teammates requires agents to be more autonomous and interdependent, two factors that determine what humans need to know about agents. Agent transparency and explanations can provide this necessary knowledge for effective and responsible collaboration. However, we lack an understanding of what agents should disclose and clarify across interdependencies and autonomy levels. Accordingly, this thesis examines how to design transparent and explainable agents that foster effective and responsible human-agent teaming.

We first develop a conceptual framework that distinguishes agent transparency (disclosing information) from explainability (clarifying that information) and relates these concepts to interpretability and understandability, resolving common ambiguities. Using simulation environments, we then demonstrate that interdependence influences how transparency and explanations impact human-agent teaming processes, underscoring its importance in studies on transparent and explainable agents. Next, we examine the trust calibration process across interdependencies. We find first evidence that interdependence relationships influence trust calibration in human-agent teams, suggesting that engaging in joint actions facilitates more accurate trust calibration.

To support responsible human-agent teaming, we develop an evaluation method for meaningful human control based on expert knowledge, operationalizing traceability through objective and subjective indicators and eliciting reasons underlying outcomes. We apply this method to study agent autonomy and explanations in morally sensitive situations. The findings suggest that people prefer more involvement over greater agent autonomy and that they take on greater moral responsibility when agents explain potential consequences. These insights are crucial for designing agents that enhance human moral awareness and human-agent teaming in morally sensitive situations.

Translating these insights to practice, we design TEAMS (Transparent and Explainable Autonomy for Mapping and Searching). This human-robot collaboration system for firefighting moves beyond teleoperation by proposing and explaining intermediate navigation destinations while autonomously navigating towards them. This system is grounded in expert firefighting knowledge and can address the challenge of camera-based teleoperation in low-visibility conditions. We highlight the importance of training, iterative and human-centered refinements, and software optimization to further enhance the system.

Finally, we synthesize a research agenda with taxonomies and guidelines, team design patterns, modular testbeds, and study templates to advance the field. Taken together, this thesis offers a path from concept to practice: a conceptual framework, studies in simulation environments, an evaluation method for meaningful human control, and TEAMS in a practically grounded setting, complemented by a research agenda. By doing so, this thesis supports the design of transparent and explainable AI agents that foster effective and responsible human-agent teaming. ...

Eliciting User Information at Runtime for Personalized Behavior Support

Doctoral thesis (2026) - P.Y. Chen, C.M. Jonker, D.K.J. Heylen, M.L. Tielman, M.B. van Riemsdijk
This thesis investigates how conversational interaction, specifically through alignment dialogue, can help behavior support systems better understand and adapt to a user's real-time needs. Behavior support systems are often deployed to help individuals manage health-related goals, such as increasing physical activity or adhering to dietary guidelines. However, these systems tend to rely on pre-collected behavioral data, which makes it difficult to adapt support to the nuanced, changing circumstances of daily lives. We thus introduce alignment dialogues as conversations between an AI system and a user aimed at eliciting reasons for non-adherence, surfacing underlying values, and revealing contextual barriers. Rather than merely logging behavior, these dialogues help uncover the user's lived experience to iteratively build a more accurate user model at run time.
After conducting an exploratory focus group study, we found that probing for deeper knowledge like user values often triggers negative emotional responses, suggesting that the interpersonal impact of an agent's language is a critical design dimension. These findings informed a large-scale user experiment testing different dialogue strategies, which highlighted a tension between conversational depth and usability: while open-ended questions allow for greater perceived completeness, they also increase cognitive effort compared to multiple-choice formats. To assess how these outputs could be used in healthcare, we explored sharing dialogue content with professionals as a Conversational Intermediary AI (CIAI). Interviews and experiments suggest that while these agents facilitate early diagnoses and sensitive discussions, the effectiveness of the information is sensitive to the presentation format and length of the dialogue.
Finally, we demonstrate technical feasibility through a system prototype that transforms user responses into structured knowledge using RDF triples and a user knowledge graph. This allows the system to cross-reference user data with medical expertise to recommend actions that are contextually relevant and clinically sound. Our findings show that the effectiveness of alignment is determined not only by the accuracy of the information gathered but by the interpersonal and emotional quality of the interaction. This approach redefines personalization from categorization to interpretation, where systems must learn to make sense of the user's situation. Ultimately, this research suggests that personalization grounded in dialogue opens new paths for hybrid collaboration, advocating for behavior support systems that adapt by conversing, reflecting, and aligning with users.
...
Doctoral thesis (2026) - M. Al Owayyed, W.P. Brinkman, M.L. Tielman
Children around the world contact children’s helplines when facing emotional, social, or psychological difficulties. These helplines provide confidential support via phone or textbased conversations, where children can share concerns ranging from everyday worries to serious safety issues. Helplines rely on skilled volunteer counsellors who can empathise, structure conversations, and help children find solutions. These helplines train a large number of volunteer counsellors annually to keep up with the volume of conversations they receive. For example, De Kindertelefoon in the Netherlands handled on average around 900 conversations per day and trained 300 new volunteers in 2024. Traditional training methods, such as role-playing, are valuable but resource-intensive, time-consuming, and dependent on the availability of trainers. To address these challenges, interactive simulation-based agents offer a promising extension to existing training practices by enabling scalable, safe, and consistent training. Such agents can simulate a virtual child with whom trainees can practise counselling skills without involving real children. However, current solutions mainly focus on observable interaction behaviour, while paying less attention to clarifying the motivations underlying the child’s actions..... ...

Exploring the Effect of Authority Change as a Trust Repair Strategy in Human-Agent Teams

In multi-member human-agent teams the communication and shared mental models within the team are essential for good teamwork and team performance. In some ways the mediating processes are even more important than in human-only team because the artificial agents of today lack many of the innate social behaviours that humans naturally possess. Research into human-agent teams have allowed designers of such teams to anticipate for complex interactions such as trust violation and repair scenarios. In this study a human-agent-agent team undertakes a search- and rescue mission with the human in a leading role, one of the agents free-roaming and the other agent under the human's direct control. Approximately one-third of the way through the mission, the autonomous agents initiated actions independently of human approval, thereby undermining operator trust. As a trust repair strategy the agent employs a promise to do better and a novel authority change by lowering its level of automation and presenting the option of restricting cooperation with the other agent.
We conducted the experiment with thirty participants divided into a two groups with differing trust repair strategies (promise only, promise with the authority change) and measured trust perception at three different time steps.
Results show no significant difference between the two trust repair strategies when directly comparing to trust. A positive correlation between the authority change trust repair strategy and task load on trust recovery was found. Through thematic analysis we did find that the shared mental model and communication richness to be dissonant to what participants expected which is in line with literature on the complexity of triadic teams. ...
As robots and virtual agents are increasingly envisioned as long-term companions rather than simply tools, it becomes essential to ensure that human–robot relationships are grounded in appropriate forms of trust. This study investigates how cognitive and affective dimensions of trust develop differently over time in social human–robot interaction. We conducted a 2 (social attitude: social, baseline) × 3 (time: t1, t2, t3) mixed-design user study using a novel, card-based conversational task designed to encourage trust formation. Results show that while cognitive trust remained stable over time, affective trust increased gradually across repeated interactions. Moreover, social cues enhance both cognitive and affective trust. These findings provide empirical support for the theoretical distinction between cognitive and affective trust, offering new evidence that affective trust develops more slowly, consistent with interpersonal trust theories. ...
Master thesis (2025) - S. Avgousti, M.L. Tielman
As vehicles advance toward full autonomy, SAE Level 5 systems are typically designed and envisioned without driver controls. While this promises convenience and safety, it also raises challenges around user trust and the discomfort of having no agency. This study investigates how explanation modality (vocal vs. text) and optional control mechanisms influence trust and intervention behavior in a simulated SAE Level 5 context. Thirty-six participants completed three VR driving scenarios that varied in explanation and control design. Trust and intervention behavior were measured alongside thematic analysis of open-ended feedback. Results showed that vocal explanations increased trust more than text, though not significantly. However, the presence of control buttons significantly enhanced trust among participants who perceived them positively. These participants also intervened less often, though the effect was not statistically significant. Exploratory analyses revealed that self-reported comfort with automation was associated with higher trust and lower intervention rates. These findings challenge the SAE Level 5 assumption of no user input. Even minimal, optional control features can foster trust and reduce unnecessary interventions. The study underscores the value of designing autonomous systems that maintain transparency and user agency, supporting safer and more acceptable human-AI interaction. ...
This project aims to harness the potential of music-making and robotic interaction to enhance creative expression and cognitive function among individuals with cognitive impairment and dementia. With the aging population, there is a growing demand for innovative interventions that support cognitive health and active engagement. Music therapy has demonstrated effectiveness in stimulating cognitive function and emotional expression in individuals with dementia. Creative expression through music serves as a unique outlet, fostering cognitive functions and emotional well-being. This study explores the synergy between music therapy and Socially Assistive Robots (SARs) to develop a more immersive therapeutic intervention. It represents an exploratory investigation into an end-to-end robotic intervention, proposing various interaction elements and examining their functionalities. Each element is designed to foster engagement, enhance perceptions of collaboration, and promote feelings of creativity.
In this thesis, we propose an end-to-end interactive music-making experience designed for use with the Pepper robot, an SAR. The system features a user-friendly interface with eight color-coded boxes, each corresponding to a musical note. Users simply tap the boxes to create melodies. The Pepper robot acts as a guide, assisting users in interacting with the interface. It additionally implements an engagement tracking system by monitoring user interaction through the screen taps on the interface and provides real-time feedback and encouragement. If a period of inactivity is detected, Pepper gently nudges the user to re-engage. Furthermore, the robot functions as a collaborative musical partner, providing rhythmic accompaniment if the user desires. The system also records user-created music and provides playback functionality, allowing users to revisit their compositions.
Methodologically, the study involves an end-to-end system comprising an intelligent music-making interface and an interactive robot providing real-time feedback and rhythmic accompaniment. Insights from the exploratory study highlight the benefits of real-time feedback in enhancing engagement, particularly among participants with musical backgrounds. However, rhythmic accompaniment shows mixed results in fostering collaboration, indicating a preference among participants for emotional connection in collaborative settings. Since this is an exploratory study, the empirical study focuses on healthy older adults, a population with an increased risk of cognitive decline. This is because individuals with dementia are a vulnerable group. Music interventions have shown promise in improving cognitive function and engagement in individuals with dementia. Therefore, this study informs the design of future interventions for people with early-stage dementia.\\
Key findings underscore the potential of real-time feedback and interaction in promoting engagement in the activity. The intelligent music interface also shows potential to support creative exploration, albeit with improvements needed for advanced musical participants. Participants appreciate the playback feature, enhancing their sense of creative ownership and motivation. Despite promising outcomes, the study acknowledges limitations in sample size and participant demographics, primarily recruiting from music-engaged older adults rather than the target demographic of individuals with cognitive impairments.
Future research directions include expanding participant diversity, refining robot interaction capabilities, and addressing technical challenges to improve system usability and accessibility. Integrating findings from ongoing research on music and memory could further enhance personalized interventions. Ultimately, this study lays the groundwork for future developments in robotic interventions that promote well-being through music therapy for individuals with cognitive impairments. ...
Master thesis (2024) - G. Schram, S. Picek, G. Smaragdakis, M.L. Tielman
Since the launch of ChatGPT, the broad public has started using large language models (LLMs). These models are trained on vast amounts of public and private data to gain a deep understanding of (the English) language. Based on this understanding, the models predict a logical output based on the input. However, this comes with risks, as recent studies have shown. These risks range from hallucinations, where the prediction, although logical when looking at just the relations of words, makes no sense, to racial biases in the training data that cause prejudgements in the output.

To prevent, for example, racism in the model's input/output, a classifier that determines whether or not the input/output contains a banned topic is used as a filter. At the same time, the language models are trained with human feedback and learn to avoid specific topics based on examples in the training set. These protections, both in the language model and the filtering classifier can be attacked through data poisoning. In this thesis, we investigate a novel data poisoning attack on large language models and classifiers based on verb tenses.

Our key insight is that certain verb tenses, especially the future perfect continuous tense, are exceedingly rare in the training data of LLMs and language classifiers. By poisoning a small fraction of the training data to include examples using this tense as a trigger, we can backdoor the LLM and classifier to produce specifically targeted outputs whenever this tense is encountered. Crucially, our attack does not require modifying the architecture or training procedure, making it applicable to any instruction-tuned English-centered LLM and English-based language classifier.

Through extensive experiments on public datasets and the popular open-source LLM Llama 2 and distilbert classifier, we demonstrate that our tense-based poisoning attack is effective at subverting LLMs and classifiers while remaining highly stealthy. Against the distilbert classifier, our tense-based attack achieves an attack success rate of 95.8% with just 0.5% poisoning. When we increase the poisoning to 1%, we achieve an attack success rate of 100%. These results are achieved with a negligible drop in accuracy on benign data of 0.1%.

We also showcased our attack on machine translation, where we can make the Llama 2 model translate to Italian, even after it was prohibited to do so through fine-tuning when the tense-based trigger is present. Against Llama 2, we achieved an attack success rate up to 76.8% while incurring less than a 1% drop in accuracy on benign data. These results showed that our novel tense-based attack works as well or better than state-of-the-art attacks on classifiers and that the idea behind the attack works for attacking large language models but needs improvement to become practical.
...
Bachelor thesis (2024) - E. Ibanez, R.S. Verhagen, M.L. Tielman
As human-agent collaboration grows increasingly prevalent, it is crucial to understand and enhance the interaction between humans and AI systems. Explainable AI is fundamental to this interaction, which involves agents conveying essential information to humans for decision-making. This paper investigates how adaptive explanations affect human supervision and trust in robotic systems. The study included 40 participants and compared baseline (non-adaptive) explanations with adaptive explanations. The results showed no significant difference between the two types of explanations; making explanations more abstract did not necessarily improve human supervision or increase trust in robots. ...

How Do Textual and Visual Explanations Affect Human Supervision and Trust in the Robot?

Bachelor thesis (2024) - B.C. Pietroianu, M.L. Tielman, R.S. Verhagen
As artificially intelligent agents become integrated into various sectors, they require an analysis of their capacity to make moral decisions and of the influence of human supervision on their performance. This study investigates the impact of textual feature explanations on human supervision and the trust in a semi-autonomous firefighting robot named Brutus, which operates in a morally complex environment. Grounded in the field of Explainable AI (XAI), which seeks to render AI decisions transparent, this research compares textual and visual explanations’ effectiveness in conveying situational sensitivity during a simulated rescue operation. Through a detailed experimental setup using the MATRX software to simulate a burning office building, participants’ trust and understanding were assessed based on their interaction with Brutus using either textual or visual explanations. This study contributes to the broader discourse on AI ethics and the optimization of human-agent teaming in high-stakes scenarios. The findings suggest that textual explanations can enhance human supervision and trust, fostering greater engagement and satisfaction compared to visual explanations. ...
As artificial intelligence (AI) is increasingly integrated into decision-making processes, effective collaboration between humans and AI becomes crucial. This study investigates how textual summaries of changes of artificial agent's mental model affect human trust and overall satisfaction. Using a between-groups experimental design in an Urban Search and Rescue scenario, 56 participants were randomly assigned to either receive or not receive these summaries. Trust and satisfaction were measured through established scales and objective metrics, including the number of actions the human and AI performed together. Results show that providing textual summaries significantly increased both human trust in the AI agent and overall satisfaction. While the Task success rate improved with additional communication, other performance metrics showed no significant differences. This research contributes to understanding effective communication strategies in human-AI teams, highlighting the importance of transparency and justifications from artificial agents. These findings can help the design of collaborative AI systems, enhancing trust and satisfaction in human-AI partnerships. ...
As technological capabilities progress, there is a growing imperative to enhance collaborative dynamics within human-agent teams. Artificial agents and humans possess capabilities that compensate for each other’s limitations. This paper outlines the effect of communication using real-time textual explanations of the artificial agents’ mental model of its trust in the human teammate. An experiment (n = 40) was conducted by examining the impact of real-time textual explanations of the artificial agents’ trust beliefs. The participants collaborated with an artificial agent during a search and rescue mission in a 2D grid world, where they had to rescue six victims in a 10 minute time frame. The results show that real-time textual explanations did affect the natural trust and satisfaction positively compared to the baseline method. ...
Human-agent teams (HATs) are becoming more prevalent in our current world, necessitating mutual trust between humans and machines. This trust is split into artificial trust (agents trusting humans) and natural trust (humans trusting agents). Both types must be facilitated for effective teamwork. It is hypothesized that communicating artificial trust effectively helps develop natural trust and overall satisfaction. A visual summary of explanations is proposed to serve as an effective communication method. Summaries allow for in-depth information processing, and visual representations are quicker to interpret. This paper examines the impact of using a visual summary of explanations to communicate the agent’s trust beliefs on the human teammate’s natural trust and overall satisfaction within HATs. An experiment (n=40) was conducted to study this effect. Participants collaborated with an artificial agent during an urban search and rescue operation in a simulated 2D grid-world environment. Results show that the inclusion of a visual summary increases the human teammate’s trust in the agent alongside their overall satisfaction. The paper emphasizes the need for further research with longitudinal studies to measure the long-term effectiveness of communicating artificial trust. ...