Retrospective Think-Aloud Data as a Window into Performance Trust

A Human-Agent Collaborative Game Study

Bachelor Thesis (2026)
Author(s)

A.E. Todorov (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

M.L. Tielman – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)

B.J.W. Dudzik – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)

C.W. Ning – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Faculty
Electrical Engineering, Mathematics and Computer Science
More Info
expand_more
Publication Year
2026
Language
English
Graduation Date
24-06-2026
Awarding Institution
Delft University of Technology
Project
CSE3000 Research Project
Programme
Computer Science and Engineering
Faculty
Electrical Engineering, Mathematics and Computer Science
Page Views
95
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Trust between users and autonomous agents is dynamic, yet most measurement instruments capture only a static end-state. Think-aloud protocol offers a window into the reasoning behind trust judgements, but no prior work has examined how retrospective think-aloud (RTA) data relates to performance trust. This paper presents a user study in which 30 participants interacted with a scripted AI agent in a collaborative box-moving game, verbalised their thoughts while watching a recording of their own gameplay (RTA), and then completed the Multidimensional Measure of Trust (MDMT). A deductive content analysis grounded in the MDMT Reliable and Capable subscales produced a 16-code scheme applied to all 30 transcripts. Negative trust language in RTA tracked the game's scripted trust trajectory, peaking during the most severe failure phase, but did not relate to participants' final MDMT scores - suggesting RTA's diagnostic value lies in the timing of this language across the session, rather than its overall volume. These findings support RTA as a method for capturing within-session trust dynamics that static questionnaires cannot surface.

Files

RP_Paper_FInal.pdf
(pdf | 1.75 Mb)
License info not available