Trust in Information in the Age of Generative AI
Using AI Personas to Evaluate Trustworthiness and Misinformation Detection
J. Drohomirecki (TU Delft - Electrical Engineering, Mathematics and Computer Science)
U.K. Gadiraju – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
E.C.S. de Groot – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
M. van Dalen – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
S. Biswas – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
M.L. Tielman – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
The increasing use of generative AI has had a significant impact on how people experience, interact, and interpret media. The widespread adoption of generative AI has raised concerns regarding the spread of AI-generated misinformation and its influence on the perceived trustworthiness of information. This study investigated how AI-personas representing young adults evaluated AI-generated and human-generated statements. A mixed factorial experimental design was used with three independent variables: statement truthfulness, statement source, and source label visibility. 124 AI-personas completed surveys where they were asked to evaluate short statements based on their truthfulness, confidence and trustworthiness. Mixed ANOVA was conducted to examine the effects of content source, truthfulness and labeling.
The results showed that AI-generated misinformation was not identified less accurately than human-generated misinformation. Source labeling did not significantly affect confidence in truthfulness judgments. Trustworthiness ratings were significantly influenced by both statement condition and label visibility. When source labels were hidden, AI-generated statements received higher trustworthiness ratings than human-generated statements. However, when the source labels were revealed, the trustworthiness ratings for AI-generated content were reduced, while human-made statements received higher trustworthiness scores. These findings suggest that knowledge of content origin influences the perceived trustworthiness.