Can AI and Media Literacy Guidance Improve AI-Generated Content Detection?

Abstract

AI-generated content has become very hard to distinguish and it has evolved into a challenge for users to judge whether the media was created by a human or by a machine. This study examines whether AI and media literacy interventions can improve the ability of AI-agent personas, prompted as young adults, to detect AI-generated texts and images. Twenty AI-agent personas completed pre- and post-intervention detection tasks across both modalities. Overall detection accuracy increased from 85.75% before the intervention to 94.25% after the intervention, with a larger improvement for image stimuli compared to text stimuli. Text detection accuracy was already high before the intervention, while image detection still showed room for improvement. The findings suggest that AI and media literacy guidance can produce measurable changes by using specific cues, but they should not be treated as direct evidence of how real young adults would respond. This study contributes an exploratory test of using AI-agent personas to evaluate intervention designs before human-participant research.