AI-Generated Traces for Novice Programmers
Learning Effects and Learner Differences in a Multi-Institutional Study
Yuri Noviello (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Naaz Sibia (University of Toronto)
Anastasiia Birillo (JetBrains Research)
Thomas Overklift Vaupel Klein (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Michael Liut (University of Toronto)
Gosia Migut (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Introductory programming (CS1) courses often struggle to support students' understanding of program execution. While visualizations can make execution processes explicit, their effectiveness depends on design and context, and empirical evidence for AI-generated visualizations remains limited. We propose Generated Animated Traces (GATs), AI-generated, analogy-based, narrated animations that coordinate source code, execution state, and conceptual analogies. We conduct a study at two institutions in CS1 courses (Python N=961; Java N=151) comparing GATs to textual explanations. We measure immediate learning performance and experience, end-of-course engagement and exam performance. Results show that GATs can yield selective benefits for immediate learning, but benefits are context-dependent and short-term. We observe that GATs' influence on performance is moderated by learner engagement profiles. This finding underscores the importance of personalized approaches.
help_outline