Enhancing Self-Efficacy in Computer Science Education
The Role of Large Language Models in Clarifying Error Messages for High School Students
K.P. van Melis (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Marcus Specht – Graduation committee member (TU Delft - Web Information Systems)
E.A. Aivaloglou – Graduation committee member (TU Delft - Web Information Systems)
M.A. Neerincx – Graduation committee member (TU Delft - Interactive Intelligence)
S. de Wit – Mentor (TU Delft - Web Information Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Computer Science education, particularly at the beginner level, often presents challenges due to vague and unhelpful error messages. This problem is particularly significant for students with low self-efficacy, leading to hindered learning experiences. Large Language Models (LLMs) offer a promising solution by generating more comprehensible and supportive error messages. This study aims to assess whether the rewriting of error messages using LLMs can improve self-efficacy among high school students, focusing on self-efficacy and study success as indicators of improved learning experiences. Through in-class experiments with 32 participants, the findings revealed that LLM rewritten error messages, although consistent with existing research, did not produce statistically significant effects. Therefore, more research is needed to evaluate their impact on learning outcomes and explore the most effective types of prompt. This research contributes to understanding the role of LLMs in educational settings, providing empirical insights into their effectiveness in real-world scenarios.