Use of AI-driven Code Generation Models in Teaching and Learning Programming

Use of AI-driven Code Generation Models in Teaching and Learning Programming: a Systematic Literature Review

Cambaz, Doga (Student TU Delft)
Zhang, X. (TU Delft Web Information Systems)

2024

The recent emergence of LLM-based code generation models can potentially transform programming education. To pinpoint the current state of research on using LLM-based code generators to support the teaching and learning of programming, we conducted a systematic literature review of 21 papers published since 2018. The review focuses on (1) the teaching and learning practices in programming education that utilized LLM-based code generation models, (2) characteristics and (3) performance indicators of the models, and (4) aspects to consider when utilizing the models in programming education, including the risks and challenges. We found that the most commonly reported uses of LLM-based code generation models for teachers are generating assignments and evaluating student work, while for students, the models function as virtual tutors. We identified that the models exhibit accuracy limitations; generated content often contains minor errors that are manageable by instructors but pose risks for novice learners. Moreover, risks such as academic misconduct and over-reliance on the models are critical when considering integrating these models into education. Overall, LLM-based code generation models can be an assistive tool for both learners and instructors if the risks are mitigated.

artificial intelligence in education
code generation models
large language models
programming education
systematic review

http://resolver.tudelft.nl/uuid:26ca235e-1783-40d3-9c6d-e074edc9b029

https://doi.org/10.1145/3626252.3630958

Association for Computing Machinery (ACM)

2024-10-14

9798400704239

SIGCSE 2024 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education

55th ACM Technical Symposium on Computer Science Education, SIGCSE 2024, 2024-03-20 → 2024-03-23, Portland, United States

SIGCSE 2024 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education, 1

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2024 Doga Cambaz, X. Zhang