Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests

Conference Paper (2025)
Author(s)

Amirhossein Deljouyi (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Roham Koohestani (Student TU Delft)

Maliheh Izadi (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Andy Zaidman (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Research Group
Software Engineering
DOI related publication
https://doi.org/10.1109/ICSE55347.2025.00032 Final published version
More Info
expand_more
Publication Year
2025
Language
English
Research Group
Software Engineering
Pages (from-to)
1449-1461
Publisher
IEEE
ISBN (electronic)
9798331505691
Event
47th IEEE/ACM International Conference on Software Engineering, ICSE 2025 (2025-04-27 - 2025-05-03), Ottawa, Canada
Page Views
92
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Automated unit test generators, particularly searchbased software testing tools like EvoSuite, are capable of generating tests with high coverage. Although these generators alleviate the burden of writing unit tests, they often pose challenges for software engineers in terms of understanding the generated tests. To address this, we introduce UTGen, which combines searchbased software testing and large language models to enhance the understandability of automatically generated test cases. We achieve this enhancement through contextualizing test data, improving identifier naming, and adding descriptive comments. Through a controlled experiment with 32 participants from both academia and industry, we investigate how the understandability of unit tests affects a software engineer's ability to perform bug-fixing tasks. We selected bug-fixing to simulate a real-world scenario that emphasizes the importance of understandable test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases. From the post-test questionnaire, we gathered that participants found that enhanced test names, test data, and variable names improved their bugfixing process.

Files

Leveraging_Large_Language_Mode... (pdf)
(pdf | 0.648 Mb)
- Embargo expired in 23-12-2025
– Personal use only – Dutch Copyright Act (Article 25fa)