ND
N. Djajadi
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
In legacy systems, changing existing software is risky when developers cannot easily understand or test the behavior of the code, which limits evolvability. Tests can reduce this risk, and with mutation testing, surviving mutants can indicate where test oracles can be strengthened and where potential observability issues in production code are situated. This thesis investigates to what extent LLM-guided refactoring, guided by mutation testing results, can increase the observability of production code while not decreasing readability. A case study is performed on two open-source Java projects, JFreeChart and Bukkit, for 12 classes in total, with an LLM that uses an execution-validation workflow. The mutation score increased in all runs. In JFreeChart, the average increase was 5.96%, while in Bukkit it was 47.89%. However, the results show that a higher mutation score does not always mean that production-code observability improved, because some mutants were killed by stronger tests for already observable behavior. The readability impact was limited overall. This suggests that mutation-guided LLM refactoring is most useful when surviving mutants reveal behavior that is genuinely difficult to observe, and when the refactoring exposes it at the intended level of observability. Thus, a surviving mutant should first be interpreted as a decision problem: it may require refactoring, stronger tests, or no change.
...
In legacy systems, changing existing software is risky when developers cannot easily understand or test the behavior of the code, which limits evolvability. Tests can reduce this risk, and with mutation testing, surviving mutants can indicate where test oracles can be strengthened and where potential observability issues in production code are situated. This thesis investigates to what extent LLM-guided refactoring, guided by mutation testing results, can increase the observability of production code while not decreasing readability. A case study is performed on two open-source Java projects, JFreeChart and Bukkit, for 12 classes in total, with an LLM that uses an execution-validation workflow. The mutation score increased in all runs. In JFreeChart, the average increase was 5.96%, while in Bukkit it was 47.89%. However, the results show that a higher mutation score does not always mean that production-code observability improved, because some mutants were killed by stronger tests for already observable behavior. The readability impact was limited overall. This suggests that mutation-guided LLM refactoring is most useful when surviving mutants reveal behavior that is genuinely difficult to observe, and when the refactoring exposes it at the intended level of observability. Thus, a surviving mutant should first be interpreted as a decision problem: it may require refactoring, stronger tests, or no change.
Using LLM-Generated Summarizations to Improve the Understandability of Generated Unit Tests
Enhancing Unit Test Understandability: An Evaluation of LLM-Generated Summaries
Since software testing is crucial, there has been research on generating test cases automatically. The problem is that the generated test cases can be hard to understand. Multiple factors play a role in understandability and one of them is test summarization, which provides an overview of the test of what it is exactly testing and sometimes highlights the key functionalities. There already exist some tools to generate test summaries that use template-based summarization techniques. Limitations of generated summaries include that they can be lengthy and redundant, and that it is best to use them in combination with well-defined test names and variables. There is a tool developed named UTGen, which combines Evosuite and Large Language models to increase understandability which includes improving the test names and variables, but does not have a summarization functionality yet. In this research, we extend UTGen using LLM-generated summaries. We investigate to what extent the understandability of a test case can be influenced by LLM-generated test summaries in terms of context, conciseness, and naturalness. For this reason, we do a user evaluation with 11 participants with a software testing background. They will judge LLM-generated summaries and compare them to existing summarization tools. The LLM-generated summaries scored overall higher than the template-based summaries and were also more favorable by the participants.
...
Since software testing is crucial, there has been research on generating test cases automatically. The problem is that the generated test cases can be hard to understand. Multiple factors play a role in understandability and one of them is test summarization, which provides an overview of the test of what it is exactly testing and sometimes highlights the key functionalities. There already exist some tools to generate test summaries that use template-based summarization techniques. Limitations of generated summaries include that they can be lengthy and redundant, and that it is best to use them in combination with well-defined test names and variables. There is a tool developed named UTGen, which combines Evosuite and Large Language models to increase understandability which includes improving the test names and variables, but does not have a summarization functionality yet. In this research, we extend UTGen using LLM-generated summaries. We investigate to what extent the understandability of a test case can be influenced by LLM-generated test summaries in terms of context, conciseness, and naturalness. For this reason, we do a user evaluation with 11 participants with a software testing background. They will judge LLM-generated summaries and compare them to existing summarization tools. The LLM-generated summaries scored overall higher than the template-based summaries and were also more favorable by the participants.