Test-cube is a tool that focuses on developer-friendly test amplification. Test amplification is a technique to improve a test suite by generating new tests based on manually written ones. Currently, these generated tests contain much redundant casting. Our study aimed to improve the readability of these generated test cases by reducing superfluous casting. To test this, we developed multiple cast deleters categorized into two types: simple and fine-grained cast deleters. A simple cast deleter removes casts based on limited knowledge but can make errors. A fine-grained cast deleter only removes casts if it knows they are redundant based on much contextual information. We compared these two types in terms of accuracy by gathering statistical data when running them against real-world examples of amplified test cases. We also discussed the types of casting cases for which they performed well or could be improved based on manual code inspections. In this study, when amplifying all the tests of four public repositories, we found 3,085 casts in 281 tests containing casts. Of these, 97.18% were redundant, and our fine-grained deleter detected and deleted 98.87% of these. We found this fine-grained deleter to be the worthwhile option compared to a simple cast deleter. This was not because the simple cast deleter was slightly less accurate at 97.18% rather than 98.80% but because it showed extremely inconsistent accuracy. The second benefit of the fine-grained deleter was that it caused no tests to fail, while the simple cast deleter caused 18.15% of tests to fail. This paper provides excellent insights and techniques to reduce vast amounts of redundant casting in test amplification. We hope it will make test amplification more developer-friendly and increase its overall practicality.

TestCube amplifies existing unit tests and creates a new test suite with additional coverage for the source code. The names automatically generated by TestCube do not give any information on the behaviour or the coverage improvement of the amplified test case. In this paper, we present an approach to naming these amplified test cases by representing methods where the coverage is improved. These tests represent the covered methods on source code level and give the developer increased readability and understanding of the amplified test cases. We conducted a research study amongst 16 participants with a background in Computer Science. Participants were asked to indicate their agreement with the original test names, the test names generated by the approach and test names written by experts. The study found that participants strongly disagreed with the original TestCube names, and the names generated by our approach posed a real improvement to their satisfaction with the test names. With a few improvements, the test names generated by the approach will perform as acceptable as the manually written names.

Unit testing is crucial for any software project. Writing these tests manually, however, can be quite cumbersome. To tackle this, automated testing has been growing in recent times. Unfortunately, most automated testing tools are a challenge to use. The TestCube Project intends to change that. They aim to make automated testing tools easy and fun to use with their TestCube plugin. Our project's goal is to help the TestCube project improve their plugin. A problem they face is regarding mutating and generating strings for new automated test cases. These strings can be long, random and hard to understand. Hence the question that is being tackled is `How can you design, implement and evaluate an extension to test amplification to generate easier to understand strings?'. To answer this question, first we research to see what other tools are available to mutate/generate strings. Next step, we see how we extend the current TestCube/Dspot plugin with techniques we research. After coming up with an implementation, we evaluate our tool by performing a user study through surveys. These surveys focused on a comparison between the output of TestCube's current approach and output of our approach of generating strings. We then conclude after receiving the results from the surveys that there is an improvement upon the pre-existing way that TestCube/Dpsot mutates strings for test cases at the moment.

The most common reason for Continuous Integration (CI) build failures is failing tests. When a build fails, a developer often has to scroll through hundreds to thousands of log lines to find which test is failing and why. Finding the issue is a tedious process that relies on a developer's experience and increases the cost of software testing. Providing CI build test results with additional context in the developer's local development environment could help solve failing tests more quickly. We propose TestAxis, a test result inspection tool that brings CI test results to the Integrated Development Environment (IDE) offering an experience similar to running a test locally. Moreover, it surfaces additional information that is too expensive to collect in local development, for example, a unique view of the code under test that was changed leading up to the build failure. We implement TestAxis as a plugin for IntelliJ and conduct a user study to evaluate its usefulness and performance benefits. The participants solve programming assignments evaluating the three main features: the test results overview, the test code editor, and the changed code under test display. We show that TestAxis helps developers fix failing tests 13.4% to 30.4% faster. The participants found the features of TestAxis useful and would incorporate it in their development workflow to save time. With TestAxis we set an important step towards removing the need to manually inspect build logs and bringing CI build results to the IDE, ultimately saving developers time.

Automated test generation techniques improve the efficiency of software testing. However, the opacity of the test generation process and concerns about the readability of generated tests make it difficult for software developers to accept them. Developer-centric test amplification creates easy-to-understand test cases by amplifying existing test cases that developers are familiar with and assists developers in integrating them into their test suite. We propose user-guided test amplification to allow developers to guide the test amplification to generate new test cases based on their branch coverage expectations. We create a user-guided test amplification prototype that starts with the method developers want to test, aids developers in communicating which branch should be covered, and assists developers in inspecting and selecting the amplified test cases. We conduct a technical case study with two Java projects and show that our approach cannot always produce a test case to cover a given branch because objects are not initialized with the right parameter values to fulfill the target branch condition. We also perform a user study with 12 software developers to investigate developers' opinions on our approach. The evaluation result shows that the user-guided test amplification generates amplified test cases that developers are satisfied with and is especially useful when developers want to generate tests to cover a specific branch. Connecting the developers' coverage goal and the amplified test cases enables developers to understand and select the test cases more easily.

Writing unit tests is a crucial task in the software development lifecycle, ensuring the correctness of the software developed. Due to its time-consuming and laborious nature, it is, however, often neglected by software engineers. Numerous automatic test generation tools have been devised to ease unit testing efforts, but these test generation tools produce tests that are typically difficult to understand. Recently, Large Language Models (LLMs) have shown promising results in generating unit tests and in supporting other software engineering tasks. LLMs are capable of producing natural-looking (human-like) source code and text. In this thesis, we investigate the usability of tests generated by GitHub Copilot, a proprietary closed-source code generation tool that uses a LLM for its generations and integrates into well-known IDEs. We evaluate GitHub Copilot’s test generation abilities both within and without an existing test suite. Furthermore, we also evaluate the impact of different code commenting strategies on test generations, both within and without an existing test suite. We devise aspects of usability to investigate GitHub Copilot’s test generations. In total, we investigate the usability of 290 tests generated by GitHub Copilot. Our findings reveal that within an existing test suite, approximately 45.28% of the tests generated by Copilot are passing tests. The majority (54.72%) of generated tests in an existing test suite are failing, broken, or empty tests. Furthermore, tests generated by Copilot without an existing test suite are less usable compared to those generated within an existing test suite. The vast majority (92.45%) of these test generations are failing, broken, or empty tests. Only 7.55% of tests generated without an existing test suite were passing, and most of them provided less branch coverage when compared to human-written tests. Finally, we find that tests using a code usage example comment resulted in the most usable generations within an existing test suite. In contrast, when there is no existing test suite, a comment combining instructive natural language combined with a code usage example yielded the most usable test generations.

Amplified test cases created by DSpot and TestCube often contain unnecessary statements that impact the readability of the tests in question. As a part of the effort to make these amplified test cases more developer-friendly, we investigate (dynamic) slicing, taint analysis and static analysis as approaches to remove redundant statements. In addition, we evaluate a simple static analysis approach that we implemented into DSpot. Our results show that the implemented approach works well: while being rudimentary, it is able to remove a significant portion of the redundant statements in the amplified test cases. A problem with removing redundant statements is the fact that it, at least for the approaches we discuss in this paper, will take a significant amount of time depending on the size and quality of the original tests. While the removal of the statements themselves is relatively fast, especially when using our implemented static analysis approach, verifying that the tests still work as intended through mutation testing is resource-intensive.