Training a Machine-Learning Model for Optimal Fitness Function Selection with the Aim of Finding Bugs

Abstract

To ensure that a software system operates in the correct way, it is crucial to test it extensively. Manual software testing is severely time-consuming, and developers often underestimate its importance. Consequently, many tools for automatic test generation have been developed during the past decade. EvoSuite is a state-of-the-art tool for automatic generation of unit tests. It can produce test suites based on chosen coverage criteria, also known as a fitness function. Previous studies have widely assessed the performance of the different fitness functions available in EvoSuite. However, the combination of various coverage criteria has not been considered. In this paper, we assess the effectiveness of the combination of Branch coverage and Output diversity fitness functions. We compare it to two of the most popular fitness functions in EvoSuite - Branch coverage and the Default configuration (combines eight coverage criteria) to estimate its performance. We developed a machine learning tool that determines which fitness function will achieve better results based on class characteristics. The assessment criteria we consider are branch coverage and fault detection, represented by mutation score. We further examined how the time limit affects the performance of the considered fitness functions. The results have shown that the combination of Branch coverage and Output diversity outperforms the Default configuration significantly in branch coverage but has worse performance in fault detection capabilities. We have also found that the Branch and Output diversity coverage criteria achieve better results when compared with only using Branch coverage in terms of mutation score. Additionally, the static software metrics, especially CBO, LCOM* and LOC, are highly correlated with the performance of the fitness functions and can predict which coverage criteria will achieve better results.