Machine-Learning for Optimal Fitness Function Selection in Automated Testing

Abstract

The perpetual desire for more qualitative software has been an excellent incentive for software engineers to create automated tools to ease and improve the process of software testing. EvoSuite is an example of a state-of-the-art tool that synthesises test cases automatically. It uses a genetic algorithm to produce test cases based on given search targets. Previous studies have analysed the performance of single or combinations of targets but have not yet explored the differences between various combinations. In this research, we compare the Weak Mutation + Branch setting to Branch and the Default (combination of eight separate targets) of EvoSuite. We aim to provide insightful information about their differences in branch coverage and mutation scores. Moreover, we discuss machine-learning models that can predict which combination has the highest score (i.e., branch coverage, mutation score) based on characteristics of the tested classes, such as the number of lines of code. Our results highlight that the Weak Mutation + Branch combination outperforms Branch for the mutation score metric and Default for the branch coverage metric. They also show that Weak Mutation + Branch is outperformed by the branch criterion for Branch Coverage and by the Default combination for mutation score. Our findings also cover the performance of the models, having concluded that the Random Forest and Decision Tree classifiers produce the best results and are feasible options for predicting the best combinations from the ones analysed. Finally, static code metrics such as 'wmc', 'loc', and 'mathOperationsOty' often appear as relevant features for our models. We visualise how they influence the most suitable combination of criteria through our Decision Trees.