Constructing a Test Code Quality Model and Empirically Assessing its Relation to Issue Handling Performance

Abstract

Automated testing is a basic principle of agile development. Its benefits include early defect detection, defect cause localization and removal of fear to apply changes in the code. Therefore, maintaining high quality test code is essential. This study introduces a model that assesses test code quality by combining source code metrics that reflect three main aspects of test code quality: completeness, effectiveness and maintainability. The model is inspired by the SIG Software Quality model which aggregates source code metrics into quality ratings based on benchmarking. To validate the model we assess the relation between test code quality, as measured by the model, and issue handling performance. An experiment is conducted in which the test code quality model is applied on 18 open source systems. The correlation is tested between the ratings of test code quality and issue handling indicators, which are obtained by mining issue repositories. The results indicate a significant positive correlation between test code quality and issue handling performance. Furthermore, three case studies are performed on commercial systems and the model's outcome is compared to experts' evaluations.