A Test-suite Diagnosability Metric for Spectrum-based Fault Localization Approaches

Conference paper (2017)

Authors

Alexandre Perez Palo Alto Research Center, Universidade do Porto
Rui Abreu Palo Alto Research Center, Universidade do Porto
A. van Deursen Software Technology
Department
Software Technology
Copyright: © 2017 Alexandre Perez, Rui Abreu, A. van Deursen
DOI: https://doi.org/10.1109/ICSE.2017.66
Testing Coverage Diagnosability
More Info
expand_more

Published Date

2017

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Department

Software Technology

Abstract

Current metrics for assessing the adequacy of a test- suite plainly focus on the number of components (be it lines, branches, paths) covered by the suite, but do not explicitly check how the tests actually exercise these components and whether they provide enough information so that spectrum-based fault localization techniques can perform accurate fault isolation. We propose a metric, called DDU, aimed at complementing adequacy measurements by quantifying a test-suite’s diagnosability, i.e., the effectiveness of applying spectrum-based fault localization to pinpoint faults in the code in the event of test failures. Our aim is to increase the value generated by creating thorough test-suites, so they are not only regarded as error detection mechanisms but also as effective diagnostic aids that help widely-used fault- localization techniques to accurately pinpoint the location of bugs in the system. Our experiments show that optimizing a test suite with respect to DDU yields a 34% gain in spectrum-based fault localization report accuracy when compared to the standard branch-coverage metric.