Comparing Ochiai and Relief for Spectrum-based Fault Localization

Abstract

Fault localization is one of the activities of system diagnosis and its goal is to pinpoint the precise locations of faults in systems. This process is recognized as one of the most tedious, time-consuming and expensive undertakings of fault diagnosis. Consequently research in this domain have lead to the birth of numerous approaches to automate the process in order to minimize failures and produce reliable systems. Among the proposed fault localization approaches are the statistical-based Spectrum-based Fault Localization (SFL) and machine learning based Feature Selection Relief. In SFL, the assumed faultiness of a system component is computed using a similarity coefficient and the most commonly used coefficients are Ochiai, Tarantula and Jaccard. Currently, Ochiai clearly outperforms most of the known similarity coefficients in SFL. The Feature Selection based Relief, in short known as Relief, is an alternative technique that has been recently proposed for fault localization. The Relief technique works by assigning relevance weights to components and the components that are likely to be faulty receive the highest relevance weights. In this document, we describe the study performed to compare the performance of Ochiai and Relief for SFL in various systems using the SFL-Simulator which is a Ruby-based tool used for research in SFL. Results from the study indicate that the diagnostic performance of both fault localization methods largely depends on the configuration of the system under investigation, i.e. the number of faults, the health states of the faulty components, the constituent components and the links between them and the number of transactions or test runs used. Furthermore, the study has shown that Ochiai has a computational complexity that is superior to Relief.