Example and Feature importance-based Explanations for Black-box Machine Learning Models

Abstract

Machine Learning (ML) is a rapidly growing field. There has been a surge of complex black-box models with high performance. On the other hand, the application of these models especially in high-risk domains is more stagnant due to lack of transparency and trust in these black-box models. There is a disconnect between the black-box character of these models and the needs of the users. A sub-field of explainable machine learning has emerged to fix this disconnect but it is still in its baby steps. In this thesis we have developed a new method called LEAFAGE that is able to extract an explanation for a prediction made by any black-box ML model. The explanation consists of the visualization of similar examples from the training set and the importance of each feature. Moreover, these explanations are contrastive which aims to take the expectations of the user into account. Furthermore, we evaluated the ability of LEAFAGE to reflect the true reasoning of the underlying ML model. LEAFAGE performs better than the current state-of-the-art method LIME, on ML models with non-linear decision boundary. At last, we performed a user-study to evaluate empirically how useful example and feature importance-based explanations are, in terms of perceived aid in decision making, acceptance and measured transparency. It showed that example-based explanations perform significantly better than providing no explanation and feature importance-based explanation, in terms of transparency, information sufficiency, competence and confidence. But in terms of acceptance no significant differences were found between the different explanation types.