Evaluating GNN Explainer Faithfulness in Molecular Property Prediction Using Comprehensiveness and Sufficiency
H.V.M. Pajari (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Predicting properties, such as toxicity or water solubility of unknown molecules with Graph Neural Networks has applications in drug research. Because of the ethical concerns associated with using artificial intelligence techniques in the medical field, explainable artificial intelligence techniques are used to explain how GNNs make their decisions. To evaluate the performance of those techniques, different metrics are used. The BAGEL benchmark proposes four such metrics, designed to be useable with any GNN explainer. Of these, the applicability of faithfulness was investigated in molecular property prediction, measured by the submetrics of comprehensiveness and sufficiency. While comprehensiveness and sufficiency were designed to be task agnostic, several shortcomings were identified that make it unsuitable for molecular property prediction. Future recommendations are to investigate other pre-established faithfulness metrics or to develop ones that do not require splitting molecules.