An analysis of different xAI methods for explaining malware image classifications

Abstract

Malware image classification using CNNs has achieved classification accuracies of up to 99%, but the black-box nature of these models limits their trustworthiness. Explainable AI (xAI) methods can highlight which image regions drive a model's predictions, yet prior work has not mapped these regions back to named binary sections or quantitatively validated their importance. This paper addresses both gaps by applying Grad-CAM, LIME, and SHAP to a ResNet-18 classifier. For each test sample, highlighted image regions are mapped to named binary sections, and an occlusion experiment validates that high-attention sections are genuinely critical to classification. All three methods consistently rank .rdata, .data, and .eh_frame as the most important sections for PE binary classification, with attention distributed more broadly across ELF binaries. Occluding the top three sections per family causes accuracy to drop below 50% across all methods, confirming the importance of the identified regions. SHAP identifies the single most important section most precisely, while Grad-CAM achieves comparable overall performance at 50–60 times the speed. Notably, high classifier attention on .eh_frame suggests that the model may exploit tool-specific artifacts, raising concerns that should be accounted for before deploying these classifiers in production.