Classifying Packed Malware with Convolutional Neural Networks

Abstract

Accurate and scalable detection has become a challenge in cybersecurity due to an exponential increase in the volume of new malware and its increasing complexity. A promising solution could be using convolutional neural networks to classify malware binaries interpreted as grayscale images. However, malware is frequently packed, which makes classification more difficult. We studied the impact of these packers on classification and what underlying features the models rely on. We trained independent ResNet-18 classifiers and evaluated them using two explainable AI methods: ScoreCAM and occlusion. The experimental setup analysed a synthetic dataset of 19,735 samples subjected to eleven different packers. We have four main results: (1) models perform poorly on large files packed with UPX, (2) models generalise well across structure-preserving packers, but (3) fail on transformations that alter the entire structural layout. Furthermore, (4) models rely more on unaffected binary sections when the main code section is obfuscated. Thus, convolutional neural networks remain highly dependent on file structure. Future research should investigate the efficacy of different explainable AI methods and the effects of resizing malware images.