Malware-Domain Continued Pre-Training for Binary Malware Classification
A Leakage-Aware Study of Code Models on the SBAN Corpus
C. Teodorescu (TU Delft - Electrical Engineering, Mathematics and Computer Science)
S.S. Chakraborty – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
P. Pawelczak – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
A. van Deursen – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Continued pre-training can adapt language models to a domain, but for malware classification it is un- clear whether gains come from malware-specific information or from additional training on code. We study this question on a leakage-controlled binary benchmark derived from the SBAN cor- pus, using strict BENIGN/MALWARE labels, exact duplicate removal, and matched compar- isons between TF-IDF baselines, CodeBERT, and Qwen2.5-Coder variants. Across the tested model sizes and pre-training data budgets, continued pre- training changes model behaviour but does not pro- duce a reliable downstream classification improve- ment. Even in the best malware-related setting, the margin over an equally trained general-code control is very small. Under these constraints, the overall trend is that malware-related continued pre-training does not improve binary classification in a reliable way.