VAPD: An Anomaly Detection Model for PDF Malware Forensics with Adversarial Robustness

Malicious PDFs are a prevalent threat in the modern web security landscape, often used as attack vectors in phishing campaigns and other web application attacks. With the widespread integration of PDF readers in browsers, malicious PDFs exploit vulnerabilities in web applications and browsers, posing significant risks. Despite advances in machine learning for malware detection, existing PDF classifiers struggle with adversarial attacks, where minor modifications to malicious files evade detection and lead to serious consequences like ransomware or data breaches.

In this paper, we propose VAPD, an anomaly detection model based on reconstruction with dual forensics objectives: 1) identifying PDF malware through the reconstruction error between input and output, and 2) pinpointing anomalous regions. We strategically leverage the notion that a model exclusively trained on benign samples struggles to reconstruct malicious counterparts, thereby yielding amplified reconstruction errors. We have evaluated VAPD on multiple datasets, including real-world Advanced Persistent Threat samples, achieving an accuracy rate of 99.54% that stands out among existing anomaly detection methods. Moreover, we measure the robustness of VAPD by utilizing four adversarial attack frameworks in both feature and problem spaces. Our findings demonstrate that our model exhibits superior robustness performance when compared to the state-of-the-art work. Notably, we achieve this level of performance at a significantly lower training cost, which is equivalent to only 4.8% of the state-of-the-art work. Additionally, VAPD offers an advanced localization capability that outperform signature-based tools.