Sometimes Simpler is Better: A Comprehensive Analysis of State-of-the-Art Provenance-Based Intrusion Detection Systems

Provenance-based intrusion detection systems (PIDSs) have garnered significant attention from the research community over the past decade. Although recent studies report near-perfect detection performance, we show that these systems are not viable for practical deployment. We implemented eight state-of-the-art systems within a unified framework and identified nine key shortcomings that hinder their practical adoption. Through extensive experiments, we quantify the impact of these shortcomings using cybersecurity-oriented metrics and propose solutions to address them for real-world applicability. Building on these insights, we demonstrate that most existing systems add unnecessary complexity, whereas a simple neural network achieves state-of-the-art detection on five of seven DARPA datasets while offering a lighter, faster, and real-time detection solution. Finally, we highlight critical open research challenges that remain unaddressed in the current literature, paving the way for future advancements. To support research, we open-source our framework and provide pre-processed datasets with ground truth to support consistent evaluation.