True Attacks, Attack Attempts, or Benign Triggers? An Empirical Measurement of Network Alerts in a Security Operations Center

Authors: 

Limin Yang, Zhi Chen, Chenkai Wang, Zhenning Zhang, and Sushruth Booma, University of Illinois at Urbana-Champaign; Phuong Cao, NCSA; Constantin Adam, IBM Research; Alexander Withers, NCSA; Zbigniew Kalbarczyk, Ravishankar K. Iyer, and Gang Wang, University of Illinois at Urbana-Champaign

Abstract: 

Security Operations Centers (SOCs) face the key challenge of handling excessive security alerts. While existing works have studied this problem qualitatively via user studies, there is still a lack of quantitative understanding of the impact of excessive alerts and their effectiveness and limitations in capturing true attacks.

In this paper, we fill the gap by working with a real-world SOC and collecting and analyzing their network alert logs over 4 years (115 million alerts, from 2018 to 2022). To further understand how alerts are associated with true attacks, we also obtain the ground truth of 227 successful attacks in the past 20 years (11 during the overlapping period). Through analysis, we observe that SOC analysts are facing excessive alerts (24K–134K per day), but only a small percentage of the alerts (0.01%) are associated with true attacks. While the majority of true attacks can be detected within the same day, the post-attack investigation takes much longer time (53 days on average). Furthermore, we observe a significant portion of the alerts are related to "attack attempts'' (attacks that did not lead to true compromises, 27%), and "benign triggers'' (correctly matched security events but had business-justified explanations, 49%). Empirically, we show there are opportunities to use rare/abnormal alert patterns to help isolate signals related to true attacks. Given that enterprise SOCs rarely disclose internal data, this paper helps contextualize SOCs' pain points and refine existing problem definitions.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {299601,
author = {Limin Yang and Zhi Chen and Chenkai Wang and Zhenning Zhang and Sushruth Booma and Phuong Cao and Constantin Adam and Alexander Withers and Zbigniew Kalbarczyk and Ravishankar K. Iyer and Gang Wang},
title = {True Attacks, Attack Attempts, or Benign Triggers? An Empirical Measurement of Network Alerts in a Security Operations Center},
booktitle = {33rd USENIX Security Symposium (USENIX Security 24)},
year = {2024},
isbn = {978-1-939133-44-1},
address = {Philadelphia, PA},
pages = {1525--1542},
url = {https://www.usenix.org/conference/usenixsecurity24/presentation/yang-limin},
publisher = {USENIX Association},
month = aug
}

Presentation Video