Credible Threat Detection? Measuring Contribution Dynamics and Quality Control in a Crowdsourced Threat Detection Ecosystem

Apurva Virkud, Gang Wang, and Adam Bates, University of Illinois Urbana-Champaign

Abstract Community exchange serves as a critical component of modern day security operations. Commercial security vendors often draw from crowdsourced intelligence and rules to power their proprietary systems. Additionally, Managed Security Service Providers (MSSPs) and Managed Detection and Response (MDR) solutions leverage both crowdsourced and commercial feeds to provide security services to organizations without their own security infrastructure. However, there is no systematic understanding of how these crowdsourced detection rules are developed or vetted. In this work, we explore the open-source and crowdsourced Sigma repository as a window into the development of threat detection rules. We find that the Sigma repository continuously iterates upon its rules, especially with tuning of false positives based on feedback from an active downstream userbase. Through an investigation of the quality control process, we observe evidence of informal rule evaluation, but also of inexperienced contributions from the broader community. Finally, we perform a comparative analysis of rules that have migrated between Sigma and other commercial Security Information and Event Management (SIEM) vendors (and vice versa) and observe divergences across platforms. Although crowdsourcing allows the community to iterate on commercial rules, we anecdotally observe that such contributions can introduce increased risk of rule evasion from the adversary. While Sigma is an active resource for vendors and the broader security community, it can benefit from more systematic and automated quality control procedures. As a first step, we introduce a tool to identify incoming duplicate rules to aid in the review process.

USENIX Security '26 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {320613,
author = {Apurva Virkud and Gang Wang and Adam Bates},
title = {Credible Threat Detection? Measuring Contribution Dynamics and Quality Control in a Crowdsourced Threat Detection Ecosystem},
booktitle = {35th USENIX Security Symposium (USENIX Security 26)},
year = {2026},
isbn = {978-1-939133-58-8},
address = {Baltimore, MD},
pages = {6897--6916},
url = {https://www.usenix.org/conference/usenixsecurity26/presentation/virkud},
publisher = {USENIX Association},
month = aug
}

Presentation Video