Bridging the Data Gap: Data Related Challenges in Evaluating Large Scale Collaborative Security Systems

Data-sharing approaches such as collaborative security have been successfully applied to systems addressing multiple classes of cyber security threats. In spite of these results, scale presents a major challenge to further advances: collaborative security systems are designed to operate at a large scale (Internet- or ISP-scale), and obtaining and sharing traces suitable for experimentation is difficult. We illustrate these challenges via an analysis of recently proposed collaborative systems. We argue for the development of simulation techniques designed specifically to address these challenges and sketch one such technique, parameterized trace scaling, which expands small traces to generate realistic large scale traces sufficient for analyzing collaborative security systems.