Five-sigma Network Events (and How to Find Them)

Wednesday, October 31, 2018 - 9:00 am9:30 am

John O'Neil, Edgewise Networks

Abstract: 

Networks are complex systems and too often, despite their best effort, no one knows everything about what's going on. And most of the knowledge about the network is about typical activity. But what about the atypical activity?

There are many reasons to want to find unusual behavior in your network. The biggest reason is that it may be a sign of something new and unexpected—rather than the usual stuff—driving the activity. This doesn't necessarily imply that a network intrusion in underway. There are many other possibilities, both innocuous and dangerous. In any case, though, unusual behavior is probably something you want to know.

There are a variety of tools related to "anomaly detection" or "outlier detection," and this talk isn't about any of them. Instead, this talk is an introduction to writing your own tools for detecting unusual network events. We'll use Python, with some easily available pip installations, and look at some simple approaches to the problem that answer some interesting questions and scale well.

The code will be made available, but the point is not that the code is a complete solution—the point is, rather, that it's a starting point for creating tools that tell netops folks what they want (and need) to know. 

John O'Neil, Edgewise Networks

John O’Neil is the Data Scientist at Edgewise Networks. He writes and designs software for data analysis and analytics, search engines, natural language processing and machine learning. He has a PhD in linguistics from Harvard University, and is the author of more than twenty papers in Computer Science, Linguistics, and associated fields, and has given talks at numerous professional and academic conferences.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

Presentation Audio

BibTeX
@conference {221782,
author = {John O{\textquoteright}Neil},
title = {Five-sigma Network Events (and How to Find Them)},
year = {2018},
address = {Nashville, TN},
publisher = {{USENIX} Association},
}