Building a DDoS Mitigation Pipeline

Monday, January 25, 2016 - 4:00pm4:30pm

Marek Majkowski, CloudFlare

Abstract: 

Over the last two years we've fully rewritten CloudFlare's DDoS mitigation pipeline. Our initial goal was to relieve the our over-worked OPS team and reduce their distractions related to reacting to DDoS'es. The system we created proved to be capable of much more than we expected. Not only it is quicker and makes less mistakes than human operators, but also it allowed us to deploy new mitigation techniques much faster. 

The main design goal of the new pipeline was to avoid latency and be able to deploy mitigation in real-time, immediately after the threat is detected. To achieve this first we use sampled packets from switches (sflow) and http logs as a data source and automatically categorize them into various attack types. Then, the categorized attack metadata runs through a rich logic expressed in our reactive programming engine, which allows us to express high level constraints. Finally, this metadata is a source for the centrally-managed iptables mitigations framework. 

While composed of many moving pieces, our framework is, at least in spirit, fairly simple, and most importantly practical. We've successfully automated mitigations to most common attacks and nowadays the OPS team rarely needs to manually deploy mitigations. In this talk we'll discuss the design of the new mitigation framework, the context behind it, our incremental development and the future work.

Marek Majkowski, CloudFlare

After fruitful encounters with such diverse topics as high performance key value databases, distributed queueing systems, making real time web communication enjoyable, and accelerating the time so that testing servers and protocols takes seconds, Marek Majkowski finally settled for working on DDoS mitigation in the CloudFlare London office, where he appreciates most the parking space for his motorbike.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@conference {206257,
author = {Marek Majkowski},
title = {Building a {DDoS} Mitigation Pipeline},
year = {2016},
address = {San Francisco, CA},
publisher = {USENIX Association},
month = jan
}

Presentation Video