AutoCTF: Creating Diverse Pwnables via Automated Bug Injection


Patrick Hulin and Andy Davis, MIT Lincoln Laboratory; Rahul Sridhar, MIT; Andrew Fasano, Cody Gallagher, Aaron Sedlacek, and Tim Leek, MIT Lincoln Laboratory; Brendan Dolan-Gavitt, New York University


Capture the Flag (CTF) is a popular computer security exercise in which teams competitively attack and/or defend programs in real time. CTFs are currently expensive to build and run; each is a bespoke affair, with challenges and vulnerabilities crafted by experts. This not only limits the educational value for players but also restricts what researchers can learn about human activities during the competition. In this work, we take steps towards making CTFs cheap and reusable by extending our LAVA bug injection system to add exploitable vulnerabilities, enabling rapid generation of new CTF challenges. New LAVA bug types, including memory corruption and address disclosure, form a sufficient set of primitives for program exploitation.

We used these techniques to create AutoCTF, a weeklong event involving teams from four universities. In order to assess how AutoCTF differed from a handmade CTF we conducted surveys and semi-structured interviews after the event. We evaluated both challenge realism and relative effort expended on bug finding and exploit development. Our preliminary results indicate that AutoCTF can form the basis for cost-effective and reusable CTFs, allowing them to be run often and easily. These CTFs can be used to train new generations of security researchers and provide empirical data on human vulnerability discovery and exploit development.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {206144,
author = {Patrick Hulin and Andy Davis and Rahul Sridhar and Andrew Fasano and Cody Gallagher and Aaron Sedlacek and Tim Leek and Brendan Dolan-Gavitt},
title = {AutoCTF: Creating Diverse Pwnables via Automated Bug Injection},
booktitle = {11th {USENIX} Workshop on Offensive Technologies ({WOOT} 17)},
year = {2017},
address = {Vancouver, BC},
url = {},
publisher = {{USENIX} Association},