Chaos Engineering Bootcamp

Tammy Butow, Gremlin


Chaos engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production. Chaos engineering can be thought of as the facilitation of experiments to uncover systemic weaknesses. These experiments follow four steps:

1. Start by defining “steady state” as some measurable output of a system that indicates normal behavior.
2. Hypothesize that this steady state will continue in both the control group and the experimental group.
3. Introduce variables that reflect real-world events like servers that crash, hard drives that malfunction, network connections that are severed, etc.
4. Try to disprove the hypothesis by looking for a difference in steady state between the control group and the experimental group.

A hands-on tutorial on chaos engineering, covering the tools and practices you need to implement chaos engineering in your organization. Even if you’re already using chaos engineering, you’ll learn to identify new ways to use chaos engineering within your engineering organization and discover how other companies are using chaos engineering—and the positive results they have had using chaos to create reliable distributed systems.

Pre-Reading List

  1. Production-Ready Microservices by Susan Fowler—especially the section on chaos testing at Uber (p. 94)
  2. Principles of Chaos
  3. Chaos Engineering: Building confidence in system behavior through experiments

Prerequisites, Skills, and Tools

  1. A basic understanding of production environments and the infrastructure required to run systems
  2. Experience with Linux, cloud infrastructure, hardware, networking, and systems troubleshooting

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@conference {212985,
author = {Tammy Butow},
title = {Chaos Engineering Bootcamp},
year = {2018},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = mar