Queueing Theory in Practice: Performance Modeling for the Working Engineer

Wednesday, November 01, 2017 - 2:30 pm3:00 pm

Eben Freeman, Honeycomb.io


Cloud! Autoscaling! Kubernetes! Etc! In theory, it's easier than ever to scale a service based on variable demand. In practice, it's still hard to take observed metrics, and translate them into quantitative predictions about what will happen to service performance as load changes. Resource limits are often chosen by guesstimation, and teams are likely to find themselves reacting to slowdowns and bottlenecks, rather than anticipating them. Queueing theory can help, by treating large-scale software systems as mathematical models that you can rigorously reason about. But it's not necessarily easy to translate between real-world systems and textbook models. This talk will cover practical techniques for turning operational data into actionable predictions. We'll show how to use the Universal Scalability Law to develop a model of system performance, and how to leverage that model to make more informed capacity planning and architectural decisions. We'll discuss what data to gather in production to better inform its predictions -- for example, why it's important to capture the shape of a latency distribution, and not just a few percentiles. We'll also talk about some of the limitations and pitfalls of performance modelling.

Eben Freeman, Honeycomb.io

Now largely reformed after stints studying theoretical mathematics and climbing rocks, Eben is fascinated by tools that help humans better understand the systems they create. He works as an engineer at Honeycomb.io.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

Presentation Audio

@conference {207183,
author = {Eben Freeman},
title = {Queueing Theory in Practice: Performance Modeling for the Working Engineer},
year = {2017},
address = {San Francisco, CA},
publisher = {{USENIX} Association},