Standing On the Shoulders of Giants: Unleashing the Power of Scriptable Load Balancers

Wednesday, 30 August, 2017 - 13:4014:30

Emil Stolarsky, Production Engineer, Shopify


Every year, our organizations continue adding more services. It’s unsustainable to have a dedicated team of SREs for each one. That’s why, as an industry, we’ve moved to the product-team SRE model. We’re now accustomed to building custom services that applications reach out to, but not middleware services that operate on requests before they reach their destination.

Load balancers have the potential to provide application-aware middleware without making changes to the application itself. However, traditional load balancers can’t be easily and deeply customized or redeployed quickly without significant risk. Instead, we can embed a scripting language to fulfill these requirements.

At Shopify, we do this with Nginx and LuaJIT via OpenResty. Our Nginx scripts deploy in 10 seconds, run through a thorough suite of automated tests, and have allowed us to solve sharding across data centers, handle some of the world’s biggest flash sales, and respond quickly to layer 7 DDoS attacks. What once took a large team of engineers can now be accomplished by one of any size.

The lessons learned from building this middleware framework are applicable to any service. By solving hard problems in your load balancers, you can benefit every application or service you run.

Emil Stolarsky, Production Engineer, Shopify

Emil is a production engineer at Shopify where he works on performance, scriptable load balancers, and DNS tooling. When he's not trying to make Shopify's global performance heat map green, he's shivering over a spiked cup of coffee in the great Canadian north.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@conference {205474,
author = {Emil Stolarsky},
title = {Standing On the Shoulders of Giants: Unleashing the Power of Scriptable Load Balancers},
year = {2017},
address = {Dublin},
publisher = {USENIX Association},
month = aug

Presentation Video 

Presentation Audio