Meatbag Systems: How Our Reliability Culture & Practice Evolved over Time

Wednesday, 26 October, 2022 - 16:0016:40 CEST

Andrew Howden and Salomé Santos, Zalando


The reliability of production systems is not only an property of how the system is designed and implemented, but also how the meatbags (people) who maintain those systems understand them, the value of those systems to the end consumers, and their roles and responsibilities.

Our journey to become more reliable has been a wild ride. Through our journey, we've had to examine what the value of reliability is, anchor the view of "reliable systems" in the value our users realize, and create a common understanding about what is "sufficiently reliable". We've had to develop a language, culture, management, and support network to create this common model of reliability and help teams up the "operational maturity curve" --- All, ultimately, in service of customers.

Within this talk, we'll examine this journey by reviewing three major incidents that happened at this organization over some time, and examine how we've evolved our approach.

Andrew Howden, Zalando SE

Andrew is a failed sports science student who wandered into software engineering by virtue of luck and the necessity to find a job in a hurry. Through the grace and patience of his colleagues, he has spent nearly the past decade learning how to be a software engineer, systems engineer, site reliability engineer and student of human factors. Most recently, he is learning how to become an engineering manager and trying to pass on what knowledge he gained so far to the next generation of software adventurers.

Salomé Santos, Zalando SE

Salomé was a math student, who followed a career in software engineering after having had her first contact with programming in the university and fallen in love with it. She started as a shy person, and in many ways still is. She grew and developed herself in a world dominated by men, taking on opportunities and improving her self confidence, and along the way she felt equal to others around her. Nowadays she is an Engineering Manager, applying her passion for people on a daily basis and aiming at have a positive impact on those she works with.

@conference {284633,
author = {Andrew Howden and Salom{\'e} Santos},
title = {Meatbag Systems: How Our Reliability Culture \& Practice Evolved over Time},
year = {2022},
address = {Amsterdam},
publisher = {USENIX Association},
month = oct

Presentation Video