Improving How We Observe Our Observability Data: Techniques for SREs

Wednesday, March 16, 2022 - 9:40 am10:20 am

Dan Shoop, iWiring

Abstract: 

Time-series charts have been around for hundreds of years, yet were originally created with a narrative intent often missed by many engineers today. This session will examine some historic time-series examples, explore what we can learn as SREs, and look at concrete charting techniques we can use to improve the cognition of our SLO narratives, engineering reports and incident retrospectives using multi-variate relationships, small multiples and sparklines, while avoiding some common pitfalls we often we find in engineering presentations.

Dan Shoop, iWiring

Dan Shoop is a Systems Reliability Engineering Manager with over 30 years of experience building distributed systems and infrastructure that are performant, highly-available, scalable and fault tolerant. While working at HBO solving Internet problems at Game of Thrones scale when that Google SRE book came out, he and his team realized they had actually been SREs most of their engineering careers, embraced the new paradigm to restructure their production operations teams, and driving their focus heavily on telemetry & observability as a critical component for understanding, measuring, monitoring and alerting on key indicators of systems health as related to its service impacts and architectural improvements. He went on to lead SRE at Venmo and has also worked at Sesame Street, United States Technical Services and operated his own consulting company. Having taken Edward Tufte's course on Information Visualization three times, he both recognizes and enjoys sharing techniques for improving the cognition and presentation of our observability data in terms of SLI/SLO narratives, and enhancing our engineering retrospectives and reports. He lives in New York City and enjoys good food, mountaineering, photography and UAVs.

SREcon22 Americas Open Access Sponsored by Blameless

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@conference {278156,
author = {Dan Shoop},
title = {Improving How We Observe Our Observability Data: Techniques for {SREs}},
year = {2022},
address = {San Francisco, CA},
publisher = {USENIX Association},
month = mar
}

Presentation Video