Jonah Berquist and Gillian Gunson, GitHub
The database team at GitHub is tasked with keeping the data available and with maintaining its integrity. Our infrastructure automates away much of our operation, but automation requires trust, and trust is gained by testing. This session highlights three examples of infrastructure testing automation that helps us sleep better at night:
- Backups: auto-restores and backup validation. Making backup data accessible to our engineers. What metrics and alerts we have in place.
- Failovers: how we continuously test our failover mechanism, orchestrator. How we setup a failover scenario, what defines a successful failover, how we automate away the cleanup. What we do in production.
- Schema migrations: how we ensure that gh-ost, our schema migration tool, which keeps rewriting our (and your!) data, does the right thing. How we test new branches in production without putting production data at risk.
Jonah Berquist, GitHub
Jonah is the Engineering Manager of the Database Infrastructure team at GitHub. He previously worked as a Senior DBA at Twitter and he began his career with databases as a remote DBA for a variety of customers at Blue Gecko. He enjoys looking at graphs and writing scripts to do his job for him.
Gillian Gunson, GitHub
Gillian has been a database infrastructure engineer at GitHub for two years. Her focus has been on performance troubleshooting and observability. Previous employers include Okta, PalominoDB, Oracle, and Disney. Gillian is based in Vancouver, BC, Canada.
author = {Jonah Berquist and Gillian Gunson},
title = {{MySQL} Infrastructure Testing Automation at {GitHub}},
year = {2018},
address = {Nashville, TN},
publisher = {USENIX Association},
month = oct
}