Reliable Data Processing with Minimal Toil

Learn about the risks involved with data processing pipelines and how Google and Slack mitigate them.

This talk shares insights into making batch jobs safer and less manual. We have researched and implemented ways to do canarying, automated global rollouts on increasingly larger target populations and different kinds of validations. All these are necessary to remove the manual work involved in updating a batch job globally across millions of users.