Challenges of Machine Learning at Scale

Motivated by the problem of predicting whether any given ad would be clicked in response to a query, in this introductory talk we outline the requirements and large-system design challenges that arise when designing a machine learning system that makes millions of predictions per second with low latency, learns quickly from the responses to those predictions, and maintains a consistent level of model quality over time. We present alternatives for meeting those challenges using diagrams of machine learning pipelines.

Concepts used in this talk: machine learning (classification), software pipelines, sharding and replication, map-reduce

Graham is an SRE at Google working on machine learning pipelines for ad click prediction.