The Evolution of Traffic Routing in a Streaming World

This talk covers the evolution of traffic and load balancing solutions on a team focused on providing real-time streaming infrastructure. It starts with the change in architecture from a co-located service model to a distributed one and then covers the system's evolution to a full-mesh connection based architecture. The rationale behind these migrations, the lessons learned, and the benefits reaped are to be covered by the talk. Key takeaways include the advantages of not co-locating services, rendezvous hashing (highest random weight hashing algorithm that allows clients to achieve distributed agreement) and its practical applications in solving traffic problems, benefits of a full mesh connection and the opportunities it unlocks, along with a few other lessons learned along the way.