Hashing Design in Modern Networks: Challenges and Mitigation Techniques

Authors: 

Yunhong Xu, Texas A&M University; Keqiang He and Rui Wang, Google; Minlan Yu, Harvard University; Nick Duffield, Texas A&M University; Hassan Wassel, Shidong Zhang, Leon Poutievski, Junlan Zhou, and Amin Vahdat, Google

Abstract: 

Traffic load balancing across multiple paths is a critical task for modern networks to reduce network congestion and improve network efficiency. Hashing which is the foundation of traffic load balancing still faces practical challenges. The key problem is there is a growing need for more hash functions because networks are getting larger with more switches, more stages, and increased path diversity. Meanwhile, topology and routing become more agile in order to efficiently serve traffic demands with stricter throughput and latency SLAs. On the other hand, current generation switch chips only provide a limited number of uncorrelated hash functions. We first demonstrate why the limited number of hashing functions is a practical challenge in today's datacenter network (DCN) and wide-area network (WAN) designs. Then, to mitigate the problem, we propose a novel approach named color recombining which enables hash functions to reuse via leveraging topology traits of multi-stage DCN networks. We also describe a novel framework based on coprime theory to mitigate hash correlation in generic mesh topologies (i.e., spineless DCN and WAN). Our evaluation using real network trace data and topologies demonstrate that we can reduce the extent of load imbalance (measured by the coefficient of variation) by an order of magnitude.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {280752,
author = {Yunhong Xu and Keqiang He and Rui Wang and Minlan Yu and Nick Duffield and Hassan Wassel and Shidong Zhang and Leon Poutievski and Junlan Zhou and Amin Vahdat},
title = {Hashing Design in Modern Networks: Challenges and Mitigation Techniques},
booktitle = {2022 USENIX Annual Technical Conference (USENIX ATC 22)},
year = {2022},
isbn = {978-1-939133-29-45},
address = {Carlsbad, CA},
pages = {805--818},
url = {https://www.usenix.org/conference/atc22/presentation/xu},
publisher = {USENIX Association},
month = jul
}

Presentation Video