Aquila: A unified, low-latency fabric for datacenter networks

Authors: 

Dan Gibson, Hema Hariharan, Eric Lance, Moray McLaren, Behnam Montazeri, Arjun Singh, Stephen Wang, Hassan M. G. Wassel, Zhehua Wu, Sunghwan Yoo, Raghuraman Balasubramanian, Prashant Chandra, Michael Cutforth, Peter Cuy, David Decotigny, Rakesh Gautam, Alex Iriza, Milo M. K. Martin, Rick Roy, Zuowei Shen, Ming Tan, Ye Tang, Monica Wong-Chan, Joe Zbiciak, and Amin Vahdat, Google

Abstract: 

Datacenter workloads have evolved from the data intensive, loosely-coupled workloads of the past decade to more tightly coupled ones, wherein ultra-low latency communication is essential for resource disaggregation over the network and to enable emerging programming models.

We introduce Aquila, an experimental datacenter network fabric built with ultra-low latency support as a first-class design goal, while also supporting traditional datacenter traffic. Aquila uses a new Layer 2 cell-based protocol, GNet, an integrated switch, and a custom ASIC with low-latency Remote Memory Access (RMA) capabilities co-designed with GNet. We demonstrate that Aquila is able to achieve under 40 microseconds tail fabric Round Trip Time (RTT) for IP traffic and sub-10 microseconds RMA execution time across hundreds of host machines, even in the presence of background throughput-oriented IP traffic. This translates to more than 5x reduction in tail latency for a production quality key-value store running on a prototype Aquila network.

NSDI '22 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

BibTeX
@inproceedings {278358,
author = {Dan Gibson and Hema Hariharan and Eric Lance and Moray McLaren and Behnam Montazeri and Arjun Singh and Stephen Wang and Hassan M. G. Wassel and Zhehua Wu and Sunghwan Yoo and Raghuraman Balasubramanian and Prashant Chandra and Michael Cutforth and Peter Cuy and David Decotigny and Rakesh Gautam and Alex Iriza and Milo M. K. Martin and Rick Roy and Zuowei Shen and Ming Tan and Ye Tang and Monica Wong-Chan and Joe Zbiciak and Amin Vahdat},
title = {Aquila: A unified, low-latency fabric for datacenter networks},
booktitle = {19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22)},
year = {2022},
isbn = {978-1-939133-27-4},
address = {Renton, WA},
pages = {1249--1266},
url = {https://www.usenix.org/conference/nsdi22/presentation/gibson},
publisher = {USENIX Association},
month = apr
}

Presentation Video