Andromeda: Performance, Isolation, and Velocity at Scale in Cloud Network Virtualization


Michael Dalton, David Schultz, Jacob Adriaens, Ahsan Arefin, Anshuman Gupta, Brian Fahs, Dima Rubinstein, Enrique Cauich Zermeno, Erik Rubow, James Alexander Docauer, Jesse Alpert, Jing Ai, Jon Olson, Kevin DeCabooter, Marc de Kruijf, Nan Hua, Nathan Lewis, Nikhil Kasinadhuni, Riccardo Crepaldi, Srinivas Krishnan, Subbaiah Venkata, Yossi Richter, Uday Naik, and Amin Vahdat, Google, Inc.


This paper presents our design and experience with Andromeda, Google Cloud Platform’s network virtualization stack. Our production deployment poses several challenging requirements, including performance isolation among customer virtual networks, scalability, rapid provisioning of large numbers of virtual hosts, bandwidth and latency largely indistinguishable from the underlying hardware, and high feature velocity combined with high availability.

Andromeda is designed around a flexible hierarchy of flow processing paths. Flows are mapped to a programming path dynamically based on feature and performance requirements. We introduce the Hoverboard programming model, which uses gateways for the long tail of low bandwidth flows, and enables the control plane to program network connectivity for tens of thousands of VMs in seconds. The on-host dataplane is based around a high-performance OS bypass software packet processing path. CPU-intensive per packet operations with higher latency targets are executed on coprocessor threads. This architecture allows Andromeda to decouple feature growth from fast path performance, as many features can be implemented solely on the coprocessor path. We demonstrate that the Andromeda datapath achieves performance that is competitive with hardware while maintaining the flexibility and velocity of a software-based architecture.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {211243,
author = {Michael Dalton and David Schultz and Jacob Adriaens and Ahsan Arefin and Anshuman Gupta and Brian Fahs and Dima Rubinstein and Enrique Cauich Zermeno and Erik Rubow and James Alexander Docauer and Jesse Alpert and Jing Ai and Jon Olson and Kevin DeCabooter and Marc de Kruijf and Nan Hua and Nathan Lewis and Nikhil Kasinadhuni and Riccardo Crepaldi and Srinivas Krishnan and Subbaiah Venkata and Yossi Richter and Uday Naik and Amin Vahdat},
title = {Andromeda: Performance, Isolation, and Velocity at Scale in Cloud Network Virtualization},
booktitle = {15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18)},
year = {2018},
isbn = {978-1-931971-43-0},
address = {Renton, WA},
pages = {373--387},
url = {},
publisher = {{USENIX} Association},