Wormhole: Reliable Pub-Sub to Support Geo-replicated Internet Services

Authors: 

Yogeshwer Sharma, Philippe Ajoux, Petchean Ang, David Callies, Abhishek Choudhary, Laurent Demailly, Thomas Fersch, Liat Atsmon Guz, Andrzej Kotulski, Sachin Kulkarni, Sanjeev Kumar, Harry Li, Jun Li, Evgeniy Makeev, and Kowshik Prakasam, Facebook; Robbert van Renesse, Cornell University; Sabyasachi Roy, Pratyush Seth, Yee Jiun Song, Benjamin Wester, Kaushik Veeraraghavan, and Peter Xie, Facebook

Abstract: 

Wormhole is a publish-subscribe (pub-sub) system developed for use within Facebook’s geographically replicated datacenters. It is used to reliably replicate changes among several Facebook services including TAO, Graph Search and Memcache. This paper describes the design and implementation of Wormhole as well as the operational challenges of scaling the system to support the multiple data storage systems deployed at Facebook. Our production deployment of Wormhole transfers over 35 GBytes/sec in steady state (50 millions messages/sec or 5 trillion messages/day) across all deployments with bursts up to 200 GBytes/sec during failure recovery. We demonstrate that Wormhole publishes updates with low latency to subscribers that can fail or consume updates at varying rates, without compromising efficiency.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {188996,
author = {Yogeshwer Sharma and Philippe Ajoux and Petchean Ang and David Callies and Abhishek Choudhary and Laurent Demailly and Thomas Fersch and Liat Atsmon Guz and Andrzej Kotulski and Sachin Kulkarni and Sanjeev Kumar and Harry Li and Jun Li and Evgeniy Makeev and Kowshik Prakasam and Robbert Van Renesse and Sabyasachi Roy and Pratyush Seth and Yee Jiun Song and Benjamin Wester and Kaushik Veeraraghavan and Peter Xie},
title = {Wormhole: Reliable {Pub-Sub} to Support Geo-replicated Internet Services},
booktitle = {12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15)},
year = {2015},
isbn = {978-1-931971-218},
address = {Oakland, CA},
pages = {351--366},
url = {https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/sharma},
publisher = {USENIX Association},
month = may
}
Download
Sharma PDF
View the slides

Presentation Video 

Presentation Audio