CodedBulk: Inter-Datacenter Bulk Transfers using Network Coding

Authors: 

Shih-Hao Tseng, Saksham Agarwal, and Rachit Agarwal, Cornell University; Hitesh Ballani, Microsoft Research; Ao Tang, Cornell University

Abstract: 

This paper presents CodedBulk, a system for high-throughput inter-datacenter bulk transfers. At its core, CodedBulk uses network coding, a technique from the coding theory community, that guarantees optimal throughput for individual bulk transfers. Prior attempts to using network coding in wired networks have faced several pragmatic and fundamental barriers. CodedBulk resolves these barriers by exploiting the unique properties of inter-datacenter networks, and by using a custom-designed hop-by-hop flow control mechanism that enables efficient realization of network coding atop existing transport protocols. An end-to-end CodedBulk implementation running on a geo-distributed inter-datacenter network improves bulk transfer throughput by 1.2-2.5x compared to state-of-the-art mechanisms that do not use network coding.

NSDI '21 Open Access Sponsored by NetApp

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {265023,
author = {Shih-Hao Tseng and Saksham Agarwal and Rachit Agarwal and Hitesh Ballani and Ao Tang},
title = {{CodedBulk}: {Inter-Datacenter} Bulk Transfers using Network Coding},
booktitle = {18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21)},
year = {2021},
isbn = {978-1-939133-21-2},
pages = {15--28},
url = {https://www.usenix.org/conference/nsdi21/presentation/tseng},
publisher = {USENIX Association},
month = apr
}

Presentation Video