Scalog: Seamless Reconfiguration and Total Order in a Scalable Shared Log

Authors: 

Cong Ding, David Chu, and Evan Zhao, Cornell University; Xiang Li, Alibaba Group; Lorenzo Alvisi and Robbert van Renesse, Cornell University

Abstract: 

The shared log paradigm is at the heart of modern distributed applications in the growing cloud computing industry. Often, application logs must be stored durably for analytics, regulations, or failure recovery, and their smooth operation depends closely on how the log is implemented. Scalog is a new implementation of the shared log abstraction that offers an unprecedented combination of features for continuous smooth delivery of service: Scalog allows applications to customize data placement, supports reconfiguration with no loss in availability, and recovers quickly from failures. At the same time, Scalog provides high throughput and total order.

The paper describes the design and implementation of Scalog and presents examples of applications running upon it. To evaluate Scalog at scale, we use a combination of real experiments and emulation. Using 4KB records, a 10 Gbps infrastructure, and SSDs, Scalog can totally order up to 52 million records per second.

NSDI '20 Open Access Sponsored by NetApp

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {246300,
author = {Cong Ding and David Chu and Evan Zhao and Xiang Li and Lorenzo Alvisi and Robbert Van Renesse},
title = {Scalog: Seamless Reconfiguration and Total Order in a Scalable Shared Log },
booktitle = {17th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 20)},
year = {2020},
isbn = {978-1-939133-13-7},
address = {Santa Clara, CA},
pages = {325--338},
url = {https://www.usenix.org/conference/nsdi20/presentation/ding},
publisher = {{USENIX} Association},
month = feb,
}

Presentation Video