Stronger Semantics for Low-Latency Geo-Replicated Storage
Authors:
Wyatt Lloyd and Michael J. Freedman, Princeton University; Michael Kaminsky, Intel Labs; David G. Andersen, Carnegie Mellon University
Abstract:
We present the first scalable, geo-replicated storage system that guarantees low latency, offers a rich data model, and provides “stronger” semantics. Namely, all client requests are satisfied in the local datacenter in which they arise; the system efficiently supports useful data model abstractions such as column families and counter columns; and clients can access data in a causally consistent fashion with read-only and write-only transactional support, even for keys spread across many servers. The primary contributions of this work are enabling scalable causal consistency for the complex column family data model, as well as novel, non-blocking algorithms for both read-only and write-only transactions. Our evaluation shows that our system, Eiger, achieves low latency (single-ms), has throughput competitive with eventually-consistent and non-transactional Cassandra (less than 7% overhead for one of Facebook’s real-world workloads), and scales out to large clusters almost linearly (averaging 96% increases up to 128 server clusters).
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
BibTeX
@inproceedings {180313,
author = {Wyatt Lloyd and Michael J. Freedman and Michael Kaminsky and David G. Andersen},
title = {Stronger Semantics for {Low-Latency} {Geo-Replicated} Storage},
booktitle = {10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13)},
year = {2013},
isbn = {978-1-931971-00-3},
address = {Lombard, IL},
pages = {313--328},
url = {https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/lloyd},
publisher = {USENIX Association},
month = apr,
}
Modern Web applications such as Facebook rely on geo-replicated storage, replicating their entire data stores in multiple data centers in order to serve each client quickly from the nearest one. The CAP theorem provides an impossibility result that limits what such services can provide: they cannot simultaneously guarantee both the strongest forms of consistency and complete availability. Since services such as Facebook strive for full availability and fast service, they generally sacrifice consistency, providing only eventual consistency, which does not order operations, complicating programming and potentially exposing inconsistencies to users. Other systems, such as Google's Spanner, provide strong consistency, but sacrifice low latency by blocking on replication across data centers.
This paper presents Eiger, a system (extending the authors' earlier COPS system) that operates at an impressive middle point between these extremes: Eiger guarantees causal consistency (operations that may be causally related are seen in the same order by all clients), while serving each client entirely from within the local data center to provide low latency. Eiger achieves these properties for read-only and write-only transactions over the popular column-family data model (used by Apache Cassandra and Google's BigTable), giving it the potential to serve as a building block for Web services.
Eiger's stronger consistency model may make it easier for developers to reason about systems they build on top of it. Will developers see this as a benefit that outweighs the overhead? Given that it provides nice semantics on top of a realistic and widely-used data model, it will be interesting to see what impact Eiger has.
connect with us