|
OSDI 2000 Abstract
Scalable, Distributed Data Structures
for Internet Service Construction
Steven D. Gribble, Eric A. Brewer, Joseph M. Hellerstein, and David Culler , UC Berkeley
Abstract
This paper presents a new persistent data management layer designed to simplify cluster-based Internet service construction. This self-managing layer,
called a distributed data structure (DDS), presents a conventional single-site data structure interface to service authors, but partitions and replicates the
data across a cluster. We have designed and implemented a distributed hash table DDS that has properties necessary for Internet services (incremental
scaling of throughput and data capacity, fault tolerance and high availability, high concurrency, consistency, and durability). The hash table uses
two-phase commits to present a coherent view of its data across all cluster nodes, allowing any node to service any task. We show that the distributed
hash table simplifies Internet service construction by decoupling service-specific logic from the complexities of persistent, consistent state
management, and by allowing services to inherit the necessary service properties from the DDS rather than having to implement the properties
themselves. We have scaled the hash table to a 128 node cluster, 1 terabyte of storage, and an in-core read throughput of 61,432 operations/s and write
throughput of 13,582 operations/s.
|
Need help? Use our Contacts page.
