Its Not Your Father’s Scale-out Storage Architecture!

Compute, Network, Memory, and Storage hardware are undergoing a disruptive transformation within the data-center. In addition to hardware shifts, the next generation applications are non-POSIX with different consistency, performance, scaling tradeoffs. This talk explores the implications of these shifts in the context of designing the next generation scale-out architecture for enterprise storage.

In this talk, we focus on ten fundamental building blocks of a scale-out storage architecture, and analyze the evolution of implementation design patterns as a function of the hardware and application evolution. The analysis is based on examples from different generation of distributed storage and database architectures namely Network File-systems, Shared-disk HPC file-systems, Shared Nothing solutions (General-purpose and Big Data specific), Cloud Storage systems, NoSQL data management, and the recent emergence of in-memory storage solutions.