Swarm: A Log-Structured Storage System for Linux

Swarm is a storage system for Linux that provides scalable, reliable, and cost-effective data storage. At its lowest level, Swarm implements a log-structured interface to a cluster of storage devices. Above the log, Swarm provides an infrastructure that allows high-level abstractions and functionality to be implemented easily and efficiently. This paper describes the design and implementation of Swarm, paying particular attention to the Swarm infrastructure and how it has been used to construct two storage systems: Sting, a log-structured file system for Linux, and ext2fs/Swarm, a Swarm-based version of the Linux ext2 file system that runs unmodified above a block device compatibility layer. The paper concludes with a discussion of our experiences using Linux as a platform for research.