Check out the new USENIX Web site.

Chunkfs architecture

The core idea of chunkfs is to split a file system up into many small fault isolation domains on disk--chunks, on the order of a few gigabytes in size (see Fig. 1. Each chunk has its own block number space, allocation bitmaps, superblock, and other traditional per-file system metadata. A small amount of metadata about the location and contents of the chunks is stored outside the chunks in a summary region. The file system namespace and available disk space are still shared, so that it still feels like one file system to the user and administrator.

Figure 1: File system divided into chunks

Splitting up the file system into chunks is easy. The difficulty is in gluing it back together again to preserve the shared namespace and disk space, while keeping chunks fault-isolated from each other. The basic rules are, (1) All cross-chunk references have explicit forward and back pointers, and (2) Keep cross-chunk references to a minimum. This section describes how we use these principles to solve the major difficulties in implementing chunkfs.

Valerie Henson 2006-10-18