usenix conference policies
Explicit Control in the Batch-Aware Distributed File System
We present the design, implementation, and evaluation of the Batch-Aware Distributed File System (BAD-FS), a system designed to orchestrate large, I/O-intensive batch workloads on remote computing clusters distributed across the wide area. BAD-FS consists of two novel components: a storage layer that exposes control of traditionally fixed policies such as caching, consistency, and replication; and a scheduler that exploits this control as necessary for different workloads. By extracting control from the storage layer and placing it within an external scheduler, BAD-FS manages both storage and computation in a coordinated way while gracefully dealing with cache consistency, fault-tolerance, and space management issues in a workload-specific manner. Using both microbenchmarks and real workloads, we demonstrate the performance benefits of explicit control, delivering excellent end-to-end performance across the wide-area.
author = {John Bent and Douglas Thain and Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau and Miron Livny},
title = {Explicit Control in the {Batch-Aware} Distributed File System},
booktitle = {First Symposium on Networked Systems Design and Implementation (NSDI 04)},
year = {2004},
address = {San Francisco, CA},
url = {https://www.usenix.org/conference/nsdi-04/explicit-control-batch-aware-distributed-file-system},
publisher = {USENIX Association},
month = mar
}
connect with us