Michael Fenn, D. E. Shaw Research, LLC
fs123 is a read-only, scalable, high-performance, network filesystem running
in production, delivering petabytes of simulation data and tens of thousands of
software packages to thousands of geographically-distributed clients.
The fs123 protocol is layered over HTTP and leverages that ecosystem (load
balancers, proxies, redirects, etc.). The fs123 protocol provides mechanisms
for loosely-coupled servers to assert that two files are the same to the
client, which allows horizontal scaling of fs123 servers.
fs123 is WAN-friendly, requiring a minimum number of round-trips for each
operation as a result of design decisions that require the client to make a
minimum number of round trips and tunable caching features.
Since 2016, fs123 has run in production on ~5000 machines across several
sites. The client uses the FUSE low-level API and can work through network
outages (or even offline) once the on-disk cache is primed. The libevent-based
server easily delivers data at 40 Gbps.
Michael Fenn, D. E. Shaw Research, LLC
Michael Fenn is a Research Engineer at D. E. Shaw Research, LLC and holds a B. S. and M. S. in Computer Science from Clemson University. He first became interested in fault-tolerant filesystems after suffering through one-too-many NFS outages. When he's not at work, he enjoys automotive track events and doing burnouts.
author = {Michael Fenn},
title = {fs123: An {Open-Source}, Network Filesystem with Pervasive Caching},
year = {2019},
address = {Portland, OR},
publisher = {USENIX Association},
month = oct
}