Thomas E. Anderson, University of Washington; Marco Canini, KAUST; Jongyul Kim, KAIST; Dejan Kostić, KTH Royal Institute of Technology; Youngjin Kwon, KAIST; Simon Peter, The University of Texas at Austin; Waleed Reda, KTH Royal Institute of Technology and Université catholique de Louvain; Henry N. Schuh, University of Washington; Emmett Witchel, The University of Texas at Austin
The adoption of low latency persistent memory modules (PMMs) upends the long-established model of remote storage for distributed file systems. Instead, by colocating computation with PMM storage, we can provide applications with much higher IO performance, sub-second application failover, and strong consistency. To demonstrate this, we built the Assise distributed file system, based on a persistent, replicated coherence protocol that manages client-local PMM as a linearizable and crash-recoverable cache between applications and slower (and possibly remote) storage. Assise maximizes locality for all file IO by carrying out IO on process-local, socket-local, and client-local PMM whenever possible. Assise minimizes coherence overhead by maintaining consistency at IO operation granularity, rather than at fixed block sizes.
We compare Assise to Ceph/BlueStore, NFS, and Octopus on a cluster with Intel Optane DC PMMs and SSDs for common cloud applications and benchmarks, such as LevelDB, Postfix, and FileBench. We find that Assise improves write latency up to 22x, throughput up to 56x, fail-over time up to 103x, and scales up to 6x better than its counterparts, while providing stronger consistency semantics.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Thomas E. Anderson and Marco Canini and Jongyul Kim and Dejan Kosti{\'c} and Youngjin Kwon and Simon Peter and Waleed Reda and Henry N. Schuh and Emmett Witchel},
title = {Assise: Performance and Availability via Client-local {NVM} in a Distributed File System},
booktitle = {14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20)},
year = {2020},
isbn = {978-1-939133-19-9},
pages = {1011--1027},
url = {https://www.usenix.org/conference/osdi20/presentation/anderson},
publisher = {USENIX Association},
month = nov
}
