The Case for Dual-access File Systems over Object Storage

Authors: 

Kunal Lillaney, Johns Hopkins University; Vasily Tarasov and David Pease, IBM Research-Almaden; Randal Burns, Johns Hopkins University

Abstract: 

Object storage has emerged as a low-cost and hyper-scalable alternative to distributed file systems. However, interface incompatibilities and performance limitations often compel users to either transfer data between a file system and object storage or use inefficient file connectors over object stores. The result is growing storage sprawl, unacceptably low performance, and an increase in associated storage costs. One promising solution to this problem is providing dual access, the ability to transparently read and write the same data through both file system interfaces and object storage APIs. In this position paper we argue that there is a need for dual-access file systems over object storage, and examine some representative use cases which benefit from such systems. Based on our conversations with end users, we discuss features which we believe are essential or desirable in a dual-access object storage file system (OSFS). Further, we design and implement an early prototype of Agni, an efficient dual-access OSFS which overcomes the shortcomings of existing approaches. Our preliminary experiments demonstrate that for some representative workloads Agni can improve performance by 20%--60% compared to either S3FS, a popular OSFS, or the prevalent approach of manually copying data between different storage systems.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {234753,
author = {Kunal Lillaney and Vasily Tarasov and David Pease and Randal Burns},
title = {The Case for Dual-access File Systems over Object Storage},
booktitle = {11th {USENIX} Workshop on Hot Topics in Storage and File Systems (HotStorage 19)},
year = {2019},
address = {Renton, WA},
url = {https://www.usenix.org/conference/hotstorage19/presentation/lillaney},
publisher = {{USENIX} Association},
month = jul,
}