Exploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage

Authors: 

Yuchong Hu, Liangfeng Cheng, and Qiaori Yao, Huazhong University of Science & Technology; Patrick P. C. Lee, The Chinese University of Hong Kong; Weichun Wang and Wei Chen, HIKVISION

Abstract: 

Erasure coding is a low-cost redundancy mechanism for distributed storage systems by storing stripes of data and parity chunks. Wide stripes are recently proposed to suppress the fraction of parity chunks in a stripe to achieve extreme storage savings. However, wide stripes aggravate the repair penalty, while existing repair-efficient approaches for erasure coding cannot effectively address wide stripes. In this paper, we propose combined locality, the first mechanism that systematically addresses the wide-stripe repair problem via the combination of both parity locality and topology locality. We further augment combined locality with efficient encoding and update schemes. Experiments on Amazon EC2 show that combined locality reduces the single-chunk repair time by up to 90.5% compared to locality-based state-of-the-arts, with only a redundancy of as low as 1:063x.

FAST '21 Open Access Sponsored by NetApp

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {264860,
author = {Yuchong Hu and Liangfeng Cheng and Qiaori Yao and Patrick P. C. Lee and Weichun Wang and Wei Chen},
title = {Exploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage},
booktitle = {19th {USENIX} Conference on File and Storage Technologies ({FAST} 21)},
year = {2021},
isbn = {978-1-939133-20-5},
pages = {233--248},
url = {https://www.usenix.org/conference/fast21/presentation/hu},
publisher = {{USENIX} Association},
month = feb,
}

Presentation Video