SelectiveEC: Selective Reconstruction in Erasure-coded Storage Systems

Authors: 

Liangliang Xu, Min Lyu, Qiliang Li, Lingjiang Xie, and Yinlong Xu, University of Science and Technology of China

Abstract: 

Erasure coding has been a commonly used approach to provide high reliability with low storage cost. But the skewed load in a recovery batch severely slows down the failure recovery process in storage systems. To this end, we propose a balanced scheduling module, SelectiveEC, which schedules reconstruction tasks out of order by dynamically selecting some stripes to be reconstructed into a batch and selecting source nodes and replacement nodes for each reconstruction task. So it achieves balanced network recovery traffic, computing resources and disk I/Os against single node failure in erasure-coded storage systems. Compared with conventional random reconstruction, SelectiveEC increases the parallelism of recovery process up to 106% and averagely bigger than 97% in our simulation. Therefore, SelectiveEC not only speeds up recovery process, but also reduces the interference of failure recovery with the front-end applications.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {254306,
author = {Liangliang Xu and Min Lyu and Qiliang Li and Lingjiang Xie and Yinlong Xu},
title = {SelectiveEC: Selective Reconstruction in Erasure-coded Storage Systems},
booktitle = {12th {USENIX} Workshop on Hot Topics in Storage and File Systems (HotStorage 20)},
year = {2020},
url = {https://www.usenix.org/conference/hotstorage20/presentation/xu},
publisher = {{USENIX} Association},
month = jul,
}

Presentation Video