Facilitating Magnetic Recording Technology Scaling for Data Center Hard Disk Drives through Filesystem-Level Transparent Local Erasure Coding


Yin Li and Hao Wang, Rensselaer Polytechnic Institute; Xuebin Zhang, Dell EMC/DSSD; Ning Zheng, Rensselaer Polytechnic Institute; Shafa Dahandeh, Western Digital; Tong Zhang, Rensselaer Polytechnic Institute


This paper presents a simple yet effective design solution to facilitate technology scaling for hard disk drives (HDDs) being deployed in data centers. Emerging magnetic recording technologies improve storage areal density mainly through reducing the track pitch, which however makes HDDs subject to higher read retry rates. More frequent HDD read retries could cause intolerable tail latency for large-scale systems such as data centers. To reduce the occurrence of costly read retry, one intuitive solution is to apply erasure coding locally on each HDD or JBOD (just a bunch of disks). To be practically viable, local erasure coding must have very low coding redundancy, which demands very long codeword length (e.g., one codeword spans hundreds of 4kB sectors) and hence large file size. This makes local erasure coding mainly suitable for data center applications. This paper contends that local erasure coding should be implemented transparently within filesystems, and accordingly presents a basic design framework and elaborates on important design issues. Meanwhile, this paper derives the mathematical formulations for estimating its effect on reducing HDD read tail latency. Using Reed-Solomon (RS) based erasure codes as test vehicles, we carried out detailed analysis and experiments to evaluate its implementation feasibility and effectiveness. We integrated the developed design solution into ext4 to further demonstrate its feasibility and quantitatively measure its impact on average speed performance of various big data benchmarks.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {202250,
author = {Yin Li and Hao Wang and Xuebin Zhang and Ning Zheng and Shafa Dahandeh and Tong Zhang},
title = {Facilitating Magnetic Recording Technology Scaling for Data Center Hard Disk Drives through {Filesystem-Level} Transparent Local Erasure Coding},
booktitle = {15th USENIX Conference on File and Storage Technologies (FAST 17)},
year = {2017},
isbn = {978-1-931971-36-2},
address = {Santa Clara, CA},
pages = {135--148},
url = {https://www.usenix.org/conference/fast17/technical-sessions/presentation/li},
publisher = {USENIX Association},
month = feb,

Presentation Audio