Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • FAST '13 Home
  • Organizers
  • Registration Information
  • Registration Discounts
  • At a Glance
  • Calendar
  • Training Program
  • Technical Sessions
  • Purchase the Box Set
  • Posters and WiPs
  • Birds-of-a-Feather Sessions
  • Sponsors
  • Activities
  • Hotel and Travel Information
  • Services
  • Students
  • Questions
  • Help Promote
  • For Participants
  • Call for Papers
  • Past Proceedings

sponsors

Platinum Sponsor
Gold Sponsor
Gold Sponsor
Gold Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
General Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » HARDFS: Hardening HDFS with Selective and Lightweight Versioning
Tweet

connect with us

http://twitter.com/usenix
https://www.facebook.com/usenixassociation
http://www.linkedin.com/groups?home=&gid=49559
http://www.youtube.com/user/USENIXAssociation

HARDFS: Hardening HDFS with Selective and Lightweight Versioning

Authors: 

Thanh Do, Tyler Harter, and Yingchao Liu, University of Wisconsin—Madison; Haryadi S. Gunawi, University of Chicago; Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau, University of Wisconsin—Madison

Abstract: 

We harden the Hadoop Distributed File System (HDFS) against fail-silent (non fail-stop) behaviors that result from memory corruption and software bugs using a new approach: selective and lightweight versioning (SLEEVE). With this approach, actions performed by important subsystems of HDFS (e.g., namespace management) are checked by a second implementation of the subsystem that uses lightweight, approximate data structures. We show that HARDFS detects and recovers froma wide range of fail-silent behaviors caused by random bit flips, targeted corruptions, and real software bugs. In particular, HARDFS handles 90% of the fail-silentfaults that result from random memory corruption and correctly detects and recovers from 100% of 78 targeted corruptions and 5 real-world bugs. Moreover, it recovers orders of magnitude faster than full reboot by using micro-recovery. The extra protection in HARDFS incurs minimal performance and space overheads.

Thanh Do, University of Wisconsin - Madison

Tyler Harter, University of Wisconsin - Madison

Yingchao Liu, University of Wisconsin - Madison

Haryadi S. Gunawi, University of Chicago

Andrea C. Arpaci-Dusseau, University of Wisconsin - Madison

Remzi H. Arpaci-Dusseau, University of Wisconsin - Madison

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {180731,
author = {Thanh Do and Tyler Harter and Yingchao Liu and Haryadi S. Gunawi and Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau},
title = {{HARDFS}: Hardening {HDFS} with Selective and Lightweight Versioning},
booktitle = {11th {USENIX} Conference on File and Storage Technologies ({FAST} 13)},
year = {2013},
isbn = {978-1-931971-99-7},
address = {San Jose, CA},
pages = {105--118},
url = {https://www.usenix.org/conference/fast13/technical-sessions/presentation/do},
publisher = {{USENIX} Association},
month = feb,
}
Download
Do PDF
View the slides

Presentation Video

Presentation Audio

MP3 Download OGG Download

Download Audio

  • Log in or    Register to post comments

Platinum Sponsors

Gold Sponsors

Silver Sponsors

Bronze Sponsors

General Sponsors

Media Sponsors & Industry Partners

© USENIX

  • Privacy Policy
  • Conference Policies
  • Contact Us