Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • FAST '13 Home
  • Organizers
  • Registration Information
  • Registration Discounts
  • At a Glance
  • Calendar
  • Training Program
  • Technical Sessions
  • Purchase the Box Set
  • Posters and WiPs
  • Birds-of-a-Feather Sessions
  • Sponsors
  • Activities
  • Hotel and Travel Information
  • Services
  • Students
  • Questions
  • Help Promote
  • For Participants
  • Call for Papers
  • Past Proceedings

sponsors

Platinum Sponsor
Gold Sponsor
Gold Sponsor
Gold Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
General Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » To Zip or Not to Zip: Effective Resource Usage for Real-Time Compression
Tweet

connect with us

http://twitter.com/usenix
https://www.facebook.com/usenixassociation
http://www.linkedin.com/groups?home=&gid=49559
http://www.youtube.com/user/USENIXAssociation

To Zip or Not to Zip: Effective Resource Usage for Real-Time Compression

Authors: 

Danny Harnik, Ronen Kat, Oded Margalit, Dmitry Sotnikov, and Avishay Traeger, IBM Research—Haifa

Abstract: 

Real-time compression for primary storage is quickly becoming widespread as data continues to grow exponentially, but adding compression on the data path consumes scarce CPU and memory resources on the storage system. Our work aims to mitigate this cost by introducing methods to quickly and accurately identify the data that will yield significant space savings when compressed.

The first level of filtering that we employ is at the dataset level (e.g., volume or file system), where we estimate the overall compressibility of the data at rest. According to the outcome, we may choose to enable or disable compression for the entire data set, or to employ a second level of finer-grained filtering. The second filtering scheme examines data being written to the storage system in an online manner and determines its compressibility.

The first-level filtering runs in mere minutes while providing mathematically proven guarantees on its estimates. In addition to aiding in selecting which volumes to compress, it has been released as a public tool, allowing potential customers to determine the effectiveness of compression on their data and to aid in capacity planning. The second-level filtering has shown significant CPU savings (up to 35%) while maintaining compression savings (within 2%).

Danny Harnik, IBM Research—Haifa

Ronen Kat, IBM Research—Haifa

Dmitry Sotnikov, IBM Research—Haifa

Avishay Traeger, IBM Research—Haifa

Oded Margalit, IBM Research—Haifa

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {180745,
author = {Danny Harnik and Ronen Kat and Dmitry Sotnikov and Avishay Traeger and Oded Margalit},
title = {To Zip or Not to Zip: Effective Resource Usage for Real-Time Compression},
booktitle = {11th {USENIX} Conference on File and Storage Technologies ({FAST} 13)},
year = {2013},
isbn = {978-1-931971-99-7},
address = {San Jose, CA},
pages = {229--241},
url = {https://www.usenix.org/conference/fast13/technical-sessions/presentation/harnik},
publisher = {{USENIX} Association},
month = feb,
}
Download
Harnik PDF
View the slides

Presentation Video

Presentation Audio

MP3 Download OGG Download

Download Audio

  • Log in or    Register to post comments

Platinum Sponsors

Gold Sponsors

Silver Sponsors

Bronze Sponsors

General Sponsors

Media Sponsors & Industry Partners

© USENIX

  • Privacy Policy
  • Conference Policies
  • Contact Us