MixApart: Decoupled Analytics for Shared Storage Systems

Authors: 

Madalin Mihailescu, University of Toronto and NetApp; Gokul Soundararajan, NetApp; Cristiana Amza, University of Toronto

Abstract: 

Distributed file systems built for data analytics and enterprise storage systems have very different functionality requirements. For this reason, enabling analytics on enterprise data commonly introduces a separate analytics storage silo. This generates additional costs, and inefficiencies in data management, e.g., whenever data needsto be archived, copied, or migrated across silos.

MixApart uses an integrated data caching and scheduling solution to allow MapReduce computations to analyze data stored on enterprise storage systems. The front-end caching layer enables the local storage performance required by data analytics. The shared storage back-end simplifies data management.

We evaluate MixApart using a 100-core Amazon EC2 cluster with micro-benchmarks and production workload traces. Our evaluation shows that MixApart provides (i) up to 28% faster performance than the traditional ingest then-compute workflows used in enterprise IT analytics, and (ii) comparable performance to an ideal Hadoop setup without data ingest, at similar cluster sizes.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

Presentation Video

Download Video

Presentation Audio

Mihailescu PDF
View the slides
BibTeX
@inproceedings {180733,
author = {Madalin Mihailescu and Gokul Soundararajan and Cristiana Amza},
title = {MixApart: Decoupled Analytics for Shared Storage Systems},
booktitle = {Presented as part of the 11th {USENIX} Conference on File and Storage Technologies ({FAST} 13)},
year = {2013},
isbn = {978-1-931971-99-7},
address = {San Jose, CA},
pages = {133--146},
url = {https://www.usenix.org/conference/fast13/technical-sessions/presentation/mihailescu},
publisher = {{USENIX}},
}
Download