MixApart: Decoupled Analytics for Shared Storage Systems
Authors:
Madalin Mihailescu, University of Toronto and NetApp; Gokul Soundararajan, NetApp; Cristiana Amza, University of Toronto
Abstract:
Distributed file systems built for data analytics and enterprise storage systems have very different functionality requirements. For this reason, enabling analytics on enterprise data commonly introduces a separate analytics storage silo. This generates additional costs, and inefficiencies in data management, e.g., whenever data needsto be archived, copied, or migrated across silos.
MixApart uses an integrated data caching and scheduling solution to allow MapReduce computations to analyze data stored on enterprise storage systems. The front-end caching layer enables the local storage performance required by data analytics. The shared storage back-end simplifies data management.
We evaluate MixApart using a 100-core Amazon EC2 cluster with micro-benchmarks and production workload traces. Our evaluation shows that MixApart provides (i) up to 28% faster performance than the traditional ingest then-compute workflows used in enterprise IT analytics, and (ii) comparable performance to an ideal Hadoop setup without data ingest, at similar cluster sizes.
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
BibTeX
@inproceedings {180733,
author = {Madalin Mihailescu and Gokul Soundararajan and Cristiana Amza},
title = {{MixApart}: Decoupled Analytics for Shared Storage Systems},
booktitle = {11th USENIX Conference on File and Storage Technologies (FAST 13)},
year = {2013},
isbn = {978-1-931971-99-7},
address = {San Jose, CA},
pages = {133--146},
url = {https://www.usenix.org/conference/fast13/technical-sessions/presentation/mihailescu},
publisher = {USENIX Association},
month = feb,
}
connect with us