Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • Home
  • Attend
    • Registration Information
    • Registration Discounts
    • Students and Grants
    • Venue, Hotel, and Travel
    • Oakland Dining Guide
  • Activities
    • Birds-of-a-Feather Sessions
    • Poster Session
  • Program
    • At a Glance
    • Technical Sessions
  • Participate
    • Call for Papers
    • Call for Posters
    • Instructions for Participants
  • Sponsorship
  • About
    • Symposium Organizers
    • Past Symposia
    • Questions?
    • Help Promote!
  • Home
  • Attend
    • Registration Information
    • Registration Discounts
    • Students and Grants
    • Venue, Hotel, and Travel
    • Oakland Dining Guide
  • Activities
  • Program
    • At a Glance
    • Technical Sessions
  • Participate
    • Call for Papers
    • Call for Posters
    • Instructions for Participants
  • Sponsorship
  • About
    • Symposium Organizers
    • Past Symposia
    • Questions?
    • Help Promote!

sponsors

Gold Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
General Sponsor
General Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

help promote

NSDI '15 button

Get more
Help Promote graphics!

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » Making Sense of Performance in Data Analytics Frameworks
Tweet

connect with us

https://twitter.com/usenix
https://www.facebook.com/usenixassociation
https://plus.google.com/108588319090208187909/posts
http://www.linkedin.com/groups/USENIX-Association-49559/about
http://www.youtube.com/user/USENIXAssociation

Making Sense of Performance in Data Analytics Frameworks

Authors: 

Kay Ousterhout, University of California, Berkeley; Ryan Rasti, University of California, Berkeley, International Computer Science Institute, and VMware; Sylvia Ratnasamy, University of California, Berkeley; Scott Shenker, University of California, Berkeley, and International Computer Science Institute; Byung-Gon Chun, Seoul National University

Abstract: 

There has been much research devoted to improving the performance of data analytics frameworks, but comparatively little effort has been spent systematically identifying the performance bottlenecks of these systems. In this paper, we develop blocked time analysis, a methodology for quantifying performance bottlenecks in distributed computation frameworks, and use it to analyze the Spark framework’s performance on two SQL benchmarks and a production workload. Contrary to our expectations, we find that (i) CPU (and not I/O) is often the bottleneck, (ii) improving network performance can improve job completion time by a median of at most 2%, and (iii) the causes of most stragglers can be identified.

Kay Ousterhout, University of California, Berkeley

Ryan Rasti, University of California, Berkeley

Sylvia Ratnasamy, University of California, Berkeley

Scott Shenker, University of California, Berkeley, and International Computer Science Institute

Byung-Gon Chun, Seoul National University

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {188988,
author = {Kay Ousterhout and Ryan Rasti and Sylvia Ratnasamy and Scott Shenker and Byung-Gon Chun},
title = {Making Sense of Performance in Data Analytics Frameworks},
booktitle = {12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15)},
year = {2015},
isbn = {978-1-931971-218},
address = {Oakland, CA},
pages = {293--307},
url = {https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/ousterhout},
publisher = {USENIX Association},
month = may
}
Download
Ousterhout PDF
View the slides

Presentation Video 

Presentation Audio

MP3 Download

Download Audio

  • Log in or    Register to post comments

Gold Sponsors

Silver Sponsors

Bronze Sponsors

General Sponsors

Media Sponsors & Industry Partners

© USENIX

  • Privacy Policy
  • Contact Us