Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • Home
  • Attend
    • Registration Information
    • Registration Discounts
    • Venue, Hotel, and Travel
    • Students and Grants
    • Co-located Events
  • Program
    • Workshop Program
  • Sponsorship
  • Participate
    • Instructions for Authors and Speakers
    • Call for Papers
  • About
    • Workshop Organizers
    • Help Promote
    • Questions
    • Past Workshops
  • Home
  • Attend
  • Program
  • Sponsorship
  • Participate
  • About

sponsors

Gold Sponsor
Gold Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

help promote

HotStorage '16 button

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home ยป Quartet: Harmonizing Task Scheduling and Caching for Cluster Computing
Tweet

connect with us

Quartet: Harmonizing Task Scheduling and Caching for Cluster Computing

Authors: 

Francis Deslauriers, Peter McCormick, George Amvrosiadis, Ashvin Goel, and Angela Demke Brown, University of Toronto

Abstract: 

Cluster computing frameworks such as Apache Hadoop and Apache Spark are commonly used to analyze large data sets. The analysis often involves running multiple, similar queries on the same data sets. This data reuse should improve query performance, but we find that these frameworks schedule query tasks independently of each other and are thus unable to exploit the data sharing across these tasks. We present Quartet, a system that leverages information on cached data to schedule together tasks that share data. Our preliminary results are promising, showing that Quartet can increase the cache hit rate of Hadoop and Spark jobs by up to 54%. Our results suggest a shift in the way we think about job and task scheduling today, as Quartet is expected to perform better as more jobs are dispatched on the same data.

Francis Deslauriers, University of Toronto

Peter McCormick, University of Toronto

George Amvrosiadis, University of Toronto

Ashvin Goel, University of Toronto

Angela Demke Brown, University of Toronto

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {196370,
author = {Francis Deslauriers and Peter McCormick and George Amvrosiadis and Ashvin Goel and Angela Demke Brown},
title = {Quartet: Harmonizing Task Scheduling and Caching for Cluster Computing},
booktitle = {8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 16)},
year = {2016},
address = {Denver, CO},
url = {https://www.usenix.org/conference/hotstorage16/workshop-program/presentation/deslauriers},
publisher = {USENIX Association},
month = jun,
}
Download
Deslauriers PDF
View the slides
  • Log in or    Register to post comments

Gold Sponsors

Silver Sponsors

Bronze Sponsors

Media Sponsors & Industry Partners

© USENIX

  • Privacy Policy
  • Contact Us