Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • OSDI '12 Home
  • Organizers
  • Registration Information
  • Registration Discounts
  • At a Glance
  • Calendar
  • Technical Sessions
  • Workshops
  • Poster Sessions and Receptions
  • Birds-of-a-Feather Sessions
  • Sponsors
  • Activities
  • Hotel and Travel Information
  • Services
  • Students
  • Questions
  • Help Promote
  • For Participants
  • Call for Papers
  • Past Proceedings

sponsors

Diamond Sponsor
Diamond Sponsor
Gold Sponsor
Gold Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Bronze Sponsor
Bronze Sponsor
Bronze Sponsor
General Sponsor
General Sponsor
General Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » Spotting Code Optimizations in Data-Parallel Pipelines through PeriSCOPE
Tweet

connect with us

http://twitter.com/usenix
https://www.facebook.com/events/284007718333937/
http://www.linkedin.com/groups/USENIX-Association-49559/about
http://www.youtube.com/user/USENIXAssociation

Spotting Code Optimizations in Data-Parallel Pipelines through PeriSCOPE

Authors: 

Zhenyu Guo, Microsoft Research Asia; Xuepeng Fan, Microsoft Research Asia and Huazhong University of Science and Technology; Rishan Chen, Microsoft Research Asia and Peking University; Jiaxing Zhang, Hucheng Zhou, and Sean McDirmid, Microsoft Research Asia; Chang Liu, Microsoft Research Asia and Shanghai Jiao Tong University; Wei Lin and Jingren Zhou, Microsoft Bing; Lidong Zhou, Microsoft Research Asia

Abstract: 

To minimize the amount of data-shuffling I/O that occurs between the pipeline stages of a distributed dataparallel program, its procedural code must be optimized with full awareness of the pipeline that it executes in. Unfortunately, neither pipeline optimizers nor traditional compilers examine both the pipeline and procedural code of a data-parallel program so programmers must either hand-optimize their program across pipeline stages or live with poor performance. To resolve this tension between performance and programmability, this paper describes PeriSCOPE, which automatically optimizes a data-parallel program’s procedural code in the context of data flow that is reconstructed from the program’s pipeline topology. Such optimizations eliminate unnecessary code and data, perform early data filtering, and calculate small derived values (e.g., predicates) earlier in the pipeline, so that less data—sometimes much less data—is transferred between pipeline stages. We describe how PeriSCOPE is implemented and evaluate its effectiveness on real production jobs.

Zhenyu Guo, Microsoft Research Asia

Xuepeng Fan, Microsoft Research Asia and Huazhong University of Science and Technology

Rishan Chen, Microsoft Research Asia and Peking University

Jiaxing Zhang, Microsoft Research Asia

Hucheng Zhou, Microsoft Research Asia

Sean McDirmid, Microsoft Research Asia

Chang Liu, Microsoft Research Asia and Shanghai Jiao Tong University

Wei Lin, Microsoft Bing

Jingren Zhou, Microsoft Bing

Lidong Zhou, Microsoft Research Asia

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {180258,
author = {Zhenyu Guo and Xuepeng Fan and Rishan Chen and Jiaxing Zhang and Hucheng Zhou and Sean McDirmid and Chang Liu and Wei Lin and Jingren Zhou and Lidong Zhou},
title = {Spotting Code Optimizations in {Data-Parallel} Pipelines through {PeriSCOPE}},
booktitle = {10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 12)},
year = {2012},
isbn = {978-1-931971-96-6},
address = {Hollywood, CA},
pages = {121--133},
url = {https://www.usenix.org/conference/osdi12/technical-sessions/presentation/guo},
publisher = {USENIX Association},
month = oct,
}
Download
Guo PDF
View the slides

Presentation Video

Presentation Audio

MP3 Download OGG Download

Download Audio

  • Log in or    Register to post comments

Diamond Sponsors

Gold Sponsors

Silver Sponsors

Bronze Sponsors

General Sponsors

Media Sponsors & Industry Partners

© USENIX

  • Privacy Policy
  • Contact Us