Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • Home
  • Attend
    • Registration Information
    • Registration Discounts
    • Venue, Hotel, and Travel
    • Students and Grants
  • Activities
    • Birds-of-a-Feather Sessions
    • Poster Session and Happy Hour
  • Program
    • At a Glance
    • Technical Sessions
  • Sponsorship
  • Participate
    • Instructions for Participants
    • Call for Papers
    • Call for Posters
  • About
    • Organizers
    • Help Promote
    • Questions
    • Past Symposia
  • Home
  • Attend
  • Activities
  • Program
  • Sponsorship
  • Participate
  • About

sponsors

Silver Sponsor
Silver Sponsor
Silver Sponsor
Silver Sponsor
Bronze Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

help promote

NSDI '16 button

Get more
Help Promote graphics!

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home ยป StreamScope: Continuous Reliable Distributed Processing of Big Data Streams
Tweet

connect with us

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams

Authors: 

Wei Lin and Haochuan Fan, Microsoft; Zhengping Qian, Microsoft Research; Junwei Xu, Sen Yang, and Jingren Zhou, Microsoft; Lidong Zhou, Microsoft Research

This paper is part of the Operational Systems Track

Abstract: 

STREAMSCOPE (or STREAMS) is a reliable distributed stream computation engine that has been deployed in shared 20,000-server production clusters at Microsoft. STREAMS provides a continuous temporal stream model that allows users to express complex stream processing logic naturally and declaratively. STREAMS supports business-critical streaming applications that can process tens of billions (or tens of terabytes) of input events per day continuously with complex logic involving tens of temporal joins, aggregations, and sophisticated userdefined functions, while maintaining tens of terabytes in-memory computation states on thousands of machines.

STREAMS introduces two abstractions, rVertex and rStream, to manage the complexity in distributed stream computation systems. The abstractions allow efficient and flexible distributed execution and failure recovery, make it easy to reason about correctness even with failures, and facilitate the development, debugging, and deployment of complex multi-stage streaming applications.

Wei Lin, Microsoft

Zhengping Qian, Microsoft Research

Junwei Xu, Microsoft

Sen Yang, Microsoft

Jingren Zhou, Microsoft

Lidong Zhou, Microsoft Research

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {194956,
author = {Wei Lin and Zhengping Qian and Junwei Xu and Sen Yang and Jingren Zhou and Lidong Zhou},
title = {StreamScope: Continuous Reliable Distributed Processing of Big Data Streams},
booktitle = {13th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 16)},
year = {2016},
isbn = {978-1-931971-29-4},
address = {Santa Clara, CA},
pages = {439--453},
url = {https://www.usenix.org/conference/nsdi16/technical-sessions/presentation/lin},
publisher = {{USENIX} Association},
month = mar,
}
Download
Lin-Wei PDF
View the slides

Presentation Audio

MP3 Download

Download Audio

  • Log in or    Register to post comments

Silver Sponsors

Bronze Sponsors

Media Sponsors & Industry Partners

Open Access Publishing Partner

© USENIX

  • Privacy Policy
  • Conference Policies
  • Contact Us