Rethinking Adaptability in Wide-Area Stream Processing Systems

Authors: 

Albert Jonathan, Abhishek Chandra, and Jon Weissman, University of Minnesota Twin Cities

Abstract: 

Adaptability is an important property of stream processing systems since the systems need to maintain low latency and high throughput execution of long-running queries. In a wide-area environment, dynamics are common not only because of the workload variability but also the nature of wide-area network (WAN) bandwidth that frequently changes. In this work, we study the adaptability property of stream processing systems designed for a wide-area environment. Specifically, we (1) discuss the challenges of reconfiguring query executions in a wide-area environment, (2) propose ideas how to adapt existing reconfiguration techniques used in centralized Cloud to a wide-area environment, and (3) discuss the trade-offs between them. A key finding is that the best adaptation technique to use depends on the network conditions, types of query, and optimization metrics.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {216845,
author = {Albert Jonathan and Abhishek Chandra and Jon Weissman},
title = {Rethinking Adaptability in Wide-Area Stream Processing Systems},
booktitle = {10th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 18)},
year = {2018},
address = {Boston, MA},
url = {https://www.usenix.org/conference/hotcloud18/presentation/jonathan},
publisher = {{USENIX} Association},
month = jul,
}