Store-Edge RippleStream: Versatile Infrastructure for IoT Data Transfer


Madhumita Bharde, Annmary Justine K, Suparna Bhattacharya, and Dileep Deepa Shree, Hewlett Packard Enterprise


Powerful edge compute frameworks address the issue of latency for IoT data processing at the edge. However, continuous application layer WAN streaming to core for consolidated deep analysis and learning consumes excessive bandwidth and becomes the bottleneck for responsiveness at the core. A state-of-the-art storage or hyperconverged system, on the other hand, advertises compelling in-built features like WAN efficient data protection and delta replication, global unified management, space and bandwidth saving through inline data compression and deduplication. Traditional storage semantics and services, however, are built for data at rest while edge analytics prioritizes responsiveness by processing and moving streaming data at an application layer. In this paper, we propose to enable streaming of IoT data transparently through storage replication. Based on this foundation, we further present light-weight storage plugins to reduce IoT data transfer by detecting and translating semantic redundancies to a deduplication friendly form. Our early results demonstrate that (a) leveraging storage to take responsibility of streaming data in an application consistent way results in efficient data transfer (b) real-world IoT time-series datasets exhibit a high degree of similarity which can be detected to reduce data transfer from edge (c) video streams for autonomous cars, the transfer of which cannot be reduced enough using traditional video compression or storage deduplication techniques, have significant semantic redundancy. Collectively, advancing research in this direction paves the way to enhance the versatility of state-of-the-art infrastructure for optimized edge computing.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {216785,
author = {Madhumita Bharde and Annmary Justine K and Suparna Bhattacharya and Dileep Deepa Shree},
title = {{Store-Edge} {RippleStream}: Versatile Infrastructure for {IoT} Data Transfer},
booktitle = {USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18)},
year = {2018},
address = {Boston, MA},
url = {},
publisher = {USENIX Association},
month = jul,