Adaptively Compressing IoT Data on the Resource-constrained Edge


Tao Lu, Marvell Technology Group; Wen Xia and Xiangyu Zou, Harbin Institute of Technology, Shenzhen, China; Qianbin Xia, Marvell Technology Group


Big IoT data needs to be frequently moved between edge and cloud for efficient analysis and storage. Data movement is costly in low-bandwidth wide area network environments. Data compression can dramatically reduce data size to mitigate the bandwidth bottleneck. However, compression is compute-intensive and compression throughput can be limited by available CPU resources. The impact of available computation capability of the resource-constrained edge on the edge-to-cloud data transfer rate is apparent. Our study reveals compressors, including gzip, bzip2, lzma, and zstd, perform very differently under various resource-constrained conditions. This motivates us to propose models for the best compressor selection under CPU, network, and storage resource limitation conditions on the edge. We implement ZipMate, a middleware that enables resource-aware and adaptive compression policy based on the model. Our evaluation shows that adaptive policies consistently outperform unitary or random compressor selection policies.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {253368,
author = {Tao Lu and Wen Xia and Xiangyu Zou and Qianbin Xia},
title = {Adaptively Compressing {IoT} Data on the Resource-constrained Edge},
booktitle = {3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 20)},
year = {2020},
url = {},
publisher = {USENIX Association},
month = jun

Presentation Video