Benchmarking in The Dark: On the Absence of Comprehensive Edge Datasets

Authors: 

Oleg Kolosov and Gala Yadgar, Technion - Israel Institute of Technology; Sumit Maheshwari and Emina Soljanin, Rutgers University

Abstract: 

Application workloads are used for system analysis, design, optimization, and evaluation. As systems and their applications evolve, new workloads are required to reflect their characteristics and bottlenecks. Edge-computing systems are a new model with unique characteristics, including heterogeneous and geodistributed components and complex failure patterns. Their workload characteristics are also unique, including unpredictable load and user mobility, combined with stringent latency and bandwidth requirements. Supporting these workloads would require resource management and allocation policies that take into account a rich set of attributes.

In this paper, we show that currently available datasets include only partial subsets of these attributes. Moreover, having been collected on existing systems, they do not reflect the unique characteristics of edge systems and applications. At the same time, current edge systems in early deployment stages are not ready for the collection of representative application workloads. We show how to bridge the gap between required and available datasets by workload composition: we combine attributes from several available datasets to create realistic representations of edge systems, their users, and their workloads.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {253376,
author = {Oleg Kolosov and Gala Yadgar and Sumit Maheshwari and Emina Soljanin},
title = {Benchmarking in The Dark: On the Absence of Comprehensive Edge Datasets},
booktitle = {3rd {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 20)},
year = {2020},
url = {https://www.usenix.org/conference/hotedge20/presentation/kolosov},
publisher = {{USENIX} Association},
month = jun,
}

Presentation Video