Mainstream: Dynamic Stem-Sharing for Multi-Tenant Video Processing

Authors: 

Angela H. Jiang, Daniel L.-K. Wong, Christopher Canel, Lilia Tang, and Ishan Misra, Carnegie Mellon University; Michael Kaminsky, Michael A. Kozuch, and Padmanabhan Pillai, Intel Labs; David G. Andersen and Gregory R. Ganger, Carnegie Mellon University

Abstract: 

Mainstream is a new video analysis system that jointly adapts concurrent applications sharing fixed edge resources to maximize aggregate result quality. Mainstream exploits partial-DNN (deep neural network) compute sharing among applications trained through transfer learning from a common base DNN model, decreasing aggregate per-frame compute time. Based on the available resources and mix of applications running on an edge node, Mainstream automatically determines at deployment time the right trade-off between using more specialized DNNs to improve per-frame accuracy, and keeping more of the unspecialized base model to increase sharing and process more frames per second. Experiments with several datasets and event detection tasks on an edge node confirm that Mainstream improves mean event detection F1-scores by up to 47% relative to a static approach of retraining only the last DNN layer and sharing all others (“Max-Sharing”) and by 87X relative to the common approach of using fully independent per-application DNNs (“No-Sharing”).

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {215993,
author = {Angela H. Jiang and Daniel L.-K. Wong and Christopher Canel and Lilia Tang and Ishan Misra and Michael Kaminsky and Michael A. Kozuch and Padmanabhan Pillai and David G. Andersen and Gregory R. Ganger},
title = {Mainstream: Dynamic {Stem-Sharing} for {Multi-Tenant} Video Processing},
booktitle = {2018 USENIX Annual Technical Conference (USENIX ATC 18)},
year = {2018},
isbn = {978-1-931971-44-7},
address = {Boston, MA},
pages = {29--42},
url = {https://www.usenix.org/conference/atc18/presentation/jiang},
publisher = {USENIX Association},
month = jul
}

Presentation Audio