Distributing Deep Neural Networks with Containerized Partitions at the Edge


Li Zhou, The Ohio State University; Hao Wen, University of Minnesota, Twin Cities; Radu Teodorescu, The Ohio State University; David H.C. Du, University of Minnesota, Twin Cities


Deploying machine learning on edge devices is becoming increasingly important, driven by new applications such as smart homes, smart cities, and autonomous vehicles. Unfortunately, it is challenging to deploy deep neural networks (DNNs) on resource-constrained devices. These workloads are computationally intensive and often require cloud-like resources. Prior solutions attempted to address these challenges by either sacrificing accuracy or by relying on cloud resources for assistance.

In this paper, we propose a containerized partition-based runtime adaptive convolutional neural network (CNN) acceleration framework for Internet of Things (IoT) environments. The framework leverages spatial partitioning techniques through convolution layer fusion to dynamically select the optimal partition according to the availability of computational resources and network conditions. By containerizing each partition, we simplify the model update and deployment with Docker and Kubernetes to efficiently handle runtime resource management and scheduling of containers.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {234801,
author = {Li Zhou and Hao Wen and Radu Teodorescu and David H.C. Du},
title = {Distributing Deep Neural Networks with Containerized Partitions at the Edge},
booktitle = {2nd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 19)},
year = {2019},
address = {Renton, WA},
url = {https://www.usenix.org/conference/hotedge19/presentation/zhou},
publisher = {USENIX Association},
month = jul