Decentralized Application-Level Adaptive Scheduling for Multi-Instance DNNs on Open Mobile Devices

Authors: 

Hsin-Hsuan Sung and Jou-An Chen, Department of Computer Science, North Carolina State University; Wei Niu, Jiexiong Guan, and Bin Ren, Department of Computer Science, William & Mary; Xipeng Shen, Department of Computer Science, North Carolina State University

Abstract: 

As more apps embrace AI, it is becoming increasingly common that multiple Deep Neural Networks (DNN)-powered apps may run at the same time on a mobile device. This paper explores scheduling in such multi-instance DNN scenarios, on general open mobile systems (e.g., common smartphones and tablets). Unlike closed systems (e.g., autonomous driving systems) where the set of co-run apps are known beforehand, the user of an open mobile system may install or uninstall arbitrary apps at any time, and a centralized solution is subject to adoption barriers. This work proposes the first-known decentralized application-level scheduling mechanism to address the problem. By leveraging the adaptivity of Deep Reinforcement Learning, the solution guarantees co-run apps converge to a Nash equilibrium point, yielding a good balance of gains among the apps. The solution moreover automatically adapts to the running environment and the underlying OS and hardware. Experiments show that the solution consistently produces significant speedups and energy savings across DNN workloads, hardware configurations, and running scenarios.

USENIX ATC '23 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

BibTeX
@inproceedings {288806,
author = {Hsin-Hsuan Sung and Jou-An Chen and Wei Niu and Jiexiong Guan and Bin Ren and Xipeng Shen},
title = {Decentralized {Application-Level} Adaptive Scheduling for {Multi-Instance} {DNNs} on Open Mobile Devices},
booktitle = {2023 USENIX Annual Technical Conference (USENIX ATC 23)},
year = {2023},
isbn = {978-1-939133-35-9},
address = {Boston, MA},
pages = {865--877},
url = {https://www.usenix.org/conference/atc23/presentation/sung},
publisher = {USENIX Association},
month = jul
}

Presentation Video