Understanding and Tackling the Hidden Memory Latency for Edge-based Heterogeneous Platform


Zhendong Wang, Zhen Wang, Cong Liu, and Yang Hu, The University of Texas at Dallas


With the burgeoning of autonomous driving, the edge-deployed integrated CPU/GPU (iGPU) platform gains significant attention from both academia and industries. NVIDIA issues a series of Jetson iGPU platforms that perform well in terms of computation capability, power consumption, and mobile size. However, these iGPU platforms typically contain very limited physical memory, which could be the bottleneck of these autonomous driving and edge computing applications. Although the introduction of the Unified Memory (UM) model in GPU programming can reduce the memory footprint, the programming legacy of the UM model initializes data on the CPU side by default as the conventional copy-and-execute model does, which causes significant latency of application execution. In this paper, we propose an enhanced unified memory management model (eUMM), which delivers a prefetch-enhanced data initialization method on the GPU side of the iGPU platform. We evaluate eUMM on the representative Jetson TX2 and Xavier AGX platforms and demonstrate that eUMM not only reduces the initialization latency significantly but also benefits the following kernel computation and the entire application execution latency.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {253362,
author = {Zhendong Wang and Zhen Wang and Cong Liu and Yang Hu},
title = {Understanding and Tackling the Hidden Memory Latency for Edge-based Heterogeneous Platform},
booktitle = {3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 20)},
year = {2020},
url = {https://www.usenix.org/conference/hotedge20/presentation/wang},
publisher = {USENIX Association},
month = jun

Presentation Video