HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism

Title{HetPipe}: Enabling Large {DNN} Training on (Whimpy) Heterogeneous {GPU} Clusters through Integration of Pipelined Model Parallelism and Data Parallelism
Publication TypeConference Paper
Year of Publication2020
AuthorsPark JH, Yun G, Yi CM, Nguyen NT, Lee S, Choi J, Noh SH, Choi Y-ri
Conference Name2020 USENIX Annual Technical Conference (USENIX ATC 20)
Date Published07/2020
PublisherUSENIX Association
ISBN Number978-1-939133-14-4
URLhttps://www.usenix.org/conference/atc20/presentation/park