USENIX supports diversity, equity, and inclusion and condemns hate and discrimination.
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Submitted by admin on January 29, 2024 - 5:10 pm
| Title | MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs |
| Publication Type | Conference Paper |
| Year of Publication | 2024 |
| Authors | Jiang Z, Lin H, Zhong Y, Huang Q, Chen Y, Zhang Z, Peng Y, Li X, Xie C, Nong S, Jia Y, He S, Chen H, Bai Z, Hou Q, Yan S, Zhou D, Sheng Y, Jiang Z, Xu H, Wei H, Zhang Z, Nie P, Zou L, Zhao S, Xiang L, Liu Z, Li Z, Jia X, Ye J, Jin X, Liu X |
| Conference Name | 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24) |
| Date Published | 04/2024 |
| Publisher | USENIX Association |
| Conference Location | Santa Clara, CA |
| ISBN Number | 978-1-939133-39-7 |
| URL | https://www.usenix.org/conference/nsdi24/presentation/jiang-ziheng |