BatchGen: An Architecture for Scalable and Efficient Batch Inference

Tairan Xu, Leyang Xue, and Zhan Lu, University of Edinburgh; Jinfu Deng and Hongyang Xiao, Tencent; Yinsicheng Jiang, Congjie He, Matej Sandor, Le Xu, and Luo Mai, University of Edinburgh