Vulcan: Automatic Query Planning for Live ML Analytics


Yiwen Zhang and Xumiao Zhang, University of Michigan; Ganesh Ananthanarayanan, Microsoft; Anand Iyer, Georgia Institute of Technology; Yuanchao Shu, Zhejiang University; Victor Bahl, Microsoft Corporation; Z. Morley Mao, University of Michigan and Google; Mosharaf Chowdhury, University of Michigan


Live ML analytics have gained increasing popularity with large-scale deployments due to recent evolution of ML technologies. To serve live ML queries, experts nowadays still need to perform manual query planning, which involves pipeline construction, query configuration, and pipeline placement across multiple edge tiers in a heterogeneous infrastructure. Finding the best query plan for a live ML query requires navigating a huge search space, calling for an efficient and systematic solution.

In this paper, we propose Vulcan, a system that automatically generates query plans for live ML queries to optimize their accuracy, latency, and resource consumption. Based on the user query and performance requirements, Vulcan determines the best pipeline, placement, and query configuration for the query with low profiling cost; it also performs fast online adaptation after query deployment. Vulcan outperforms state-of-the-art ML analytics systems by 4.1×-30.1× in terms of search cost while delivering up to 3.3× better query latency.

NSDI '24 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {295623,
author = {Yiwen Zhang and Xumiao Zhang and Ganesh Ananthanarayanan and Anand Iyer and Yuanchao Shu and Victor Bahl and Z. Morley Mao and Mosharaf Chowdhury},
title = {Vulcan: Automatic Query Planning for Live {ML} Analytics},
booktitle = {21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)},
year = {2024},
isbn = {978-1-939133-39-7},
address = {Santa Clara, CA},
pages = {1385--1402},
url = {},
publisher = {USENIX Association},
month = apr