SmartNIC-Enabled Live Migration for Storage-Optimized VMs with PYROCUMULUS

Jiechen Zhao, University of Toronto and Microsoft Research Asia; Ran Shu, Lei Qu, Ziyue Yang, and Rui Ma, Microsoft Research Asia; Derek Chiou, Microsoft and UT Austin; Natalie Enright Jerger, University of Toronto; Peng Cheng and Yongqiang Xiong, Microsoft Research Asia

Cloud providers offer storage-optimized VMs equipped with locally attached storage to meet the high performance requirements of cloud users. Live migration (LM) is crucial for such VMs to improve availability and manageability. However, providers do not enable LM for storage-optimized VMs. Host-managed LM suffers from high resource overheads and varied user performance, while offloading LM to SoC SmartNICs or disks cannot reliably accomplish LM in a reasonable time. The fundamental challenges are (1) consistency, demanding a high resource budget, and (2) network contention, preventing LM from converging. We propose Pyrocumulus, an LM approach on FPGA SmartNICs, enabling SLA-aware, fast, and low-overhead LM for storage-optimized VMs. We exploit hardware customizability and efficient network accessibility of the FPGA SmartNIC with LM protocol, architecture, and algorithm designs. Results from our FPGA SmartNIC prototype show that Pyrocumulus reduces user latency variances during LM up to 12.4×, lowers LM time by up to 19.6×, and saves cost up to 3×, while only taking 0.9%/3.8% compute/memory overhead of a mid-end FPGA SmartNIC.

NSDI '26 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {316758,
author = {Jiechen Zhao and Ran Shu and Lei Qu and Ziyue Yang and Rui Ma and Derek Chiou and Natalie Enright Jerger and Peng Cheng and Yongqiang Xiong},
title = {{SmartNIC-Enabled} Live Migration for {Storage-Optimized} {VMs} with {PYROCUMULUS}},
booktitle = {23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26)},
year = {2026},
isbn = {978-1-939133-54-0},
address = {Renton, WA},
pages = {1995--2010},
url = {https://www.usenix.org/conference/nsdi26/presentation/zhao-jiechen},
publisher = {USENIX Association},
month = may
}

Presentation Video