MDev-NVMe: A NVMe Storage Virtualization Solution with Mediated Pass-Through


Bo Peng, Shanghai Jiao Tong University, Intel; Haozhong Zhang, Intel; Jianguo Yao, Shanghai Jiao Tong University; Yaozu Dong, Intel; Yu Xu and Haibing Guan, Shanghai Jiao Tong University


The fast access to data and high parallel processing in high-performance computing instigates an urgent demand on the I/O improvement of the NVMe storage within datacenters. However, unsatisfactory performance of the former NVMe virtualization demonstrates that NVMe storage devices are often underutilized within cloud computing platforms. NVMe virtualization with high performance and device sharing has captured the attention of researchers. This paper introduces MDev-NVMe, a new virtualization implementation for NVMe storage device with: (1) full NVMe storage virtualization running native NVMe driver in guest, and (2) a mediated pass-through mechanism with an active polling mode which can achieve both high throughput, low latency performance and a good device scalability. This paper subsequently evaluates MDev-NVMe on Intel OPTANE and P3600 NVMe SSD by comparison with the mainstream virtualization mechanisms using application-level I/O benchmarks. With polling, MDev-NVMe can demonstrate a 142% improvement over native (interrupt-driven) throughput and over 2.5 times the Virtio throughput with only 70% native average latency and 31% Virtio average latency with a reliable scalability. Finally, the advantages of MDev-NVMe and the importance of polling are discussed, offering evidence that MDev-NVMe is a superior virtualization choice with high performance and promising levels of maintenance.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {216039,
author = {Bo Peng and Haozhong Zhang and Jianguo Yao and Yaozu Dong and Yu Xu and Haibing Guan},
title = {{MDev-NVMe}: A {NVMe} Storage Virtualization Solution with Mediated {Pass-Through}},
booktitle = {2018 USENIX Annual Technical Conference (USENIX ATC 18)},
year = {2018},
isbn = {978-1-939133-01-4},
address = {Boston, MA},
pages = {665--676},
url = {},
publisher = {USENIX Association},
month = jul

Presentation Audio