Harmonic: Hardware-assisted RDMA Performance Isolation for Public Clouds


Jiaqi Lou, University of Illinois Urbana-Champaign; Xinhao Kong, Duke University; Jinghan Huang, University of Illinois Urbana-Champaign; Wei Bai, Microsoft; Nam Sung Kim, University of Illinois Urbana-Champaign; Danyang Zhuo, Duke University


Performance isolation is essential for sharing resources in multi-tenant public clouds. Compared with traditional kernel-based networking, RDMA presents unique challenges especially because RDMA NIC's complex microarchitecture resources are often hidden from users. Current RDMA isolation methods overlook these microarchitecture resources, leading to insufficient performance isolation. Consequently, a faulty/malicious tenant can exploit these microarchitecture resources to compromise well-behaved tenants' network performance. In this paper, we introduce the first microarchitecture-resource-aware RDMA performance isolation solution for public clouds, Harmonic. It consists of two key components designed to be conscious of the RDMA NIC's microarchitectural resources: (1) a programmable intelligent PCIe switch (prototyped with FPGA) and (2) an RDMA-friendly rate limiter. At runtime, these two components allow us to accurately monitor and modulate the RDMA NIC resource usage per tenant. We evaluate Harmonic with a state-of-the-art RDMA performance isolation test suite (Husky) and a popular in-memory database application (Redis). We demonstrate that Harmonic can not only successfully pass Husky but also provide Redis with 1.4× higher throughput than the best alternative isolation solution.

NSDI '24 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {295657,
author = {Jiaqi Lou and Xinhao Kong and Jinghan Huang and Wei Bai and Nam Sung Kim and Danyang Zhuo},
title = {Harmonic: Hardware-assisted {RDMA} Performance Isolation for Public Clouds},
booktitle = {21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)},
year = {2024},
isbn = {978-1-939133-39-7},
address = {Santa Clara, CA},
pages = {1479--1496},
url = {https://www.usenix.org/conference/nsdi24/presentation/lou},
publisher = {USENIX Association},
month = apr

Presentation Video