STYX: Exploiting SmartNIC Capability to Reduce Datacenter Memory Tax

Authors: 

Houxiang Ji, University of Illinois Urbana-Champaign; Mark Mansi, University of Wisconsin-Madison; Yan Sun, University of Illinois Urbana-Champaign; Yifan Yuan, Intel Labs; Jinghan Huang and Reese Kuper, University of Illinois Urbana-Champaign; Michael M. Swift, University of Wisconsin-Madison; Nam Sung Kim, University of Illinois Urbana-Champaign

Abstract: 

Memory optimization kernel features, such as memory deduplication, are designed to improve the overall efficiency of systems like datacenter servers, and they have proven to be effective. However, when invoked, these kernel features notably disrupt the execution of applications, intensively consuming the server CPU's cycles and polluting its caches. To minimize such disruption, we propose STYX, a framework for offloading the intensive operations of these kernel features to SmartNIC (SNIC). STYX first RDMA-copies the server's memory regions, on which these kernel features intend to operate, to an SNIC's memory region, exploiting SNIC's RDMA capability. Subsequently, leveraging SNIC's (underutilized) compute capability, STYX makes the SNIC CPU perform the intensive operations of these kernel features. Lastly, STYX RDMA-copies their results back to a server's memory region, based on which it performs the remaining operations of the kernel features. To demonstrate the efficacy of STYX, we re-implement two memory optimization kernel features in Linux: (1) memory deduplication (ksm) and (2) compressed cache for swap pages (zswap), using the STYX framework. We then show that a system with STYX provides a 55-89% decrease in 99th-percentile latency of co-running applications, compared to a system without STYX, while preserving the benefits of these kernel features.

USENIX ATC '23 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

BibTeX
@inproceedings {288792,
author = {Houxiang Ji and Mark Mansi and Yan Sun and Yifan Yuan and Jinghan Huang and Reese Kuper and Michael M. Swift and Nam Sung Kim},
title = {{STYX}: Exploiting {SmartNIC} Capability to Reduce Datacenter Memory Tax},
booktitle = {2023 USENIX Annual Technical Conference (USENIX ATC 23)},
year = {2023},
isbn = {978-1-939133-35-9},
address = {Boston, MA},
pages = {619--633},
url = {https://www.usenix.org/conference/atc23/presentation/ji},
publisher = {USENIX Association},
month = jul
}

Presentation Video