Buffer-based End-to-end Request Event Monitoring in the Cloud


Kaihui Gao, Tsinghua University and Alibaba Group; Chen Sun, Alibaba Group; Shuai Wang and Dan Li, Tsinghua University; Yu Zhou, Hongqiang Harry Liu, Lingjun Zhu, and Ming Zhang, Alibaba Group


Request latency is a crucial concern for modern cloud providers. Due to various causes in hosts and networks, requests can suffer from request latency anomalies (RLAs), which may violate the Service-Level Agreement for tenants. However, existing performance monitoring tools have incomplete coverage and inconsistent semantics for monitoring requests, resulting in the difficulty to accurately diagnose RLAs.

This paper presents BufScope, a high-coverage request event monitoring system, which aims to capture most RLA-related events with consistent request-level semantics in the end-to-end datapath of request. BufScope models the datapath of request as a buffer chain and defines RLA-related events based on different properties of buffers, so as to end-to-end monitor the root causes of RLA. To achieve consistent semantics for captured events, BufScope designs a concise request-level semantic injection mechanism to make events captured in networks have the victim requests' ID, and offloads the realization to SmartNICs for low overhead. We have implemented BufScope on commodity SmartNICs and programmable switches. Evaluation results show that BufScope can diagnose 95% RLAs with <0.07% network bandwidth overhead and <1% application throughput decline.

NSDI '22 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {278298,
author = {Kaihui Gao and Chen Sun and Shuai Wang and Dan Li and Yu Zhou and Hongqiang Harry Liu and Lingjun Zhu and Ming Zhang},
title = {Buffer-based End-to-end Request Event Monitoring in the Cloud},
booktitle = {19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22)},
year = {2022},
isbn = {978-1-939133-27-4},
address = {Renton, WA},
pages = {829--843},
url = {https://www.usenix.org/conference/nsdi22/presentation/gao-kaihui},
publisher = {USENIX Association},
month = apr

Presentation Video