MuCache: A General Framework for Caching in Microservice Graphs


Haoran Zhang, Konstantinos Kallas, and Spyros Pavlatos, Rajeev Alur, Sebastian Angel, and Vincent Liu, University of Pennsylvania


This paper introduces MuCache, a framework for extending arbitrary microservice applications with inter-service caches. MuCache significantly improves the performance of microservice graphs (commonly found in large applications like Uber or Twitter) by eliminating the need for one microservice to call another when the relevant state has not changed. MuCache is enabled by a novel non-blocking cache coherence and invalidation protocol for graph topologies that minimizes critical-path overhead. For this protocol, we prove a strong correctness result: any execution observed by the cache-enabled microservice application could have been observed by the original application without caches. Our evaluation on well-known microservice benchmarks shows that MuCache reduces the median request latency by up to 2.5×, and increases throughput by up to 60%.

NSDI '24 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {295493,
author = {Haoran Zhang and Konstantinos Kallas and Spyros Pavlatos and Rajeev Alur and Sebastian Angel and Vincent Liu},
title = {{MuCache}: A General Framework for Caching in Microservice Graphs},
booktitle = {21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)},
year = {2024},
isbn = {978-1-939133-39-7},
address = {Santa Clara, CA},
pages = {221--238},
url = {},
publisher = {USENIX Association},
month = apr

Presentation Video