Afternoon Tutorial 1: Designing Modern Software Caches

Monday, February 24, 2020 - 1:30 pm5:00 pm

Grand Ballroom D

Roy Friedman, Technion—Israel Institute of Technology; Benjamin Manes, Vector


Caching is one of the most basic and most effective mechanisms for boosting computing storage systems' performance.

In this tutorial we will survey recent developments in designing software cache libraries, while using Guava, Caffeine, and Ristretto as running examples. We will start by explaining some basic concepts in caching terminology and discuss why software caches are different than hardware designs and the challenges of designing an effective software cache library.

In particular, we will address issues like concurrency, memory management, timers handling, and cache management (admission and eviction). We will also address open research directions.

Specifically, the tutorial will include 4 sections:

  1. An introduction to software caching: basic concepts, differences from hardware caches, principle of locality, significance of workload, other challenges—a total of 30 minutes + 5 minutes for questions.
  2. General design concerns: effective concurrency, watermarks for evacuation, timers handling—60 minutes + 10 minute break at the end.
  3. Cache management: A survey of modern cache management schemes (replacement and evacuations) including ARC, CAR, Hyperbolic, W-TinyLFU, FRD, and Mini-Sim—60 minutes + 10 minutes break at the end.
  4. Open research directions: Including, e.g., entry cost (weight, latency), capacity allocations for multiple tenants, prediction of workload changes—30 minutes + 5 for questions.

Roy Friedman, Technion—Israel Institute of Technology

Roy Friedman is a professor in the Department of Computer Science at the Technion—Israel Institute of Technology. His research interests include Network Streaming Protocols, Caching, Replication, Fault-Tolerance, Dependability, High Availability, Consistency, and Mobile Computing. Roy Friedman serves as an associate editor for the IEEE TDSC and PC co-chair or OPODIS 2019. In the past, he served as PC co-chair for ACM DEBS 2015, ACM SYSTOR 2014 and Autonomics 2009 as well as vice-chair for IEEE ICDCS 2013+2006 and EuroPar 2008+2003, and fast abstract chair for IEEE DSN 2013. He has published more than 150 papers and holds 3 USA patents. Formerly, Roy Friedman was an academic specialist at INRIA (France) and a researcher at Cornell University (USA). He is a founder of PolyServe Inc. (acquired by HP) and holds a Ph.D. and a B.Sc. from the Technion.

Benjamin Manes, Vector

Ben Manes is CTO of Vector, a software company offering solutions for the trucking industry. Previously, when at Google, Ben co-authored Google Guava's Cache based on his successful ConcurrentLinkedHashMap library. In collaboration with Roy Friedman and his team at the Technion, Ben developed Caffeine cache. Like its predecessors, Caffeine has seen wide adoption in the Java ecosystem. He is currently advising the Ristretto team who aim to replicate Caffeine for the Go community. Ben holds an M.Sc. and two B.Sc. from the Illinois Institute of Technology.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

@conference {245982,
author = {Roy Friedman and Benjamin Manes},
title = {Afternoon Tutorial 1: Designing Modern Software Caches},
year = {2020},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = feb
Training Materials (Must be a registered conference attendee to download)