Morning Tutorial 1: Persistent Memory Programming on Conventional Hardware

Monday, February 24, 2020 - 9:00 am12:30 pm

Grand Ballroom D

Terence Kelly


Often overlooked in the excitement surrounding novel byte-addressable non-volatile memory (NVM) hardware is a purely *software* abstraction of persistent memory that can be implemented on conventional hardware, without NVM. Persistent memory invites the same programming style, with similar advantages: Applications become dramatically simpler because they store persistent data in memory and manipulate it with CPU instructions, eliminating large, complex, opaque external persistent stores such as relational databases and key-value stores. Furthermore, persistent application data exist only in the in-memory format, eliminating the need for translation to/from a different persistent format. Finally, programmers think only in the paradigm of imperative algorithms for in-memory data structures; there's no need to mentally context switch to a different paradigm, e.g., declarative SQL manipulating relational data.

We begin with a thorough review of the one commonality underlying all persistent memory programming platforms and frameworks for C/C++ from both industry and academia: the art of laying out application data in file-backed memory mappings. We master a handful of essential techniques, idioms, and tricks that enable developers to confidently write efficient and correct software in the persistent memory style. We learn the tradeoffs between relocatable vs. fixed-address persistent data structures. We flag the pitfalls surrounding multi-threaded persistent memory programming and learn to circumvent them. We survey crash-tolerance mechanisms for the persistent memory style of programming, including those that require special NVM hardware (NVDIMMs or Optane) and also those that admit implementation on conventional hardware (volatile DRAM and block storage). We show how to write from scratch, in two dozen lines of straightforward C code, an efficient and portable crash-tolerance mechanism that works on both conventional hardware and NVM. We learn from extensive industry experience with successfully retrofitting crash-tolerance onto complex legacy production software that was not originally designed to survive crashes.

The tutorial will consider working C/C++ programs, showing how to evolve non-persistent data structures into persistent data structures, and then showing how to prevent crashes from corrupting them. At each stage in the progression, code deltas illustrate the steps that the developer must take and the factors that she must consider. Example code and explanatory documentation will be provided. Most importantly, students will learn how to write such software themselves.

Students are encouraged but not required to prepare by reading the following article:

"Persistent Memory Programming on Conventional Hardware"
ACM Queue magazine, Vol. 17, No. 4, July/August 2019

Terence Kelly

Terence Kelly studied Computer Science at Princeton and the University of Michigan. He was a researcher at HP Labs for 14 years, the last five of which devoted to software support for non-volatile memory. His research publications on persistent memory programming have appeared in ASPLOS, FAST, DISC, USENIX ATC, and EuroSys; his research publications on multi-threaded programming have appeared in OSDI and POPL. Kelly's persistent memory research led to several tech transfers, notably to HP Indigo printing presses and to the HP Advanced File System. His practitioner-oriented articles on persistent memory programming have appeared in ACM Queue and USENIX ;login:. Kelly now teaches and evangelizes the persistent memory style of programming. He has released three software packages related to persistent memory programming. Kelly's publications and patents are listed at

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

@conference {245972,
author = {Terence Kelly},
title = {Morning Tutorial 1: Persistent Memory Programming on Conventional Hardware},
year = {2020},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = feb
Training Materials (Must be a registered conference attendee to download)
Code Bundle (Must be a registered conference attendee to download)