Lessons Learnt in Trying to Build New Storage Technologies

Dr. Antony Rowstron, Microsoft Research


A decade ago, cloud storage was dominated by tape, HDD, and flash. Today this is still true, but will it hold in 2034? For over the last decade at Microsoft Research Cambridge, we have been trying to build out new storage technologies for the cloud. This started with wondering how we could build out extremely low-cost HDD-based archival storage—and the challenges and frustrations of trying to do this—became the opportunity to begin to really think about how to build cloud-scale archival storage from the media up. We picked glass, and in Project Silica, we have been working on building out the technologies to make glass-based archival storage real. If we had known how hard Project Silica was going to be, we may never have started, but nearly a decade on, we now have a set of principles and thoughts on how to build out innovative novel storage systems from the media up. I will share some principles that we learnt along the way and also talk about how we are thinking of creating other future storage technologies.

Antony Rowstron, Microsoft Research

Ant is a Distinguished Engineer at Microsoft Research, Cambridge, UK, leading a team looking at future hardware technologies for the cloud across storage, networking, and computing, and most are focused on new optical-based technologies. The most well-known project is probably Project Silica, which is trying to use glass for long-term archival storage. Ant is a systems researcher at heart who has spent most of his career working at the intersection of Storage, Networking, and Distributed Systems, and he is best known as one of the original inventors of structured overlays or Distributed Hash Tables (DHTs) (called Pastry) and the first large-scale key-value storage system (PAST SOSP’01). In 2016, he was awarded the ACM SIGOPS Mark Weiser Award, and in 2021, the ACM EuroSys Lifetime Achievement Award. In September 2020, he was elected a Fellow of the Royal Academy of Engineering.

@conference {294822,
author = {Antony Rowstron},
title = {Lessons Learnt in Trying to Build New Storage Technologies},
year = {2024},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = feb

Presentation Video