Carving Perfect Layers out of Docker Images

Authors: 

Dimitris Skourtis, Lukas Rupprecht, Vasily Tarasov, and Nimrod Megiddo, IBM Research

Abstract: 

Container frameworks such as Docker have changed the way developers package, share, and deploy applications. Container images form a cornerstone of this new model. Images contain applications and their required runtime dependencies and can be easily versioned, stored, and shared via centralized registry services. The ease of creating and sharing images has led to a rapid adoption of container technology but registries are now faced with the task of efficiently storing and serving millions of images. While in theory identical parts of Docker images can be shared across images and stored only once as layers, in practice this provides limited benefits as layers are rarely fully identical.

In this paper, we argue that Docker image layers should be reorganized in order to maximize their overlap and, thereby, reduce storage and network consumption. This argument is based on the observation that many layers only differ in a small number of files but would otherwise be identical. We present a set of design challenges that need to be solved when realizing such a reorganization approach, e.g., how to optimally reorganize layers, how to deal with the scale of current registries, and how to integrate the approach into the container image lifecycle. We then present a mathematical formulation of the problem and evaluate it on a set of real Docker images. Our preliminary results provide storage savings of 2:3x, and indicate that promising network savings are possible.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {234845,
author = {Dimitris Skourtis and Lukas Rupprecht and Vasily Tarasov and Nimrod Megiddo},
title = {Carving Perfect Layers out of Docker Images},
booktitle = {11th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 19)},
year = {2019},
address = {Renton, WA},
url = {https://www.usenix.org/conference/hotcloud19/presentation/skourtis},
publisher = {{USENIX} Association},
month = jul,
}