UBCIS: Ultimate Benchmark for Container Image Scanning


Shay Berkovich, BlackBerry Limited; Jeffrey Kam, University of Waterloo; Glenn Wurster, BlackBerry Limited

Short Preliminary Work Paper


Containers are regularly used in modern cloud-native deployment practices. They support agile and continuous integration/continuous deployment (CI/CD) paradigms, isolating services. As containers become more ubiquitous, container security becomes crucial as well. Scanning container images for known vulnerabilities caused by vulnerable software is a critical security activity of the CI/CD process. Both commercial and open-source tools exist for container image scanning. Results from these scanners, however, are inconsistent. Inconsistent results make it hard for developers to choose the best solution for their environment. In this paper, we present the Ultimate Benchmark for Container Image Scanning (UBCIS), a benchmark for evaluating image scanners. UBCIS contains a classification of known vulnerabilities in common base container images, as well as a framework for running container vulnerability scanning tools. UBCIS makes it possible to evaluate scanners. We discuss intricacies of classifying vulnerabilities, presenting a process that can be used when determining the relevance of vulnerability. Finally, we provide recommendations for choosing the best scanner for a specific environment.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {256934,
author = {Shay Berkovich and Jeffrey Kam and Glenn Wurster},
title = {{UBCIS}: Ultimate Benchmark for Container Image Scanning},
booktitle = {13th {USENIX} Workshop on Cyber Security Experimentation and Test ({CSET} 20)},
year = {2020},
url = {https://www.usenix.org/conference/cset20/presentation/berkovich},
publisher = {{USENIX} Association},
month = aug,

Presentation Video