sponsors
usenix conference policies
You are here
(Big)Data in a Virtualized World: Volume, Velocity, and Variety in Cloud Datacenters
Robert Birke, Mathias Bjoerkqvist, and Lydia Y. Chen, IBM Research Zurich Lab; Evgenia Smirni, College of William and Mary; Ton Engbersen, IBM Research Zurich Lab
Virtualization is the ubiquitous way to provide computation and storage services to datacenter end-users. Guaranteeing sufficient data storage and efficient data access is central to all datacenter operations, yet little is known of the effects of virtualization on storage workloads. In this study, we collect and analyze field data from production datacenters that operate within the private cloud paradigm, during a period of three years. The datacenters of our study consist of 8,000 physical boxes, hosting over 90,000 VMs, which in turn use over 22 PB of storage. Storage data is analyzed from the perspectives of volume, velocity, and variety of storage demands on virtual machines and of their dependency on other resources. In addition to the growth rate and churn rate of allocated and used storage volume, the trace data illustrates the impact of virtualization and consolidation on the velocity of IO reads and writes, including IO deduplication ratios and peak load analysis of co-located VMs. We focus on a variety of applications which are roughly classified as app, web, database, file, mail, and print, and correlate their storage and IO demands with CPU, memory, and network usage. This study provides critical storage workload characterization by showing usage trends and how application types create storage traffic in large datacenters.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Robert Birke and Mathias Bjoerkqvist and Lydia Y. Chen and Evgenia Smirni and Ton Engbersen},
title = {({Big)Data} in a Virtualized World: Volume, Velocity, and Variety in Cloud Datacenters},
booktitle = {12th USENIX Conference on File and Storage Technologies (FAST 14)},
year = {2014},
isbn = {ISBN 978-1-931971-08-9},
address = {Santa Clara, CA},
pages = {177--189},
url = {https://www.usenix.org/conference/fast14/technical-sessions/presentation/birke},
publisher = {USENIX Association},
month = feb
}
connect with us