A Tale of One Billion Time Series

Thursday, June 07, 2018 - 3:15 pm4:10 pm

Ruiyao Yao, Baidu


Monitoring system is vital for service stability and availability. To support Baidu’s massive services and machines, the metrics being collected has grown to 1 billion. These metrics must be stored in a reliable and efficient database, which must support real-time insertion of new data and various queries, ranging from aggregation, alerting, to reports and visualization, with diversified time granularities.

Our time-series database (TSDB) consists of three layers, a memory database based on Redis stores hot data, a HBase stores warm data, and a HDFS stores cold data. To achieve efficient insertion, we extensively apply batch and asynchronous methods to the write path, in addition to HBase’s ability of high throughput writing. To improve reading performance, we design specialized data model and embed multi-layer down-sampling mechanism into HBase. The memory database incorporates compression techniques to serve real-time, frequent, and small queries, while preserving memory consumption at a reasonable level. All the data are backed up in a separate HDFS to support offline analysis.

In this talk, we will explore the challenges of large scale time-series processing, and introduce our practice of building TSDB. We will also share some successful experiences, such as retention policy, and trade-offs between cost and performance.

Ruiyao Yao, Baidu

Currently a Senior Engineer at Baidu, Beijing. Responsible for the design and development of data platform of AIOps.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@conference {214905,
author = {Ruiyao Yao},
title = {A Tale of One Billion Time Series},
year = {2018},
publisher = {USENIX Association},
month = jun

Presentation Video 

Presentation Audio