PromTCS: An Efficient Two-Dimensional Metadata Management and Tailored Hybrid Storage Architecture for Performance Monitoring Time Series

Time series databases (TSDBs) are crucial for performance monitoring in Internet applications and infrastructure. As the scale of time series grows, the storage overhead of metadata (labels) and data points (samples) increases significantly. In spite of prior efforts in hybrid storage to reduce storage costs, they still require large-capacity, expensive, high-bandwidth storage devices to deliver superior performance.

We propose PromTCS, which introduces two novel techniques: (1) Two-dimensional metadata management. We propose a two-dimensional metadata model and a suite of indexes to reduce metadata overhead while ensuring high performance. (2) Tailored hybrid storage architecture. We place hot data on DRAM and NVMe SSDs and cold data on HDDs, and present three optimizations: (i) a compaction-free time series storage engine for NVMe SSDs, TreeSeries, to reduce write amplification (WA) and optimize performance; (ii) hardware-aware write strategies, early migration and lazy flushing, to mitigate the adverse impact of bandwidth cross-tier bandwidth gaps; and (iii) a metadata-driven parallel write-ahead logging (WAL) on SSD, to reduce thread contention during writes and shorten crash recovery time. We implement a PromTCS prototype and evaluate it on synthetic and real-world datasets using both benchmark and production workloads. Compared with advanced TSDBs on pure NVMe SSDs, PromTCS achieves comparable performance at much lower deployment cost. Under the same hybrid storage, it delivers 3.2--5.9× higher insertion throughput and 38.1%--82.1% lower query latency than the advanced TSDBs.