Data Storage at Librato

Librato runs a multi-tenant time-series data storage, analysis, and alerting platform as a service. In this talk we describe our internal data processing and storage infrastructure, which relies heavily on Apache’s Storm and Cassandra projects to process around 250k writes per second and store roughly 10 billion data samples. We give infrastructure details, recounting our initial design decisions as well as various scaling challenges that have forced us to refactor our storage designs. Finally we relate the system and performance metrics that we’ve found useful in monitoring our storage infrastructure.

As the developer evangelist for Librato, Dave Josephsen hacks on tools, writes about statistics, systems monitoring, alerting, metrics collection and visualization, and generally does anything he can to help engineers and developers close the feedback loop in their systems. He writes the "iVoyer" column on systems monitoring in ;login: magazine.