Labeling the Invisible: A Scalable Framework for Labeling Fail-Slow Failures in Cloud Storage Systems

Fail-slow failures, characterized by prolonged performance degradation without complete system crashes, pose a significant threat to the reliability of large-scale cloud storage systems. However, the scarcity of high-quality labeled datasets limits the development of effective detection and diagnosis techniques, while manual labeling at cloud scale is labor-intensive and error-prone. In this paper, we present SlowSight, a novel tool designed to enable efficient and accurate labeling of fail-slow failures in cloud environments. SlowSight employs a multi-view framework that jointly models temporal deviations of individual components and behavioral divergence among peer components. To reduce spurious detections, a knowledge-driven filtering module removes anomaly candidates inconsistent with fail-slow meanings. Furthermore, SlowSight adopts pattern-centric aggregation to capture multi-metric degradation patterns and propagate consistent labels across similar anomaly instances, improving labeling accuracy and consistency. We deploy and evaluate SlowSight at Huawei Cloud on two production-scale disk fail-slow datasets, achieving F₁-scores of 0.851 and 0.903, and on a fail-slow dataset generated via failure injection with an F₁-score of 0.945, demonstrating its effectiveness and generalizability. To the best of our knowledge, SlowSight is the first framework designed for scalable fail-slow failure labeling in cloud storage systems. Additionally, we have publicly made the code available to facilitate further research.