Smart Monitoring System for Anomaly Detection on Business Trends in Alibaba

Monday, May 22, 2017 - 1:40pm2:05pm

Zhaogang Wang, Alibaba Group

Abstract: 

Anomaly detection based on time series analysis approaches has been a focused theme in the field of monitoring, especially for business indicators monitoring. In Alibaba, hundreds of major KPIs need to be monitored in real time to detect the abnormal events and raise alarms. The effectiveness of the alarms is strictly evaluated by human operators. Therefore, we proposed an intelligent anomaly detection method to make the business monitoring system more scalable and easier to maintain.

There are two major problems in traditional anomaly detection approaches:

  • How to get accurate predictions based on seasonal time series data with complex noises and interferences.
  • How to determine the segmental thresholds dynamically, and learn from human feedbacks, e.g. the manual labelling data, to improve the accuracy of anomaly detection, and tolerate the error, event contradictions within the labelling data. 

 

For the first problem, as a tradeoff among computation performance, accuracy and robustness, we introduced a specific method to pre-process the data, and chose the STL method to do predictions on our business data.

For the second problem, we proposed a closed loop feedback method to determine the initial segment thresholds, and utilized the human labelling data to update the thresholds continuously.

Zhaogang Wang, Alibaba Group

Zhaogang Wang,
Senior technical specialist, Alibaba Group.
2009-2015, Senior Engineer, SRE team, Baidu.

Areas of interest:
Intelligent monitoring system,
Fault diagnosis

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@conference {202733,
author = {Zhaogang Wang},
title = {Smart Monitoring System for Anomaly Detection on Business Trends in Alibaba},
year = {2017},
publisher = {USENIX Association},
month = may,
}

Presentation Video 

Presentation Audio