Trjtdtk

I'm trying to understand the paper : https://netman.aiops.org/wp-content/uploads/2018/05/PID5338621.pdf about Robust and Rapid Clustering of KPIs for Large-Scale Anomaly Detection.

Clustering is done using ROCKA algorithm.

Steps:

1.) Preprocessing is conducted on the raw KPI data to remove amplitude differences and standardize data.

2.) In baseline extraction step, we reduce noises, remove the extreme values (which are likely anomalies), and extract underlying shapes, referred to as baselines, of KPIs. It's done by applying moving average with a small sliding window.

3.) Clustering is then conducted on the baselines of sampled KPIs, with robustness against phase shifts and noises.

4.) Finally, we calculate the centroid of each cluster, then assign the unlabeled KPIs by their distances to these centroids.

I understand ROCKA mechanism.

Now, i'm trying to understand DONUT algorithm which is applied for "Anomaly Detection".

How it works is :

DONUT applies sliding windows over the KPI to get short series x and tries to recognize what normal patterns x follows. The indicator is then calculated by the difference between reconstructed normal patterns and x to show the severity of anomalies.
In practice, a threshold should be selected for each KPI. A data point with an indicator value larger than the threshold is regarded as an anomaly.

Now my question is :

IT seems like DONUT is not robust enough against time information related anomalies. Meaning that it works on a set of sliding windows and it ignores the relationship between windows. So the window becomes a very critical parameter here. So it might generate high false positives. What I'm understanding wrong here?

Please help and make me understand how DONUT will capture the relationship between sliding windows.

The code is available on github and the sliding windows are actually not overlap. Below code from file donut/utils.py, method minibatch_slices_iterator:

start = 0
stop1 = (length // batch_size) * batch_size
while start < stop1:
 yield slice(start, start + batch_size, 1)
 start += batch_size

I haven't inspect deeper but maybe the windows are not related, as you said.