Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art

In plain words

Anomaly detection on a time series means flagging moments where a signal stops behaving the way it usually does. Many methods exist, from old statistical forecasting models to classic machine-learning clustering and to neural networks. This paper asks which family works best. It picks 20 methods from the three families and runs them all on the same labelled series.

The main result, as the authors report it: simple statistical models score best on most of the test data and run fastest. Neural networks do better only on the one dataset whose anomalies look normal in isolation and become anomalous only in context. The neural models are also slow to run in some cases. Read the numbers as one team's benchmark, not a general verdict.

TL;DR

• 20 univariate detectors, grouped as statistical, classical ML and neural, are each scored by AUC on five labelled datasets §5.1 tab. 6.
• On the four Yahoo sets the statistical AR and MA models score highest, and neural models score lowest §6.1.
• On the NYC taxi set the neural models do better, but K-Means has the highest AUC in tab. 6 and the text does not discuss it §6.1.
• Runtime varies widely: AR and MA are the fastest, and LSTM and GRU are the slowest §6.2 tab. 7.
• Author-reported, one run per setting, no official code found.

Problem

Many detectors need a model of normal behaviour, and a forecast-residual detector is only as good as its forecaster §2.5. The authors note that no earlier study compared statistical, classical ML and deep methods on univariate series under one protocol §4.1. They also argue that anomalies in time series are often contextual or collective, not single outliers §2.5 §6.4.

Method

This is an evaluation study, not a new detector. Its method is the protocol below, applied to 20 existing detectors §5.1 §5.2.

Anomaly definition. An anomaly is a point or a run of points that departs strongly from the bulk of the data, and anomalies form a very small share of the series §2.1 §2.5. The paper studies point, collective and contextual anomalies §2.2.

Forecast-residual scoring. For forecast-based detectors, a model predicts each value from the w values before it. The error between prediction and observation is the anomaly score, and a point is flagged when the score exceeds a threshold δ §2.5 §3.3. The threshold is chosen on training data, but the paper does not give one rule for every method §3.3.

Statistical detectors §3.1.

1. AR(p): a linear autoregression on p past values. Coefficients come from least squares. The residual is the score. The series must be stationary §3.1.
2. MA(q): the current value is a linear combination of q past errors. There is no closed-form fit, so an iterative estimator is used §3.1.
3. ARMA(p,q): combines AR and MA. Choosing p and q is the main difficulty §3.1.
4. ARIMA: ARMA applied to a d-times differenced series, to remove trend §3.1.
5. SES: exponentially weighted average, with smoothing α in [0,1] §3.1.
6. Double and triple exponential smoothing: adds trend (β) and seasonal (γ) terms §3.1.
7. PCI: a weighted average of neighbouring points, with a bound set from a Student t-factor, the residual standard deviation s and window k. A point outside the bound is an anomaly §3.1.

Classical ML detectors §3.2.

1. K-Means on sliding subsequences (STSC): the score is the distance to the nearest centroid. A subsequence is an anomaly if the distance exceeds δ. The authors cite prior work that subsequence clustering may be uninformative §3.2.
2. DBSCAN on values: points are core, border or anomaly. A point that is neither core nor border is an anomaly. The output is a label, not a score §3.2.
3. LOF on windows: the test window is scored against the training windows, and the window is an anomaly when its LOF exceeds δ §3.2.
4. Isolation Forest on windows: anomalies have short average path lengths. The score is 2 to the power of minus the mean path length over c(w) §3.2.
5. One-class SVM: each window is mapped to a 2-D point [x_t, x_{t-1}] (for t>1) and an RBF one-class SVM is fitted on training data §3.2.
6. XGBoost as a one-step forecaster. The threshold is a λ-percentile of the training errors. The paper does not state λ §3.2 tab. 4.

Neural detectors §3.3. All use a sliding window of past values as input, except as noted, and are scored by forecast residual.

1. MLP: fully connected, one step ahead, residual threshold §3.3 fig. 8.
2. CNN: 1-D convolution and max-pooling blocks, then a dense layer §3.3 fig. 9.
3. CNN with batch normalization §3.3.
4. CNN with a residual block: output is ψ(φ(x)+x), with φ a convolution block §3.3 fig. 10.
5. WaveNet-style network with dilated convolutions, dilation doubling per layer §3.3 fig. 11 fig. 12.
6. LSTM: a stateful two-layer network. Residual vectors over a prediction window are fitted to a multivariate Gaussian by maximum likelihood, and a point is flagged from its likelihood against δ §3.3 fig. 13 fig. 14. The text describes the LSTM input as one timestamp per step §3.3.
7. GRU: the LSTM model with GRU cells §3.3 fig. 15.
8. Autoencoder: encoder and decoder trained to reconstruct normal windows; the reconstruction error is the score §3.3 fig. 16.

Preprocessing §4.3 §5.1.

• Each dataset is standardized, (x − μ)/σ. The paper does not say whether μ and σ are per series or per dataset §4.3.1.
• Min-max normalization is avoided because it is sensitive to outliers §4.3.1.
• Detrending and deseasonalizing are discussed, but the paper reports no results with them §4.3.2 §6.

Protocol §5.1 §5.2.

• Each series is split 30% train, 70% test. Test series with no anomaly are dropped §5.1.
• Deep models use 10% of training data as validation §5.1. Table 2 instead says 0.1 of the test set tab. 2.
• Forecasting models are tuned on the naive-model metric NMM, the MSE of the model divided by the MSE of a predict-previous-value baseline. The authors aim to keep NMM below 1 §5.2.
• Score: AUC over the test points §4.4. Runtime is total training plus prediction time per dataset §4.4.

# Evaluation of one detector on one series  [§5.1] [§4.4]
# Input:  series (N,), labels (N,) in {0, 1}
# Output: AUC on the test split, or None

def evaluate(series, labels, detector):
    # 1. Standardize the series  [§4.3.1]  (scope not stated)
    series = (series - mean(series)) / std(series)
    # 2. Split 30% train, 70% test  [§5.1]
    train, test = split(series, train_ratio=0.3)
    train_labels, test_labels = split(labels, train_ratio=0.3)
    # 3. Skip series with no anomaly in the test part  [§5.1]
    if not any(test_labels):
        return None
    # 4. Fit on train only; deep models use 10% of train as validation  [§5.1]
    model = fit(detector, train)
    # 5. Score every test point  [§2.5] [§3.1]-[§3.3]
    scores = score(model, test)
    # 6. AUC over the test points  [§4.4]
    return roc_auc(test_labels, scores)

# Forecast-residual scoring  [§2.5] [§3.3]
# Input:  series (N,), window w, threshold delta
# Output: flag per point

def flag_anomalies(series, model, w, delta):
    # 1. Build (window, next value) pairs from the series  [§2.5]
    inputs, targets = sliding_windows(series, w)  # (N-w, w), (N-w,)
    # 2. Predict the next value from each window  [§3.3]
    predictions = model(inputs)                   # (N-w,)
    # 3. Residual as printed in the paper, not absolute  [§3.3]
    residuals = predictions - targets
    # 4. Flag points whose residual exceeds the threshold  [§3.3]
    return residuals > delta

Hyperparameters

Hardware: two Intel Xeon CPUs at 2.30 GHz, about 12.4 GB RAM, one Tesla K80 GPU with 12 GB. Every computation ran in a single process and thread tab. 1. The paper does not say whether the deep models trained on the GPU when timed.

Software: statsmodels for statistical models, scikit-learn for classical ML, TensorFlow and Keras for deep models §5.2.

Sliding window and splits tab. 2:

Statistical models tab. 3:

Classical ML tab. 4:

Deep models tab. 5:

Cited as tab. 5. Prediction horizon and LSTM reset rules are not stated.

Results

Author-reported. Setup: 20 detectors, five datasets, 30/70 split, AUC on the test points §5.1 §4.4. Datasets §4.2:

• UD1: 67 real Yahoo traffic series, about 1400 hourly steps, 1.9% anomalous.
• UD2: 100 synthetic Yahoo series, about 1421 steps, 0.3% anomalous, point anomalies inserted at random.
• UD3: 100 synthetic Yahoo series with seasonality, about 1680 steps, 0.3% anomalous.
• UD4: 100 synthetic Yahoo series with changepoints, about 1680 steps, 0.5% anomalous.
• NYCT: one NYC taxi-demand series, 10320 half-hour steps, five collective anomalies (marathon, Thanksgiving, Christmas, New Year, a snowstorm).

AUC per dataset tab. 6. The table in the paper is split in two; here it is combined. Values are copied as printed.

Author claims §6.1 §6.4 fig. 17 fig. 18:

• Statistical models have the best average AUC over UD1 to UD4, and the authors say four of the five best are statistical. The average plot fig. 17 is a figure, so its values are not reproduced here.
• Neural models score lowest on the Yahoo sets. The NYCT result is the reverse §6.1.
• The authors attribute the statistical models' NYCT failure to contextual anomalies, where the anomalous values look like normal ones §6.1.

Runtime, total on UD1-UD4 (367 series) and per series tab. 7:

Training and inference on NYCT, in seconds tab. 8:

Author claims §6.2: AR and MA are fastest. Some deep models infer fast, but LSTM and GRU infer slowly, more than most classical methods §6.2.

Paper vs code

No official code. The paper gives no code link and no project page. Searches of GitHub for the author names and for title words returned no repository tied to the authors.

Pitfalls

• Threshold rule is unstated for most methods. The paper says the threshold comes from training data §3.3 but gives no rule for MLP, CNN, WaveNet, LSTM or GRU. AUC does not need a threshold, but any deployed use does.
• Scores are unclear for binary-output methods. DBSCAN outputs a label §3.2 and the paper does not say how it obtained a continuous score for AUC. The same question applies to K-Means in its binary form.
• Sign and direction of the residual rule. Forecast detectors flag when prediction - target > δ §3.3. The autoencoder and the other residual rules use a norm e = ‖x − x̂‖ §3.3. Using the signed rule would miss the opposite-direction errors.
• LSTM likelihood rule reads inverted. The paper says a point is an anomaly when P(e) > δ §3.3. An anomaly should have low likelihood. Read the rule as a low-likelihood test, and treat the printed form as a likely typo.
• PCI formula. The weighted-average denominator is written as an unweighted sum of the X values §3.1. Normalize by the sum of weights unless the paper says otherwise, and note that the one-sided and two-sided windows differ in the factor k versus 2k §3.1.
• PCI parameters are outside the recommended range. The paper recommends k between 3 and 15 and α between 0.85 and 0.99 §3.1, but Table 3 uses k = 30 and α = 98.5 tab. 3.
• Validation split conflicts. Text says 10% of training data §5.1; Table 2 says 0.1 of the test set tab. 2.
• NYCT window length. Table 2 lists w = 30 for the Yahoo sets only tab. 2. The window for NYCT is not stated.
• Standardization scope. Per series or per dataset is not stated §4.3.1. The dataset-level choice changes the scale for every series.
• Duplicate timing values. The DBSCAN and LOF rows in tab. 7 are identical, and tab. 8 gives AR's training and inference times as the same value. The paper does not explain these. Treat them as possible copy errors.
• Series count mismatch. The runtime section counts 367 series §6.2, and the conclusion says 368 §7.
• K-Means on NYCT. tab. 6 marks K-Means as highest on NYCT (0.9137), and the text does not discuss it §6.1.
• Figure captions. The NYCT figure caption repeats the UD1-UD4 average caption fig. 18. The seasonal-data caption describes a stock price, while the text describes a home-sales index fig. 4 §2.4.
• Typos in the hyperparameter table. The GRU row says "2 stateful LSTM layers" tab. 5. The text says GRU equals the LSTM model except for cell type §3.3.
• Isolation Forest notation. The paper writes c(w) and c(p) for the same normalizer §3.2.
• OC-SVM. The "upper bound of outliers 0.7" is not mapped to ν tab. 4. Window-to-2-D mapping is described for t > 1 only §3.2.
• Stateful LSTM and GRU. The paper does not state when state is reset.
• Deep timing on GPU. The paper does not say whether training and inference times used the GPU tab. 1.
• Author-reported single runs. Table values are single numbers without variance or seeds, and the test series are short (roughly 1400 to 1680 steps).
• The "best" average claim depends on Figure 17, which this card does not reproduce.

Lineage

• Uses the NYC taxi series from the Numenta Anomaly Benchmark §4.2. The wiki card for that benchmark is NAB.
• Related work the paper cites: forecasting comparisons such as the M3 competition, a 22-algorithm IoT comparison, and FuseAD, DeepAnT and TwitterAD §4.1.
• Superseded by: not stated.