Series decomposition

Overview

Series decomposition splits one series, or its features, into parts that are easier to model. Three forms appear in these cards.

An explicit split takes a smooth trend, with a moving average, and calls the remainder the part the trend does not explain. The paper calls that remainder "seasonal" 2205.13504 §4. The split is borrowed from Autoformer and FEDformer, which the paper credits 2205.13504 §3. A learned split subtracts nothing from the input. The encoder maps the input to two sets of features, one trend-like and one seasonal-like, and each set is trained with its own loss 2202.01575 §3. A third form peels parts off inside the network. Each encoder stack removes a seasonal part and a growth part from its residual, and the decoder combines the level, growth and season it kept 2202.01381 §4.1.

The explicit form is a preprocessing step. Each part is forecast by its own model, and the two forecasts are summed 2205.13504 §4. The learned form produces features for a shared regressor. The paper pretrains the encoder with contrastive losses and fits a ridge regression on the frozen features 2202.01575 §4.1. The residual form has no separate regressor. The decoder projects the summed growth and season and adds the level 2202.01381 §4.1.3. In all forms the choice of smoother or feature design, and the choice of model for each part, are design decisions.

Keep decomposition apart from level shifting. Subtracting the last observed value, as NLinear does, changes the level of the input but does not split it into parts 2205.13504 §4. ETSformer's level is different: it is a component of its output, smoothed from the data and held over the horizon 2202.01381 §4.1.2. See Variants.

The evidence is three cards that do not share a protocol. 2205.13504 trains end-to-end linear forecasters with a look-back of 336 and one run per setting app. B.2 tab. 2. 2202.01575 reports a frozen encoder with a ridge head and does not run the linear baselines §4.1 tab. 1. 2202.01381 picks its look-back and K on validation, reports the mean of three runs, and compares with Transformer and recurrent forecasters rather than linear ones §5 app. C.1. Every number below is author-reported. See Contradictions before comparing across them.

Variants

Moving-average trend and remainder (DLinear)

The input window is smoothed with a moving average of kernel size 25. Edge values are repeated at both ends so the length stays the same 2205.13504 §4 2205.13504 app. B.2. The trend is the smoothed series, and the remainder is the input minus the trend 2205.13504 §4. Two separate linear layers map each part to the forecast horizon, and the outputs are summed. Card: Are Transformers Effective for Time Series Forecasting?.

The paper uses kernel 25 2205.13504 app. B.2, and the card records no other kernel sizes. On the main table, DLinear beats plain Linear in more cells than it loses, as counted in "When to use which".

Plain linear, no split (for comparison)

The paper's plain Linear model has no decomposition. One linear layer maps the whole window to the horizon 2205.13504 §4. It is the reference for measuring what the split adds.

Not a decomposition: subtract the last value (NLinear)

The last observed value is subtracted from the window, the linear layer runs on the offsets, and the value is added back 2205.13504 §4. The stated aim is to reduce the shift between training and test distributions 2205.13504 app. D.1. Its trend is not split off, so it belongs with level handling. The paper reports it alongside DLinear as a separate model 2205.13504 tab. 2.

Learned trend and seasonal features (CoST)

The encoder's output has d = 320 channels, split into a trend part and a seasonal part of 160 channels each 2202.01575 §3. The trend part is the average of L+1 causal convolutions whose kernel sizes are powers of two, from 1 up to 2^L 2202.01575 §3.1. The seasonal part takes a discrete Fourier transform over time, applies a learnable complex map to each frequency bin, and transforms back 2202.01575 §3.2. The trend part is trained with a MoCo-style contrastive loss over time steps. The seasonal part is trained with amplitude and phase contrastive losses over frequency bins 2202.01575 §3.1 2202.01575 §3.2. The two losses are combined with a weight alpha 2202.01575 §3. After pretraining, the encoder is frozen and a ridge regression predicts the horizon 2202.01575 §4.1. Card: CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting.

Residual peeling inside the network (ETSformer)

Each encoder stack takes the residual from the stack before it. It first extracts a seasonal part with frequency attention, which keeps the K Fourier bases with the largest amplitude for each dimension, and subtracts it. It then extracts a growth part with multi-head exponential smoothing attention and subtracts that. A smoothing equation updates the level, using the growth and season from the same stack 2202.01381 §4.1.2 2202.01381 §4.2.2. In the decoder, each stack damps its last growth state with a learned factor, extends its season past the window on the same bases, and the last level is held flat over the horizon 2202.01381 §4.1.3 2202.01381 §4.2.2. The paper combines the summed growth and season with one linear map and adds the level to the result 2202.01381 §4.1. Card: ETSformer: Exponential Smoothing Transformers for Time-series Forecasting.

No period is set by hand in this form: the bases are chosen by amplitude 2202.01381 §4.2.2. The paper's ablation, at horizon 24 only, shows that removing the season hurts most on ETTh2, ECL and Traffic, and removing the level hurts most on ETTm2 2202.01381 tab. 2.

When to use which

Counts for DLinear are from its MSE values, one run per setting 2205.13504 tab. 2. The CoST figures are from its own tables, with the same caveat 2202.01575 tab. 1 2202.01575 tab. 3. The ETSformer figures come from its main table, with the same caveat 2202.01381 tab. 1.

The paper's gaps between Linear and DLinear are often small. ETTh1 at T=96 gives 0.375 MSE for Linear and DLinear, with NLinear at 0.374 2205.13504 tab. 2. The paper gives no spread for these values, so a gap of that size is not a reliable ranking.

Implementation sketch

Language-agnostic sketch of the moving-average split. The paper's kernel and edge handling are in 2205.13504 app. B.2. The parts are then passed to separate forecasters, as described in Variants.

# Split a window into trend and remainder  [arxiv:2205.13504 §4]
# Input: x (B, L, C)   Output: trend, remainder, each (B, L, C)

def decompose(x, kernel=25):
    # Smooth along time; edge values are repeated so the length stays L
    # [arxiv:2205.13504 app. B.2]
    trend = moving_average(x, kernel)
    # Whatever the trend does not explain  [arxiv:2205.13504 §4]
    remainder = x - trend
    return trend, remainder

# Forecast each part with its own layer, then sum  [arxiv:2205.13504 §4]
trend, remainder = decompose(x)
y_hat = trend_layer(trend) + remainder_layer(remainder)

The learned split works on encoder features instead of the raw input. It is the sketch for the CoST variant, not for DLinear.

# Split encoder features into trend and seasonal parts  [arxiv:2202.01575 §3]
# Input: v (B, T, d), d = 320   Output: trend, season, each (B, T, d/2)

def disentangle(v, kernels):
    # Trend: average of causal convolutions with kernel sizes 2^i  [§3.1]
    trend = mean([causal_conv(v, k) for k in kernels])
    # Seasonal: Fourier transform over time  [§3.2]
    freq = rfft(v, dim=time)                    # (B, F, d) complex
    # One learned complex map per frequency bin  [§3.2]
    freq = einsum("bfd,fdk->bfk", freq, A) + B  # A: (F, d, d/2), B: (F, d/2)
    season = irfft(freq, n=T, dim=time)         # (B, T, d/2)
    return trend, season

The residual form is a third sketch. Parts are peeled off one stack at a time, and the order matters: the season comes out before the growth 2202.01381 §4.1.2. The sketch follows the paper's text, not the code.

# One encoder stack of residual peeling  [arxiv:2202.01381 §4.1.2]
# Input: residual (B, L, d), level (B, L, m)
# Output: season, growth, updated level, residual for the next stack

def peel_stack(residual, level, K):
    # Seasonal part: keep the K largest Fourier bases  [§4.2.2]
    season = frequency_attention(residual, K)   # (B, L + H, d)
    residual = residual - season[:, :L]
    # Growth part: exponential smoothing on successive differences  [§4.2.1]
    growth = exp_smoothing_attention(residual)  # (B, L, d)
    residual = layer_norm(residual - growth)
    # Level: smoothing update using this stack's growth and season  [§4.1.2]
    level = level_update(level, growth, season[:, :L])
    return season, growth, level, residual

Contradictions

Reported MSE at the same horizon, DLinear vs CoST. The two papers report different values for the same dataset and horizon, under different protocols. At each horizon below, DLinear's MSE in Table 2 is lower than CoST's MSE in Table 1 2205.13504 tab. 2 2202.01575 tab. 1:

• ETTh1, T=336: CoST 0.812, DLinear 0.439.
• ETTh1, T=720: CoST 0.970, DLinear 0.472.
• ETTh2, T=720: CoST 1.971, DLinear 0.605.
• Electricity, T=336: CoST 0.196, DLinear 0.169.
• Electricity, T=720: CoST 0.232, DLinear 0.203.
• Weather, T=720: CoST 0.533, DLinear 0.323.

Neither paper runs the other's model. CoST's table has no DLinear row, and DLinear's table has no CoST row 2202.01575 tab. 1 2205.13504 tab. 2. The protocols differ: CoST uses a frozen encoder with a ridge head, and DLinear trains end-to-end with look-back 336. This page does not resolve the gap.

Reported MSE at the same horizon, DLinear vs ETSformer. The two papers share six datasets: ETTm2, Electricity (ECL in the ETSformer table), Exchange, Traffic, Weather and ILI. Neither paper runs the other's model 2202.01381 tab. 1 2205.13504 tab. 2. On those six datasets, DLinear's reported MSE is lower than ETSformer's in every one of the 24 rows of the ETSformer table. Two examples: Traffic at T=96, 0.607 for ETSformer and 0.410 for DLinear; ETTm2 at T=720, 0.414 and 0.397 2202.01381 tab. 1 2205.13504 tab. 2. The protocols differ: ETSformer picks its look-back from 96, 192, 336 and 720 on validation 2202.01381 tab. 3 2202.01381 app. C.1, while DLinear fixes L=336 2205.13504 app. B.2. This page does not resolve the gap.

Linear look-back inside the DLinear paper. Table 4 uses look-back 96 for Linear and Table 2 uses 336, so their Linear numbers differ 2205.13504 tab. 4 2205.13504 tab. 2. That affects the Linear baseline, not the decomposition comparison within Table 2.

Papers

• Are Transformers Effective for Time Series Forecasting?: proposes LTSF-Linear, including DLinear (moving-average split plus two linear layers) and NLinear (level subtraction), and compares them with Transformer forecasters on nine benchmarks.
• ETSformer: Exponential Smoothing Transformers for Time-series Forecasting: replaces self-attention with exponential smoothing attention for growth and frequency attention for seasonality, and forecasts level, growth and season from separate parts; compares with Transformer and recurrent forecasters.
• CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting: learns trend and seasonal representations with a time-domain and a frequency-domain contrastive loss, then fits a ridge regressor on the frozen features; compares with representation learners and end-to-end forecasters.