Direct multi-step forecasting

Overview

Direct multi-step (DMS) forecasting predicts all T future steps from a look-back window in one model call. The alternative, iterated multi-step (IMS), predicts one step, feeds it back as an input, and repeats, so errors can build up over the horizon 2205.13504 §1 2205.13504 §2.

The one study on this page argues for a DMS-style linear baseline. A single linear layer maps the recent values of each variate to the whole horizon, and the paper compares it with published Transformer forecasters on nine benchmarks 2205.13504 §4 2205.13504 §5.1. In the main table, the best linear model has lower MSE than the best Transformer in each of 36 dataset-horizon cells 2205.13504 tab. 2. These are author-reported numbers. The table gives no run count or spread, and the Transformer numbers (except Pyraformer) are copied from another paper, not re-run [arxiv:2205.13504 tab. 2].

The paper hypothesises that the gains of Transformer forecasters come largely from the DMS strategy they use 2205.13504 §4. Its evidence for this is a simplification: Informer reduced to one linear layer has lower MSE than Informer in every Exchange-Rate and ETTh1 cell it reports, at look-back 96 2205.13504 tab. 4. No table in the card compares IMS and DMS on the same model, so the page does not show that DMS beats IMS directly.

Variants

All three variants below come from the same paper and share its DMS output. Each variant has a card: Are Transformers Effective for Time Series Forecasting?.

Single linear layer (LTSF-Linear)

One weight matrix maps the last L = 336 values of each variate to all T future values. The weights are shared across variates, so no correlation between variates is modelled 2205.13504 §4 2205.13504 app. B.2. This is the base model. Plain Linear does not beat every Transformer cell: at ETTh2, T=720, its MSE is 0.698 against 0.463 for FEDformer 2205.13504 tab. 2.

Trend and remainder split (DLinear)

A moving average with kernel size 25 separates the input into a slow trend and a remainder. Each part gets its own linear layer, and the two forecasts are summed 2205.13504 §4 2205.13504 app. B.2. The paper borrows the decomposition from Autoformer and FEDformer 2205.13504 §3. On Exchange-Rate, DLinear has the lowest MSE of the three linear models at all four horizons 2205.13504 tab. 2. At T=720 it is 0.643 against 0.964 for plain Linear 2205.13504 tab. 2.

Last-value anchor (NLinear)

The last observed value is subtracted from the input window, the linear layer runs, and the value is added back to the output 2205.13504 §4. The paper says the purpose is to reduce the shift between training and test periods 2205.13504 §4 2205.13504 app. D.1. On ETTh1 at T=720 its MSE is 0.440 against 0.624 for plain Linear, and on ETTh2 at T=720 it is 0.394 against 0.698 2205.13504 tab. 2.

Transformer forecasters (the baseline class)

Informer, Autoformer, FEDformer, Pyraformer and LogTrans are the Transformer forecasters the paper tests against. The paper says these models use the DMS strategy 2205.13504 §4. Their own cards are not in this wiki, so they are named here only.

When to use which

Implementation sketch

The two strategies differ in how many model calls they make. The DMS core is a single call. The IMS loop is shown for contrast.

# Iterated multi-step (IMS)  [arxiv:2205.13504 §1] [§2]
# Each prediction is appended to the window and fed back as an input
def ims_forecast(x, step_model, T):
    window = x                                     # (B, L, C)
    preds = []
    for t in range(T):
        # one step ahead, then slide the window by one  [§1]
        y_next = step_model(window)                # (B, 1, C)
        preds.append(y_next)
        window = concat(window[:, 1:, :], y_next, axis=1)
    return concat(preds, axis=1)                   # (B, T, C)

# Direct multi-step (DMS)  [arxiv:2205.13504 §4]
# One call returns all T steps; no prediction is fed back
def dms_forecast(x, model):
    # the model's output layer has T outputs per variate  [§4]
    return model(x)                                # (B, T, C)

Contradictions

• "In all cases" against the table. The abstract says the linear models beat the Transformers "in all cases" 2205.13504 abstract. Table 2 supports this for the best linear model in each cell. It does not hold for every linear model: plain Linear loses at ETTh2, T=720 [arxiv:2205.13504 tab. 2].
• Repeat baseline. On Exchange-Rate at T=96, the last-value Repeat baseline ties DLinear on MSE (0.081) and beats it on MAE (0.196 against 0.203) 2205.13504 tab. 2.

No conflict with other papers is known, because this page cites one card.

Papers

• Are Transformers Effective for Time Series Forecasting?: one linear layer with a DMS output, and two variants, DLinear and NLinear, compared against published Transformer forecasters on nine benchmarks.