Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting

In plain words

Forecasting many related sensors, such as traffic counters or power meters, means predicting each sensor's next values from its recent past. The strongest recent methods pass information between sensors along a road or grid graph. That makes them complicated and slow to train. The authors argue that the real bottleneck is simpler: two situations can look the same in their recent history and still have different futures. This can happen across different sensors, or for the same sensor at different times of day or days of the week.

The fix they propose is to give every sensor a learned identity, and every time-of-day and day-of-week slot a learned identity too. These identities are joined to the history and fed through a plain stack of small neural layers. The model uses no graph and no recurrence. The authors report that it is more accurate than the graph-based baselines they test and that it trains faster per epoch. These are the authors' own numbers.

TL;DR

• A graph-free MLP forecaster for multivariate time series. It argues that sample indistinguishability in space and time is the main bottleneck §1.
• Each input is embedded linearly, then concatenated with a spatial identity, a time-of-day identity and a day-of-week identity, giving a 4D vector §3 eq. 1 eq. 2.
• L residual MLP blocks (L = 3 in the paper) and a linear regression head predict F = 12 future steps, trained with MAE §3 eq. 3 eq. 4 eq. 5.
• Author-reported: lowest average MAE on all five benchmarks (PEMS04, PEMS07, PEMS08, PEMS-BAY, Electricity) and the lowest seconds per epoch tab. 2 tab. 3.
• The reported Electricity setup differs from the released config (input 168 in the paper, 336 in code). See Paper vs code.

Problem

Multivariate time series (MTS) forecasting covers traffic, energy and similar domains §1. Spatial-temporal graph neural networks (STGNNs) are the strong recent methods, but the authors note that their gains are getting smaller while the models grow more complex §1.

The paper names the cause as indistinguishability. Samples are cut from the series with a sliding window of length P+F §1 fig. 1. Different sensors, or the same sensor at different periods, can have similar histories but different futures §1 fig. 1. A plain regression model such as an MLP cannot tell these samples apart, because it sees only the history §1. The paper cites earlier work that credits GCN layers with relieving spatial indistinguishability §1.

The paper gives no formal measure of indistinguishability and no threshold for when it matters §1.

Method

STID is a plain MLP with three kinds of learned identity. It processes one window of history for one series at a time. Notation: P input steps, F output steps, N series, D hidden size §2.

1. Embedding §3 eq. 1. A linear layer maps the P-step history of series i to a D-dimensional vector H. The paper writes this for one series. The code applies one shared linear map to every series stid_arch.py.

2. Identity concatenation §3 eq. 2. Three trainable tables are attached to H:

• a spatial identity E_i per series, shape N × D, randomly initialised §2;
• a time-of-day identity T^TiD_t, one row per slot of the day (N_d slots) §2;
• a day-of-week identity T^DiW_t, one row per day of the week (N_w = 7) §2.

The paper says the temporal identities are shared by all windows that fall on the same time slot or the same weekday §3. The result is a 4D-wide vector Z per series §3.

3. Encoder §3 eq. 3. L residual blocks. Each block applies a linear layer, a non-linearity, a second linear layer, and adds its input back. The paper's activation is written as σ; the code uses ReLU mlp.py.

4. Regression §3 eq. 4. A linear layer maps the final 4D representation to F future values for each series §3.

5. Loss §3 eq. 5. Mean absolute error over all series and all F future steps. Optimised with Adam §3.

# STID forward pass for one window  [§3]
# Input:  history (B, P, N)  past values, P steps, N series
#         time_of_day (B,)   time-slot index of the window's last step
#         day_of_week (B,)   day index 0..6 of that step
# Output: forecast (B, N, F)

def stid_forward(history, time_of_day, day_of_week):
    # Embed each series' P-step history with a linear map  [eq. 1]
    hidden = linear_embedding(history)              # (B, N, D)
    # Look up one learned spatial identity per series  [§3]
    spatial = spatial_identity                      # (N, D)
    # Look up learned time-of-day and day-of-week identities  [§2] [§3]
    slot = time_of_day_identity[time_of_day]        # (B, D)
    weekday = day_of_week_identity[day_of_week]     # (B, D)
    # Concatenate the four parts into one 4D vector per series  [eq. 2]
    z = concat([hidden, spatial, slot, weekday])    # (B, N, 4D)
    # Apply L residual MLP blocks  [eq. 3]
    for block in mlp_blocks:
        # Linear, non-linearity, linear, then add the block input  [eq. 3]
        z = block.fc2(relu(block.fc1(z))) + z       # (B, N, 4D)
    # Map the 4D representation to F future values  [eq. 4]
    return regression(z)                            # (B, N, F)

# One training step  [§3] [eq. 5]
# Loss: mean absolute error over all series and all future steps

def training_step(history, target, time_of_day, day_of_week):
    # Predict the F future steps for all N series  [eq. 4]
    forecast = stid_forward(history, time_of_day, day_of_week)
    # Average |forecast - target| over the N series and the F steps  [eq. 5]
    loss = mean(abs(forecast - target))
    # Update identities and linear weights by backprop and Adam  [§3]
    backprop(loss)
    adam_step()
    return loss

Hyperparameters

Values are as the paper states them unless marked as code. The paper gives no batch size, epoch count, weight decay, dropout or gradient clipping.

Results

Author-reported, from single runs. The paper gives no variance, no seeds and no repeat counts. Setup: five datasets tab. 1, forecast horizon F = 12, reported at steps 3, 6 and 12 and as an average over the 12 steps §4.2.

Datasets tab. 1

STID, all metrics tab. 3

Average MAE, all methods tab. 3. Only the Avg. MAE column is copied here. RMSE and MAPE for the baselines are in tab. 3. "N/A" means the baseline needs a pre-defined graph, which Electricity lacks §4.2.

The paper states that STID is best at almost all horizons on all datasets and does not need a pre-defined graph §4.2. The table shows exceptions, for example PEMS04 RMSE at 12 steps, where GMAN is best tab. 3.

Training time, seconds per epoch tab. 2.

Ablation §4.4 fig. 3. On PEMS04 the paper removes one identity at a time: the spatial identity E, the time-of-day identity T^TiD, or the day-of-week identity T^DiW. The text says all three help, and that the spatial identity matters most. The exact values are only in fig. 3, which this card does not transcribe.

Visualisation §4.5 fig. 4. On PEMS08, t-SNE of the spatial identities shows clusters, and the authors say nearby sensors tend to share similar patterns. Time-of-day identities show a daily pattern, and weekday identities differ from weekend ones §4.5.

Paper vs code

Official code: the paper links it in §4.1, and the repository owner is the first author. Code cloned at commit e8b313bc591bdd0101a1619962c9b503e75127c0, Apache-2.0 LICENSE. I read the model, the config files for PEMS04 and Electricity, the scaler, the loss, the metrics and the runner. I did not read the PEMS07, PEMS08 or PEMS-BAY configs line by line; I checked their key hyperparameters with grep.

Discrepancies, each with both citations:

• Embedding input. The paper embeds only the traffic history X in R^P §3 eq. 1. The code flattens all input_dim channels, which is 3 for PEMS: traffic value, time of day and day of week. So the embedding sees 3P inputs stid_arch.py stid_arch.py PEMS04.py. Electricity uses input_dim 1 Electricity.py.
• Electricity input and output length. The paper says P = 168 and F = 12 for Electricity §4.1. The released config uses 336 and 336 generate_training_data.py. The paper's Electricity column in tab. 3 is labelled @3, @6, @12 and Avg. The code evaluates horizons up to 336 Electricity.py. The paper does not say which setup produced the Electricity column.
• Electricity size and capacity. Paper: D = 32 and L = 3 for all datasets §4.1. Code: D = 2048, L = 1, identity sizes 8, 24 time slots, and 321 nodes Electricity.py Electricity.py Electricity.py Electricity.py Electricity.py. Table 1 lists 336 variables tab. 1.
• Electricity split. Paper: 6:2:2 for all datasets except PEMS-BAY §4.2. Code: 0.7/0.1/0.2 generate_training_data.py.
• Learning rate. Paper: 0.001 §4.1. Code, PEMS: 0.002 PEMS04.py, with a MultiStepLR halving at epoch 1 PEMS04.py. Code, Electricity: 0.0005 Electricity.py.
• Dropout. Paper: none stated. Code: 0.15 in every MLP block mlp.py.
• Weight decay and clipping. Paper: not stated. Code: weight decay 1e-4 for PEMS and 5e-4 for Electricity PEMS04.py Electricity.py; gradient norm clipped at 5.0 PEMS04.py.
• Epochs and batch size. Paper: not stated. Code: 100 epochs, batch 64 PEMS04.py PEMS04.py.
• Time-of-day feature. The paper's notation uses t §3. The code takes the identity from the last observed history step only, using [:, -1, :] stid_arch.py stid_arch.py.

No discrepancies were found in the core formulas: the four-part concatenation, the residual MLP and the regression head match eq. 2 to eq. 4 stid_arch.py mlp.py.

Pitfalls

• Loss scale depends on the dataset. For PEMS, RESCALE is true, so the loss is computed after the inverse transform, in original units base_tsf_runner.py base_tsf_runner.py. For Electricity, RESCALE is false, so the loss is on normalised values generate_training_data.py. The paper states neither choice.
• Zero readings are masked. For PEMS the masked loss and metrics skip entries whose target equals 0.0, with a tolerance of 5e-5. For Electricity they skip NaN mae.py. The paper says nothing about missing data.
• Scaler statistics. The mean and std come from the first 60% or 70% of the series, and only for the traffic channel. Time features are not normalised z_score_scaler.py z_score_scaler.py.
• Time-slot index by float cast. The time-of-day feature is stored as i % 288 / 288 in float, then multiplied back by 288 and cast to a long integer generate_training_data.py stid_arch.py. A float round trip just below an integer would truncate to the previous slot. I did not run this check, so treat it as a hypothesis to verify.
• Learning-rate schedule. The code starts at 0.002 and halves the rate after epoch 1, which would give 0.001 from epoch 2 on. This is my reading of standard MultiStepLR behaviour PEMS04.py base_tsf_runner.py. I did not run it. It may explain the paper's stated 0.001, but the paper does not say so.
• No seeds, no repeats. I found no seed setting in basicts/runners/, basicts/launcher.py or experiments/. The paper reports one number per cell with no spread. Expect run-to-run variation, and compare against the paper's gaps with that in mind.
• Model selection. The checkpoint with the best validation MAE is kept base_tsf_runner.py. The paper does not say how its reported test numbers were chosen.
• Identities depend on sampling rate. The time-of-day table has 288 rows for 5-minute data and 24 for hourly Electricity Electricity.py. Changing the sampling rate changes the table size, and the paper's notation doesn't show this.
• Efficiency numbers are hardware-specific. The timings in tab. 2 come from one server with a 2080Ti §4.3. They measure the authors' framework and data loader, not the model alone.
• Electricity result is hard to reproduce from the paper alone. The paper gives P = 168 and F = 12, but the code uses 336 and 336, a different width and depth, and a different node count. Implement the released config if you need to match the table, and say which config you used.

Lineage

• Builds on: nothing named in the paper as a direct precursor. It takes its motivation from prior STGNN work and from STNorm, which the paper credits with showing that GCNs relieve spatial indistinguishability §1.
• Compared against: HI, VAR, LSTM, DCRNN, STGCN, Graph WaveNet, AGCRN, StemGNN, GMAN, MTGNN and STNorm §4.1. These are not cards in this wiki yet, so nothing from them is used here.
• Superseded by: not stated in the content available to this card.