FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective

In plain words

Forecasting asks what comes next, given the recent past of many related signals at once. Many earlier models split the job in two: one part handles how the signals affect each other, another handles how each signal changes over time. The authors argue that this split is awkward and misses the way space and time are tied together. Their fix is to lay the recent window out as one graph, in which every reading of every signal at every time step is a node linked to every other node. Graph layers then mix information across all of these nodes, and the method does that mixing in the frequency domain, where it is cheap. The reported results on standard benchmarks are strong, but they are the authors' own, and the released code does not match the paper in several places.

TL;DR

• Models a lookback window of N series and T steps as one fully connected graph with N·T nodes, then forecasts with stacked Fourier-domain graph layers §4.1 §4.2 §4.3.
• Each layer multiplies the spectrum by a learned complex d-by-d matrix that is shared by all nodes, so the cost grows with n log n rather than n² §4.2 eq. 6.
• Author-reported: best MAE and RMSE on all six datasets of the main table at horizon 12, with the best MAPE on four of them tab. 1 §5.2.
• The official code is MIT-licensed and does not follow the paper's equations closely. It uses diagonal weights, a fixed time compression and no adjacency matrix. See Paper vs code.

Problem

Multivariate forecasting maps a window of T steps of N series to the next τ steps §3. Many models use a graph network for relations between series and a temporal network for each series' dynamics §3. The authors say these two networks are hard to make compatible and that the pairing misses the joint space-time dependencies §4.1. Treating the whole window as one graph avoids the pairing, but classic graph networks such as GCN and GAT become expensive, with roughly quadratic cost in the number of nodes, and they are hard to optimise on such graphs §4.2.

Method

Hypervariate graph. Each element of the input window X_t ∈ R^{N×T} becomes one node, so the graph has NT nodes. Its node feature is the scalar value, and its adjacency matrix is all ones, i.e. the graph is fully connected §4.1 Def. 1 §4.3. The paper's figure of three series illustrates the construction fig. 1.

Node embedding. Each scalar is embedded into d dimensions with a learned row vector E_φ ∈ R^{1×d}, applied to all nodes §4.3.

Fourier graph operator (FGO). For layer k the paper defines a weight matrix W_k ∈ R^{d×d} and the adjacency A_k, which shares the sparsity pattern of A. The layer is written so that F(X) S_k = F(A_k X W_k), where F is the discrete Fourier transform §4.2 eq. 4 eq. 5. For the fully connected graph, the paper uses one complex d-by-d matrix S per layer that is the same at every frequency, which it calls an n-invariant FGO §4.2. Appendix C.2 compares this with a version that has a separate matrix for each frequency: on ECG the n-invariant model has 0.18M parameters against 82.96M, and on COVID-19 1.06M against 130.99M app. C.2.

FourierGNN. The FGO is stacked K times. The output is a sum over orders k of a non-linear activation applied to the spectrum times the running product of FGOs, plus a bias §4.2 eq. 6. The default is K = 3 for all datasets app. F.4, and K = 3 gives the best result in the depth study app. H.2. The paper states that the k = 0 term, which uses the identity FGO, acts as a residual connection §4.2.

Forecasting head. The model takes the DFT over the NT nodes, applies FourierGNN, takes the inverse DFT, reduces the time axis from T to l steps with a linear map, flattens to N × (l·d), and applies a feed-forward network with three linear layers and LeakyReLU between them, giving N × τ outputs §4.3 eq. 8 app. F.4. Figure 2 shows this pipeline fig. 2.

Theory. Proposition 1 states that multiplying by the FGO recursion in Fourier space equals a sum of multi-order graph filters in the time domain, with per-order weights §4.2 eq. 7. The paper argues that a single multiplication in Fourier space costs O(nd log n + nd²), against O(n²d + nd²) for the equivalent time-domain shift, so the K-order model costs O(nd log n + K·nd²) §4.2.

# FourierGNN forecast on one batch  [§4.3] [eq. 8] [app. F.4]
# Input:  window (B, N, T)
# Output: forecast (B, N, tau)

def forecast(window):
    # 1. Embed each value of the hypervariate graph into d dims  [§4.3]
    nodes = window.reshape(B, N * T, 1) * E_phi  # (B, NT, d); E_phi (1, d)

    # 2. Discrete Fourier transform over all NT nodes  [§4.3]
    spectrum = DFT(nodes)                        # (B, NT, d), complex

    # 3. Stacked Fourier graph layers, summed over orders  [eq. 6] [eq. 8]
    mixed = fourier_graph_network(spectrum)      # (B, NT, d), complex

    # 4. Back to the time domain  [§4.3]
    values = IDFT(mixed)                         # (B, NT, d), real
    grid = values.reshape(B, N, T, d)

    # 5. Reduce time from T to l steps, then flatten  [§4.3] [app. F.4]
    compact = linear_over_time(grid, T, l)       # (B, N, l, d)
    flat = compact.reshape(B, N, l * d)

    # 6. Feed-forward head: three linear maps with LeakyReLU  [app. F.4]
    return feed_forward(flat)                    # (B, N, tau)

# Fourier graph network, the sum over orders  [eq. 6] [§4.2]
# S[k]: complex (d, d) weight, shared by all NT frequencies  [§4.2]
# b[k]: complex bias, shape (d,)  [eq. 6]

def fourier_graph_network(spectrum):
    # Order 0 uses the identity FGO and gives the residual term  [§4.2]
    total = act(spectrum + b[0])
    # The running product S_{0:k} is built one order at a time  [eq. 6]
    partial = spectrum
    for k in range(1, K + 1):
        partial = partial @ S[k]                 # F(X) S_{0:k}, (B, NT, d)
        total = total + act(partial + b[k])      # act: ReLU  [app. F.4]
    return total                                 # (B, NT, d), complex

Key equations, as written in the paper:

\operatorname{FourierGNN}(X, A) = \sum_{k=0}^{K} \sigma\big(\mathcal{F}(X)\,\mathcal{S}_{0:k} + b_k\big), \quad \mathcal{S}_{0:k} = \prod_{i=0}^{k} \mathcal{S}_i eq. 6

\mathcal{Y}^{\mathcal{G}}_t = \sum_{k=0}^{K} \sigma\big(\mathcal{F}(\mathbf{X}^{\mathcal{G}}_t)\,\mathcal{S}_{0:k} + b_k\big) §4.3, eq. 8

Hyperparameters

Values from the paper are cited to the paper. Where the paper says nothing, the value is taken from the official code and marked as such.

Results

Author-reported. Setup: input length 12 and horizon 12 unless stated, on the datasets of app. F.1 (the main table shows six of them), trained as in Hyperparameters, on a single RTX 3080. The paper's baseline settings follow each repository's recommended configuration where the appendix says so app. F.2. The FourierGNN row is set in bold here. The paper's own bold and underline marks are dropped, so read the numbers directly. A dash means the paper reports no value.

Main table, Solar / Wiki / Traffic, horizon 12 tab. 1

Main table, ECG / Electricity / COVID-19, horizon 12 tab. 1

The paper reports the average gain over the best baseline on each dataset as 9.4% in MAE and 10.9% in RMSE §5.2. It describes the model as a new state of the art on all datasets §5.2. The MAPE column does not support that wording on Solar and Traffic, where the paper's own table shows a lower MAPE for FEDformer and LSTNet.

COVID-19 by horizon tab. 2

The paper reports average MAE and RMSE gains of 30.1% and 30.2% over the best baseline across these horizons §5.2. On MAPE, TAMP-S2GCNets is lower at horizons 3, 6 and 12 tab. 2.

Efficiency, Traffic and Wiki tab. 3 (averages over five rounds)

Against GraphWaveNet, the paper reports 32.2% fewer parameters on Traffic and 9.5% fewer on Wiki §5.3.

Ablation on METR-LA, horizon 12 tab. 4 (the paper's ablation variants)

Table 4 is the only ablation in the main text. The appendix ablation adds COVID-19 results app. H.3. A METR-LA comparison against eight other graph and temporal models is in the appendix app. G.

Figures: the paper shows temporal adjacency heatmaps for eight COVID-19 counties over twelve steps fig. 3, and a learned spatial adjacency for twenty METR-LA detectors laid next to a road map fig. 4. Both are qualitative.

Paper vs code

Official code at commit f239290b6f6881c53c5dda094bb0a7255eb0ea2d, in the repository linked from the abstract. Its LICENSE names the first author. The repository has main.py, model/FourierGNN.py, data/data_loader.py, utils/utils.py and a README. It has no test suite and no hyperparameter search script. The code is released under the MIT licence.

• Diagonal weights, not full matrices. The paper gives each layer a d-by-d complex weight W_k §4.2 eq. 6. The code multiplies only along the diagonal, using einsum('bli,ii->bli'), so each feature has one complex scalar per layer FourierGNN.py FourierGNN.py FourierGNN.py FourierGNN.py FourierGNN.py FourierGNN.py.
• Nested activations. Eq. 6 applies the activation once to F(X)·S_{0:k} + b_k eq. 6. The code feeds each layer's activated output into the next layer, so the activation sits inside the recursion FourierGNN.py FourierGNN.py.
• Extra soft-shrink, not in the paper. After each layer the code applies softshrink with λ = 0.01 FourierGNN.py FourierGNN.py FourierGNN.py FourierGNN.py.
• ReLU on real and imaginary parts separately. The code applies ReLU to the real and imaginary parts independently FourierGNN.py, which is not the same as a complex activation. The paper does not say which it intends app. F.4.
• Half spectrum. The code uses a real FFT over the flattened NT nodes, keeps only the (NT/2 + 1) non-negative frequencies, and applies the complex weights there FourierGNN.py FourierGNN.py. The paper's DFT is over all NT points §4.3. The 2-D extension in app. C.4 is not used.
• No adjacency. The paper's A_k, with the sparsity of the all-ones A §4.3, is absent from the code, which never builds A FourierGNN.py.
• Residual term. The code adds the input spectrum back after the FGO stack FourierGNN.py FourierGNN.py. This matches the k = 0 residual described in §4.2.
• Time compression fixed at 8. The paper tunes l per dataset and does not reduce ECG app. F.4. The code always uses a learned map from T to 8 FourierGNN.py FourierGNN.py.
• FFN width fixed at 64. The first FFN layer is always 64 wide FourierGNN.py. The paper uses 256 for COVID-19 app. F.4.
• Hard-coded GPU. The model moves itself to cuda:0 on construction FourierGNN.py, and the training and validation loops send tensors to cuda:0 main.py main.py main.py. The code will not run on a machine without CUDA.
• Training schedule. The paper gives no epoch count or decay §5.1. The code trains for 100 epochs by default and multiplies the learning rate by 0.5 every five epochs main.py main.py main.py main.py.
• No model selection. The paper says parameters were chosen on the validation set §5.1. The code saves a checkpoint every epoch main.py and never picks one. The test() function loads the checkpoint of epoch 48 by hard-coded number main.py, and main does not call it.
• Unused early stopping. The --early_stop flag is parsed but never read main.py.
• Normalisation applies to four of seven datasets. The paper says all datasets are min-max normalised §5.1. The code scales only when the dataset type is '1'. Traffic is type '0' and is not scaled main.py data_loader.py. ECG, COVID-19, Electricity and METR-LA use the scaled path data_loader.py main.py. For Solar and Wiki the scaled array is written to a local variable, while the splits read the unscaled self.data data_loader.py data_loader.py.
• Solar is filtered to daytime. The code keeps only timestamps between 08:00 and 17:00 data_loader.py. The paper does not describe this filter app. F.1. The Solar path is absolute (/data/solar) and uses a Windows separator main.py data_loader.py.
• COVID-19 split needs flags. The defaults are 7:2:1 main.py. The README gives 6:2:2 for COVID-19 as a command-line option README.md.
• Batch size for ECG. The appendix gives 32 app. F.4. The README gives 4 README.md. The code default is 32 main.py.
• MAPE is clipped. The code divides by |v| + 1e-5 and caps each per-element error at 5, i.e. 500% utils.py. The paper's MAPE formula has no cap or epsilon app. F.3.
• Data files absent. The repository contains the loader but none of the data files that main.py names (data/traffic.npy, data/ECG_data.csv, data/covid.csv, data/electricity.csv, data/metr.csv, data/wiki.csv, /data/solar) main.py.
• Seeds and variance. No seed is set in the code, and the paper's main tables report single values without spread tab. 1 tab. 2.

No discrepancies were found beyond those listed above.

Pitfalls

• Dimensions disagree. The paper gives 593 Solar variables in the text and 592 in the dataset table app. F.1. The README uses 592.
• Ambiguous FFN equation. The appendix gives the first FFN weight as (T·d) by d_ffn1 app. F.4, but the text reduces time to l steps and flattens to N × (l·d). The two readings differ in input size unless l = T. The code uses l = 8 throughout.
• Complex activation not defined. The paper says ReLU app. F.4 without saying how it applies to complex numbers. The code uses it on each part separately, and adds a soft-shrink the paper does not mention.
• Real and complex parameters. The paper's biases b_k are complex §4.2. The code stores a real and an imaginary bias for each feature FourierGNN.py FourierGNN.py FourierGNN.py. This is equivalent to one complex bias, but the code's parameter layout differs from the paper's notation.
• Flatten order matters. The code flattens the window variate-major (N outer, T inner) before the FFT FourierGNN.py. Frequencies therefore mix time and variate axes in a fixed order. The paper's 2-D option app. C.4 would treat the axes separately. The paper does not state the flatten order.
• Metrics are in raw units for three datasets. MAE, RMSE and MAPE are computed on whatever scale the loader produces. For Traffic, Solar and Wiki that is raw units, not the normalised units the paper describes §5.1 data_loader.py data_loader.py data_loader.py. Compare the numbers with that in mind.
• MAPE values depend on the cap. When a per-element relative error exceeds 5, the code caps it at 5. The large Solar and Wiki MAPE values in the paper may be affected tab. 1. The paper does not say whether its numbers come from the capped code.
• Benchmark claims overreach. The text claims a new state of the art on all datasets §5.2. The paper's own tables show lower MAPE for other models on Solar, Traffic and at several COVID-19 horizons tab. 1 tab. 2.
• Dataset count. The experiments section says seven datasets §5. The main table's caption says six tab. 1. METR-LA appears only in the ablation and appendix app. G app. H.3.
• Training length is not stated. The paper gives the learning rate but no epoch count or decay §5.1. The code runs 100 epochs with a decay main.py main.py.
• Cost is operation counts, not memory. The complexity claims §4.2 count FFT and matrix work. The spectrum the code keeps is NT/2 + 1 by d complex values per batch item FourierGNN.py, which the paper does not quantify. The efficiency table is wall-clock time on the authors' GPU tab. 3 §5.1.
• Hyperparameter search is partly unreleased. The paper reports grid searches over embedding size, batch size, FFN widths and l app. F.4. The code has no search script and no logs for these runs.
• README flags are unclear. The README lists "hidden size 128 -- hidden size 256" for several datasets README.md. The second flag probably overrides the first, but the README does not say so.
• Interpretation claims are qualitative. The claims about learned temporal, spatial and time-varying dependencies rest on heatmaps for a few counties and detectors §5.4 fig. 3 fig. 4. No quantitative test is reported.

Lineage

• Builds on: the graph baselines it compares against, StemGNN, AGCRN, GraphWaveNet and MTGNN tab. 1; the paper treats GCN and GAT as the classic graph layers it replaces §4.2; the frequency-domain operator follows the Fourier neural operator line discussed in the related work §2.
• Compared against: VAR, SFM, LSTNet, TCN, DeepGLO, Reformer, Informer, Autoformer, FEDformer, GraphWaveNet, StemGNN, MTGNN and AGCRN tab. 1; TAMP-S2GCNets and CoST for COVID-19 across horizons tab. 2; STGCN, DCRNN, StemGNN, AGCRN, GraphWaveNet, MTGNN, Informer and CoST on METR-LA app. G.