Frequency-domain learning

Overview

A learned layer works on the discrete Fourier transform (DFT) of its input instead of on the raw values, then maps the result back to the original domain. In the time-series variants the invariant is that the output keeps the input's shape, so the change sits inside the network. In the DFT basis, each coefficient describes the whole signal at one frequency, so a learned mix of coefficients acts on the entire window at once. The paper notes that a frequency-domain layer with complex weights is equivalent to a global circular convolution in time 2311.06184 §3.2 eq. 9. The authors argue that point-wise MLPs model temporal mappings poorly and that a series' energy concentrates in a few frequency components 2311.06184 §1 §3.2 fig. 1. CoST uses the same kind of learned spectral layer, but inside a seasonal branch of a contrastive encoder, where the period need not be known in advance 2202.01575 §3.2 2202.01575 §1. FourierGNN applies the transform to every value of a multivariate window at once, treating the window as one graph 2311.06190 §4.1 §4.2. Two other cards use frequency information without a learned spectral layer. ETSformer keeps the K largest Fourier bases of a residual and carries them into the horizon 2202.01381 §4.2.2. The face forgery detector FDFL feeds fixed DCT coefficients of 8x8 image blocks into a learned branch next to an RGB branch 2103.09096 §3.2. For the neighbouring operator-learning idea, see Fourier neural operator.

Variants

Full frequency-domain forecaster (FreTS)

Two learners run on the DFT of a hidden representation of a multivariate window. One runs along the channel axis and one along the time axis. Each applies a frequency-domain MLP to the spectrum and inverts the transform. A two-layer feed-forward head then outputs the whole horizon in one step, avoiding autoregressive error build-up 2311.06184 §3.1 eq. 3 eq. 4 eq. 5. Card: Frequency-domain MLPs are More Effective Learners in Time Series Forecasting.

• Short-term, horizon 12: MAE 0.120 on Solar and 0.011 on Traffic, the lowest in that table 2311.06184 tab. 1.
• Long-term: the lowest MAE on most rows of the long-term table, with PatchTST ahead on a few Traffic and Electricity rows 2311.06184 tab. 2. See Contradictions.
• Ablation: removing either learner worsens MAE on all four settings the authors tested (Electricity, METR-LA, Exchange, Weather) 2311.06184 tab. 3.
• Efficiency: the authors state complexity O(N log N + L log L) and report fewer parameters and faster training than several Transformer forecasters 2311.06184 §4.3. These figures are the authors' own.

Frequency-domain MLP as a drop-in for linear forecasters

Replace the linear layer in an existing linear forecaster (DLinear or NLinear) with the frequency-domain MLP, and keep the rest of the model 2311.06184 §4.2 tab. 4. The gains are small. For example, DLinear on Exchange at horizon 96 moves from 0.037 to 0.036 MAE, and on Weather at horizon 96 from 0.041 to 0.038 MAE 2311.06184 tab. 4. The paper runs one seed and reports no variance, so these gaps have no error bars.

Learnable Fourier seasonal branch in a contrastive encoder (CoST)

CoST pretrains a dilated temporal convolution encoder that outputs two parts: a trend part and a seasonal part. The trend part is trained with a MoCo-style time-domain contrastive loss 2202.01575 §3.1. The seasonal part goes through a DFT over time, then a learnable Fourier layer, then an inverse DFT back to the time domain. The layer has a separate complex affine map for each frequency bin. Unlike the single d-by-d weight of the FreTS layer 2311.06184 §3.2 eq. 6, CoST's weights depend on the bin index, so the layer is not translation invariant by design 2202.01575 §3.2. Each bin of the seasonal output is split into amplitude and phase, and both are trained with a contrastive loss in which the other series in the batch are negatives 2202.01575 §3.2. After pretraining, the encoder is frozen and a ridge regressor forecasts from its features 2202.01575 §4.1. Card: CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting.

• Multivariate average MSE / MAE 0.590 / 0.524, against 0.750 / 0.607 for TS2Vec and 0.972 / 0.666 for an end-to-end TCN, author-reported 2202.01575 §4.2 tab. 1.
• Univariate average MSE / MAE 0.193 / 0.283, against 0.203 / 0.298 for TS2Vec 2202.01575 tab. 7.
• Component ablation, ETT averages, multivariate MSE: the learnable Fourier layer alone 0.895, the frequency loss alone 0.905, both together 0.862, and the full model with the trend branch 0.781 2202.01575 tab. 3. The table has no seasonal-only row without these two pieces, so it does not isolate the seasonal branch's own contribution.
• End-to-end supervised training of the same architecture gives 1.376 multivariate MSE in the same table, against 0.781 for the frozen contrastive encoder with ridge 2202.01575 tab. 3. This is one comparison in one setting, reported by the authors.
• The frequency-loss weight alpha is 5E-04 in the other experiments. On two ETTh2 horizons, the largest value tested (0.1) gives the lowest multivariate MSE 2202.01575 tab. 2.

Fourier graph operator over a space-time graph (FourierGNN)

FourierGNN does not transform each series separately. It takes a lookback window of N series and T steps, makes every scalar value a node of one fully connected graph with N·T nodes, embeds each node into d dimensions, and applies a DFT over all nodes 2311.06190 §4.1 §4.3. Each layer multiplies the spectrum by one complex d-by-d matrix that is shared by all frequencies, which the paper calls an n-invariant operator. Layers are summed over K orders, with K = 3 as the default 2311.06190 §4.2 eq. 6 app. F.4. The authors argue that this avoids both the quadratic cost of classic graph layers and the need to pair a graph network with a separate temporal network 2311.06190 §4.1 §4.2. Card: FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective.

• Author-reported horizon 12 on the main table: the lowest MAE on all six datasets (Solar 0.120, Wiki 0.041, Traffic 0.011, ECG 0.052, Electricity 0.051, COVID-19 0.123) 2311.06190 tab. 1. The paper's MAPE column does not show the same lead on every dataset. See Contradictions.
• Ablation on METR-LA, horizon 12: MAE 0.050 for the full model, against 0.053 to 0.055 when the embedding, the dynamic operator, the residual or the summation is removed 2311.06190 tab. 4.
• Cost: on Traffic it has 190,564 parameters, against 280,860 for GraphWaveNet, which the authors report as 32.2% fewer 2311.06190 tab. 3 §5.3. Sharing one matrix across frequencies is what keeps this small. On ECG, the per-frequency version has 82.96M parameters, against 0.18M for the shared one 2311.06190 app. C.2.

Top-K Fourier bases for seasonality (ETSformer frequency attention)

ETSformer replaces Transformer self-attention with two blocks: exponential smoothing attention for growth, and frequency attention for seasonality. The frequency block takes a DFT of the residual along time, skips the zero frequency, keeps the K bases with the largest amplitude in each dimension, and rebuilds them as cosines that run over the window and continue into the horizon 2202.01381 §4.2.2 eq. 4. Bases are chosen by amplitude from the input's own spectrum 2202.01381 §4.2.2. Card: ETSformer: Exponential Smoothing Transformers for Time-series Forecasting.

• Author-reported: the lowest MSE on 22 of the 24 multivariate rows of the main table, across ETT, ECL, Exchange, Traffic, Weather and ILI 2202.01381 tab. 1.
• Ablation at horizon 24: removing the seasonal part is the worst variant on ETTh2, ECL and Traffic, while removing the level is the worst on ETTm2 2202.01381 tab. 2.
• Interpretability: on a synthetic series with known trend and season, the season output follows the ground truth. The authors call this qualitative 2202.01381 §5.3.
• Code: the released repository drops the Nyquist bin for even lengths and divides amplitudes by the window length, which the paper's equations do not show encoder.py encoder.py.

DCT-coefficient branch for face forgery detection (FDFL)

This is not a time-series method, and its frequency step is fixed, not a learned spectral layer. The RGB image is converted to YCbCr. A 2D DCT is taken of each 8x8 block of each channel, and coefficients with the same frequency index are gathered into one channel. This gives 192 channels at one-eighth resolution 2103.09096 §3.2 fig. 4. A learned block of grouped and ordinary convolutions with channel attention reads these channels. Its output is fused with an Xception RGB branch by a 1x1 convolution, between the entry and middle flows 2103.09096 §4.1 §4.2.2 tab. 2. Card: Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection.

• Author-reported, FF++ c40: video AUC 0.924 and pAUC(0.1) 0.810, against 0.861 and 0.652 for the paper's Xception baseline 2103.09096 tab. 4. The full method also adds a single-center loss, which the paper treats as a separate contribution §3.3.
• Frequency branch alone, with softmax loss only: AUC 0.906 and pAUC(0.1) 0.769 against the same baseline 2103.09096 tab. 3. This row is the only one in the paper that isolates the frequency branch from the loss.
• Fusion: with softmax loss only, a 1x1 convolution with groups = 1 has the best AUC and pAUC of six fusions tried 2103.09096 tab. 2.
• The paper says the DCT front end may suit compressed images, but does not test this 2103.09096 §3.2. The paper tests only face forgery on FF++.
• Training settings (optimizer, learning rate, batch size, epochs) are not in the published text 2103.09096 §4.1. An implementation must choose them and report the choice.

When to use which

Implementation sketch

Core operations shared by the variants: a real DFT, a learned layer on the spectrum, and the inverse transform. The full FreTS model applies this block along two axes. Its learned layer uses complex weights and biases.

# Frequency-domain MLP layer on a complex spectrum  [arxiv:2311.06184 §3.2] [eq. 6]
# S: (B, F, d) complex; W: (d, d) complex; b: (d,) complex
def freq_mlp(S, W, b):
    # Complex matrix product plus bias  [eq. 6]
    out = S @ W + b
    # The activation sigma is unnamed in the paper. The released code applies
    # ReLU to real and imaginary parts separately  [arxiv:2311.06184 §3.2]
    return sigma(real(out)) + 1j * sigma(imag(out))

# One frequency-domain block along the time axis  [arxiv:2311.06184 §3.1] [eq. 4]
# x: (B, L, d) real; output: (B, L, d) real
def freq_block_time(x, W, b):
    # Real DFT along time  [arxiv:2311.06184 §3.1] [eq. 1]
    S = rfft(x, axis=time)              # (B, L//2+1, d) complex
    # Learned layer on the spectrum  [eq. 6]
    S = freq_mlp(S, W, b)
    # Inverse DFT back to L steps  [arxiv:2311.06184 §3.1] [eq. 2]
    return irfft(S, n=L, axis=time)     # (B, L, d) real

For the channel learner, apply the same block along the channel axis, with its own weights, and keep the time axis fixed 2311.06184 §3.1 eq. 3. The paper's equations have no residual term around the blocks, and the released code adds one. See Contradictions.

CoST's seasonal branch replaces the single weight with one matrix per bin. The paper's equation for the layer is in 2202.01575 §3.2.

# Learnable Fourier layer, one complex affine map per bin  [arxiv:2202.01575 §3.2]
# V: (B, T, d) real; A: (F, d, d_S) complex; b: (F, d_S) complex
def learnable_fourier_layer(V, A, b):
    # Real DFT along time, F = T//2 + 1 bins  [arxiv:2202.01575 §3.2] [app. B]
    S = rfft(V, axis=time)                       # (B, F, d) complex
    # Separate complex affine map for each bin i  [arxiv:2202.01575 §3.2]
    out = einsum('bid,idk->bik', S, A) + b       # (B, F, d_S) complex
    # Inverse DFT back to T steps  [arxiv:2202.01575 §3.2]
    return irfft(out, n=T, axis=time)            # (B, T, d_S) real

FourierGNN uses the same DFT-learn-inverse pattern, but its transform runs over the flattened N·T node axis, and it stacks a shared-weight layer K times with a sum over orders 2311.06190 §4.2 eq. 6. Its released code uses diagonal weights, not the full matrix in the equations. See Contradictions.

Contradictions

The cards answer different questions, so their numbers are not comparable. FreTS reports MAE and RMSE from an end-to-end forecaster 2311.06184 tab. 2. CoST reports MSE and MAE from a frozen encoder with a ridge head 2202.01575 tab. 1. FourierGNN reports MAE, RMSE and MAPE from an end-to-end model 2311.06190 tab. 1. FreTS and FourierGNN use overlapping benchmarks (Solar, Wiki, Traffic, ECG, Electricity and COVID-19 in the FreTS short-term table 2311.06184 tab. 1 and the FourierGNN main table 2311.06190 tab. 1), but their horizons, baselines and training protocols differ, so the two sets of numbers should not be read against each other. Within several papers, the text and the code disagree with the paper's own equations or tables:

• FreTS, text versus tables. The text says FreTS beats every baseline on every long-term dataset 2311.06184 §4.1. In the long-term table, PatchTST has lower MAE and RMSE on Traffic at horizons 48 and 96, and on Electricity at horizon 336 2311.06184 tab. 2.
• FreTS, equations versus released code. The equations use a full complex d-by-d weight matrix 2311.06184 §3.2 eq. 6. The released code multiplies each feature by a single diagonal entry, so only diagonal weights are trained. The channel learner is never executed in the released code, so the channel-learner results in 2311.06184 tab. 3 cannot be reproduced from it. The code also adds a residual connection and a soft-shrink step that the equations do not show. The card's Paper vs code section lists these differences.
• FourierGNN, text versus tables. The text describes a new state of the art on all datasets 2311.06190 §5.2. The paper's own tables show lower MAPE for other models on Solar (FEDformer and LSTNet) and on Traffic (LSTNet) 2311.06190 tab. 1, and at horizons 3, 6 and 12 on COVID-19, where TAMP-S2GCNets is lower 2311.06190 tab. 2.
• FourierGNN, equations versus released code. The equations use a full complex d-by-d weight matrix per layer 2311.06190 §4.2 eq. 6. The released code multiplies only along the diagonal, so each feature has one complex scalar per layer FourierGNN.py. The code also applies the activation inside the recursion, adds a soft-shrink step the paper does not describe, and builds no adjacency matrix. It applies ReLU to the real and imaginary parts separately, and it fixes the time compression at 8 for every dataset FourierGNN.py FourierGNN.py. The card's Paper vs code section lists these differences.
• FreTS and FourierGNN, identical numbers. Both papers, from the same first author, report MAE 0.120 on Solar, 0.011 on Traffic and 0.041 on Wiki at horizon 12 for their model 2311.06184 tab. 1 2311.06190 tab. 1. Their Wiki RMSE differs (0.074 and 0.076). The cards do not say whether the two rows come from the same run or setup, so this should be checked before either set of numbers is quoted as a reproduction.
• CoST, equations versus released code. The paper computes amplitude and phase on the complex output before the inverse FFT 2202.01575 §3.2. The code takes the FFT of the time-domain seasonal output, after L2 normalisation and dropout cost.py encoder.py.
• CoST, negatives in the frequency loss. The paper uses the other N-1 original samples as negatives 2202.01575 §3.2. The code uses all other 2B-1 items, including the other augmented view cost.py.
• CoST, text versus table. The text gives a 4.71% univariate gain over the best feature-based baseline 2202.01575 §4.2. The rounded averages in 2202.01575 tab. 7 give about 4.9%.
• ETSformer, text versus table. The text says ETSformer is best on 35 of 40 multivariate settings and within the top two on all 40 2202.01381 §5.1. The main table shows 24 settings, and the other 16 are not in the source, so the count cannot be checked here.
• FDFL, text versus table. The text gives a 6.4% AUC improvement on c40 2103.09096 §4.3. The table values 0.924 and 0.861 differ by 6.3 points 2103.09096 tab. 4. The softmax-only baseline AUC is 0.860 in one table and 0.861 in another 2103.09096 tab. 1 tab. 3.

Papers

• Frequency-domain MLPs are More Effective Learners in Time Series Forecasting: FreTS, a forecaster with two frequency-domain MLPs (channel and time) and a drop-in frequency-domain MLP for linear models.
• CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting: a contrastive encoder whose seasonal branch is a learnable Fourier layer trained with a frequency-domain loss; a ridge regressor forecasts from the frozen features.
• FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective: a space-time graph of N·T value nodes, with stacked Fourier graph layers whose shared complex weights act on the spectrum over all nodes.
• ETSformer: Exponential Smoothing Transformers for Time-series Forecasting: a frequency attention block that keeps the top-K Fourier bases of a residual for seasonality, inside an exponential-smoothing Transformer.
• Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection: a DCT-coefficient branch on YCbCr 8x8 blocks, fused with an RGB branch, for face forgery detection.