Discrete cosine transform (DCT)

Overview

The discrete cosine transform (DCT) writes a finite sequence as a weighted sum of cosines at increasing frequencies. For a window of T samples it is a fixed T × T matrix, built once and applied to the data. A learned model can then work on the frequency coefficients instead of the raw values, and an inverse transform returns the result to the original domain. The only card here uses the DCT along the time axis of human pose sequences; the paper credits an earlier motion-prediction work with the same use 2207.01567 §3.1. The matrix entries and their scale are defined in 2207.01567 eq. 1.

Variants

Only one variant has a card in this wiki. Other uses of the DCT, such as the image-forensics uses listed on Frequency-domain learning, are not covered here.

DCT over the time axis of pose sequences

What it is: the window of observed poses is viewed as C channels, one per joint coordinate, each a sequence of T = 50 frames. The DCT matrix is applied along the frame axis, so every channel becomes T frequency coefficients. The network acts on these coefficients, and the inverse transform maps the output back to frames 2207.01567 §3.1 2207.01567 §4.2. Card: Back to MLP: A Simple Baseline for Human Motion Prediction.

Key mechanism: the time-mixing layers are T × T, so the transform and the layers share the same window length. The network produces T frames, and the prediction horizon is taken as the first few of them 2207.01567 §3.2.

Reported effect: the paper's ablation without the DCT reaches 9.9 mm at 80 ms and 110.5 mm at 1000 ms on Human3.6M, against 9.6 mm and 109.4 mm with it 2207.01567 tab. 6. The differences, computed from that table, are 0.3 mm and 1.1 mm.

When to use which

Implementation sketch

Our own pseudocode of the time-domain use in siMLPe. The shapes follow the paper's setting: T = 50 frames and C = 66 channels for 22 joints in 3D 2207.01567 §4.2.

# Fixed DCT along time  [arxiv:2207.01567 §3.1] [arxiv:2207.01567 eq. 1]
# Input:  window x of shape (T, C), T = 50, C = 66
# Output: a prediction of shape (T, C) in the time domain

# Build the T x T basis once, for this window length  [arxiv:2207.01567 eq. 1]
D = dct_matrix(T)                    # (T, T), fixed, not learned
D_inv = inverse(D)                   # (T, T), maps coefficients back

# Move the window into the frequency domain  [arxiv:2207.01567 §3.1]
coeffs = D @ x                       # (T, C) frequency coefficients

# Learned layers act on the coefficients; T is fixed in their shapes
# [arxiv:2207.01567 §3.2]
coeffs = learned_layers(coeffs)      # (T, C)

# Return to the time domain  [arxiv:2207.01567 §3.1]
y = D_inv @ coeffs                   # (T, C)

Where variants would differ: the basis itself (here a cosine basis of size T), and whether the learned layers see the coefficients or the frames. Only the first is covered by a card.

Contradictions

None known. Only one card uses the DCT. Within that paper, the ablation shows a small gain from the DCT 2207.01567 tab. 6, and no other card reports a different result.

Papers

• Back to MLP: A Simple Baseline for Human Motion Prediction: applies a fixed 50-point DCT along time to observed poses before its time-mixing layers, and reports a small gain over the same model without it.