Wavelet transform

Overview

A wavelet transform writes a function as a coarse summary plus detail terms at each scale. The variant on file is a multiwavelet basis: piecewise polynomials of degree below k on dyadic intervals form nested spaces V_n, and each level adds a wavelet space W_n orthogonal to V_n 2109.13459 §2.2 eq. 2 eq. 3.

The filter matrices H and G map the finer scaling coefficients to the next coarser scaling coefficients s and detail coefficients d eq. 6 eq. 7. The inverse uses the same filters, corrected by Gram matrices Sigma, which are the identity for the Legendre basis eq. 8 eq. 9 app. C.1. In 2D and 3D the filters are Kronecker products of the 1D ones 2109.13459 §2.2.

The paper's premise is that the kernel of a smooth integral operator, apart from finitely many singularities, is sparse in this basis 2109.13459 §2.1 §2.4. The model then learns a small kernel for each scale 2109.13459 §2.3.

The wiki holds one paper for this concept, so this page describes the transform as that paper uses it.

Variants

Legendre multiwavelets

The polynomials are orthogonal with respect to the uniform measure on [0, 1] 2109.13459 app. C.2. Sigma is the identity app. C.1, so the reconstruction filters equal the decomposition ones. The paper's headline model uses this basis 2109.13459 §3. Its reported error is the lowest of the listed baselines on KdV, Burgers, Darcy and Navier-Stokes tab. 1 tab. 2 tab. 3 tab. 7. Card: Multiwavelet-based Operator Learning for Differential Equations.

Chebyshev multiwavelets

The polynomials are orthogonal with respect to the weight 1/sqrt(1 - (2x-1)^2) on [0, 1] 2109.13459 app. C.3. The paper's eq. 8 and eq. 9 put Sigma on the reconstruction side only eq. 8 eq. 9. The paper extends multiwavelets to non-uniform measures, which is what allows this basis 2109.13459 §2.2 app. B. In the card's tables the Chebyshev model has a higher error than Legendre at every resolution listed tab. 1 tab. 2 tab. 7. Against FNO it is mixed: lower error on the KdV and Darcy tables, but higher on Burgers at s = 256 tab. 7 and on Navier-Stokes at nu = 1e-5 tab. 3.

Random filters (control)

The paper's control, MWT Rnd, uses random filters. Its Darcy error is about 0.24 at every resolution tab. 2, far above the fixed filters. The paper's claim that random filters do not learn is not checked by the authors' code, which does not implement this row (see the card).

When to use which

The paper's Legendre headline model uses 1000 training and 200 test samples §3.

Implementation sketch

Language-agnostic core of one decomposition and one reconstruction step. The filters H0, H1, G0, G1 come from the Legendre or Chebyshev construction app. C.2 app. C.3. The learned operator acts between the two steps eq. 11.

# One decomposition step, fine to coarse  [eq. 6] [eq. 7]
# Input:  s_fine (2M, k) scaling coefficients on the finer grid
# Output: s (M, k) coarse scaling, d (M, k) wavelet coefficients

def decompose(s_fine):
    # Even and odd fine nodes, paired as in eq. 6 and eq. 7
    s_even = s_fine[0::2]
    s_odd = s_fine[1::2]
    # Coarse scaling coefficients from both halves  [eq. 6]
    s = s_even @ H0.T + s_odd @ H1.T
    # Wavelet (detail) coefficients  [eq. 7]
    d = s_even @ G0.T + s_odd @ G1.T
    return s, d

# One reconstruction step, coarse to fine  [eq. 8] [eq. 9]
# Input:  s (M, k), d (M, k)
# Output: s_fine (2M, k)

def reconstruct(s, d):
    # Even and odd fine nodes, each corrected by its Sigma  [eq. 8] [eq. 9]
    s_even = (s @ H0 + d @ G0) @ Sigma0.T
    s_odd = (s @ H1 + d @ G1) @ Sigma1.T
    # Interleave the two sets back onto the fine grid
    return interleave(s_even, s_odd)

Implementation notes:

• The paper's eq. 7 is printed with H^(1) in the second term. The matrix form in the appendix uses G^(1) app. C.1, and the sketch above uses G1. Use G1 2109.13459 §2.2 app. C.1.
• Between the two steps, a learned operator acts on d and s: U_d = A d + B s, U_s = C d, with a learned coarsest-scale map eq. 11. The paper's operator is written this way, and the paper reuses A, B and C at every scale 2109.13459 §2.3.
• For Chebyshev, keep the decomposition on the raw H and G, and apply Sigma only in reconstruction eq. 8 eq. 9. Compute Sigma and G with the quadrature rather than copying the four-decimal values in the appendix app. C.3.

Contradictions

• Filter order for Navier-Stokes. The paper states k = 4 for all experiments §3. Its Navier-Stokes appendix uses k = 3 app. D.1.
• Kernel type, paper and code. The paper describes A, B and C as single-layer CNNs following a linear layer §2.3 §3, and for Navier-Stokes a single-layer 3D convolution app. D.1. The authors' 1D code uses Fourier-domain kernels for all three models.py. The 2D and 3D code uses spectral kernels for A and convolutions for B and C models.py. Implement the code's version if you want to match its numbers, and say so.
• Headline numbers. The abstract quotes 0.0033 for KdV and 0.0018 for Burgers. The tables give 0.00338 to 0.00418 for KdV tab. 1 and 0.00178 to 0.00199 for Burgers tab. 7.
• Burgers at s = 2048. Section 3.5 gives 0.00189, and the table gives 0.00186 §3.5 tab. 7.

Papers

• Multiwavelet-based Operator Learning for Differential Equations: a multiwavelet decomposition with Legendre or Chebyshev filters and learned kernels per scale, compared with FNO and other neural operators on PDE benchmarks.