---
kind: concept
slug: neural-operator
title: Neural operator
summary: >-
  Learns a map from an input function, such as PDE coefficients or a blade shape, to an output field, so one model is not tied to one grid. Cards: a multiwavelet operator that beats FNO on PDE tables, a branch-trunk operator with lower field error than FNO on one compressor dataset, and DeepONet uncertainty bars. Gains depend on the baseline; finer grids lose accuracy.
tags: [neural-operator, pde, operator-learning, wavelets, cfd-surrogate, transformer, uncertainty-quantification]
papers: ["2109.13459", "2201.07766", "2609.16066"]
related: [fourier-neural-operator, self-attention, wavelet-transform, physics-informed-neural-network, uncertainty-estimation, calibration]
status: draft
updated_at: 2026-10-24
---

## Overview

A neural operator learns a map between functions rather than between fixed
vectors. For a partial differential equation, the input is a function such as
an initial state or a material coefficient, and the output is the solution
function [arxiv:2109.13459 §2.1]. Data are point values of both on a grid
[arxiv:2109.13459 §3]. In the compressor card the input is a blade shape and
the output is a set of flow fields [arxiv:2609.16066 §3.3].

Earlier networks for this task were tied to one grid resolution or one PDE
instance, and had to be retrained when either changed [arxiv:2109.13459 §1].
Neural operators avoid this by learning the map itself, without fixing the
grid. The shared invariant is that the learned weights describe the map, not
the values at particular grid points, so the same model can be queried at a
different resolution [arxiv:2109.13459 §1] [arxiv:2109.13459 §2.3]. The
authors argue that existing neural operators still do not exploit the
structure of the operator's kernel [arxiv:2109.13459 §2.4].

Three cards in this wiki carry this slug: the multiwavelet card, the
transformer-based compressor card and the uncertainty card, which wraps a
DeepONet rather than defining a new operator. Other neural operators appear
only as baselines, with the numbers the cards report.

## Variants

### Multiwavelet operator (MWT)

The input is split into scales with a fixed multiwavelet decomposition built
from orthogonal polynomials. The filters are computed, not learned
[arxiv:2109.13459 §2.2] [arxiv:2109.13459 app. B]. A small learned kernel acts
on the detail and coarse terms at each scale, and the same kernels are reused
at every scale [arxiv:2109.13459 §2.3]. There are two polynomial choices. The
Legendre basis is the headline variant. The Chebyshev basis needs an extra
correction matrix in reconstruction [arxiv:2109.13459 app. C.2]
[arxiv:2109.13459 app. C.3].

The paper's premise is that the operator kernel is sparse in this basis when it
is smooth away from its singularities. The argument covers smooth kernels only
[arxiv:2109.13459 §2.4]. Card:
[Multiwavelet-based Operator Learning for Differential Equations](/papers/2109.13459).

### Branch-trunk operator with Galerkin attention (TNO)

This variant follows the branch-trunk framework of deepONet, which the card
credits to Lu et al. [arxiv:2609.16066 §1] [arxiv:2609.16066 §3.1]. A branch
network encodes the design variables, here 28 blade-geometry values. A trunk
network encodes each query point (r, z) on a meridional plane. The two feature
vectors are combined and passed to a shared output network that returns five
flow fields at that point [arxiv:2609.16066 eq. 1] [arxiv:2609.16066 §3.1].
The trunk is a stack of attention blocks in which the attention is
Galerkin-type rather than softmax. Keys and values are layer-normalised, and
the keys and values are summarised over all points before the queries are
applied [arxiv:2609.16066 §3.1] [arxiv:2609.16066 eq. 3]. The card gives the
recurrence of these blocks [arxiv:2609.16066 eq. 2].

Because the trunk takes query coordinates as input, the output is evaluated
point by point. The card trains and tests on 4096 points per field and does
not test a different point count [arxiv:2609.16066 §3.3]. Card:
[Compressor-cascade performance from a transformer-based operator](/papers/2609.16066).

Performance numbers are computed from the predicted fields rather than learned
separately. The card averages the predicted field over the span at two axial
stations and applies the metric formulas to the averages [arxiv:2609.16066 §3.2]
[arxiv:2609.16066 tab. 6].

### Uncertainty for DeepONets (PA-BNN-FP, PA-GAN-FP)

This is not a new operator layer. It wraps a DeepONet, which has a branch
network on the sampled input function and a trunk network on the query point,
and whose output is their inner product [arxiv:2201.07766 §2.2]. The predictive
variance adds a noise term to the spread over posterior samples
[arxiv:2201.07766 §3.1].

For operator learning the DeepONet is pre-trained on clean data. At inference
it is given a few noisy points, and a prior is placed over it. The
physics-agnostic variants use a Bayesian neural network prior (PA-BNN-FP) or a
GAN generator prior (PA-GAN-FP) [arxiv:2201.07766 §4.3] [arxiv:2201.07766 §4.5.2].
With a GAN prior, inference runs on a low-dimensional latent input
[arxiv:2201.07766 §4.3].

The Darcy test uses 20 noisy λ points and 10 noisy u points at inference
[arxiv:2201.07766 §6.5]. In the paper's ×10² units, the GAN prior is more
accurate and the BNN prior is better calibrated. For u in distribution the
relative L2 error is 7.7 for the GAN prior and 14.6 for the BNN prior, and the
RMSCE is 2 for the BNN and 8.6 for the GAN. Out of distribution the errors are
35.5 and 48, and the RMSCE is 9.7 for the BNN and 20.3 for the GAN
[arxiv:2201.07766 §6.5]. The card says the BNN variant costs nothing extra
[arxiv:2201.07766 §6.5]. These are results for one Darcy problem.

Post-training calibration is a separate step. The card's example, on a mixed
PDE rather than an operator, cuts RMSCE by about a factor of 3 with 2 to 10
left-out realisations in one case [arxiv:2201.07766 §6.3]. Calibration uses
training-distribution data and may not help out of distribution
[arxiv:2201.07766 §5.2]. Card:
[Uncertainty Quantification in Scientific Machine Learning](/papers/2201.07766).

### Baselines named in the cards

The MWT card's tables also report four other neural operators: GNO, MGNO, LNO
and FNO [arxiv:2109.13459 §3]. LNO is described there as a low-rank kernel,
similar to an unstacked DeepONet [arxiv:2109.13459 §3]. The card gives no
description of the others. Their papers have no card in this wiki, so this
page uses only the numbers the MWT card reports for them. FNO's own layer is
covered on [Fourier neural operator](/concepts/fourier-neural-operator).

The TNO card compares against MLP, UNet, deepONet and FNO, all trained on the
same fields [arxiv:2609.16066 §4.1] [arxiv:2609.16066 tab. 4]. Those baselines
have no card in this wiki either, so this page cites only the TNO card's
numbers for them.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Smooth PDE data, and the grid may change between training and use | MWT, Legendre variant | Lowest error of the listed models on the card's KdV, Darcy and Burgers tables: 0.00338 at s=64 against 0.0125 for FNO [arxiv:2109.13459 tab. 1]. Transfer to finer grids loses accuracy (see below). |
| Chebyshev basis is required | MWT, Chebyshev variant, with the Sigma correction | Weaker than Legendre at every resolution in the card's tables, though it beats FNO on KdV and Darcy [arxiv:2109.13459 tab. 1] [arxiv:2109.13459 tab. 2]. |
| Training grid is much coarser than the test grid | Train at higher resolution where possible | On Burgers, a model trained at s=128 has test error 0.0368 at s=2048 and 0.0456 at s=8192. A model trained at s=512 has 0.0140 and 0.0241 at the same test sizes [arxiv:2109.13459 tab. 4]. |
| Learned filters are not an option | None in this card | With random filters the error stays near 0.243 at every Darcy resolution [arxiv:2109.13459 tab. 2]. |
| Discontinuous or singular kernels | Not covered by the MWT argument | The sparsity premise assumes smooth kernels away from the diagonal [arxiv:2109.13459 §2.4]. |
| Field predictions for many metrics from one model, with a few thousand CFD runs | TNO | Mean field error on the validation split is 0.083% against 0.572% for FNO and 0.820% for MLP [arxiv:2609.16066 tab. 4]. One predicted field gives several metrics [arxiv:2609.16066 §3.2]. Single dataset, author-reported. |
| Small training sets for a performance metric | TNO, with caution | On the efficiency row, TNO has the lowest error at every size from 500 to 2500 samples. On pressure ratio and mass flow it is not lowest at 500 samples [arxiv:2609.16066 tab. 8]. The card gives no run below 500 samples. |
| Operator learning from a few noisy inference points, with error bars | PA-GAN-FP or PA-BNN-FP | The GAN prior has the lower relative error (u in distribution 7.7 against 14.6); the BNN prior has the better RMSCE (2 against 8.6) [arxiv:2201.07766 §6.5]. One problem, author-reported. |
| Accuracy matters more than calibration on inputs unlike the training set | PA-GAN-FP, with caution | Out of distribution the GAN prior has lower error (35.5 against 48) but worse RMSCE (20.3 against 9.7) [arxiv:2201.07766 §6.5]. |
| Pre-training inputs are noisy | Not supported in this card | DeepONet pre-training needs clean λ; noisy λ is listed as future work [arxiv:2201.07766 §3.3] [arxiv:2201.07766 §6.5]. |
| Need a reproducible reference implementation | No card gives one | The MWT code has no license file. The TNO card records no code and the data are available only on request [arxiv:2609.16066 §5]. The uncertainty card found no official code [arxiv:2201.07766 §6.6]. |

Implementation: the MWT code has no license file, so copy nothing from it. The
method can be written from the paper's equations and pseudocode
[arxiv:2109.13459 §2.2] [arxiv:2109.13459 §2.3]. The TNO card also records no
code, so write from its equations. Several TNO details are not stated, listed
below.

## Implementation sketch

The MWT and FNO layers stack operator layers between a lift and a projection.
The TNO variant does not. It has a branch network on the design variables and a
trunk network on the query points, and combines them. The training loss for
MWT is the relative L2 error [arxiv:2109.13459 §3]. The TNO card uses a mean
squared error on normalised fields [arxiv:2609.16066 §3.3].

```pseudocode
# Neural operator, stacked-layer outline  [arxiv:2109.13459 §2.3] [§3]
# Input:  samples a_i = a(x_i) on an N-point grid  [arxiv:2109.13459 §3]
# Output: predicted solution u on the same grid

def neural_operator(a):
    # Map the input to a wider channel space; the paper does not name this step
    x = lift(a)
    for layer in range(num_layers):
        # Variant-specific: MWT splits scales, applies learned kernels
        # per scale and reconstructs  [arxiv:2109.13459 §2.3]
        x = operator_layer(x)
        # Non-linearity between layers  [arxiv:2109.13459 §2.3]
        if layer < num_layers - 1:
            x = relu(x)
    # Map back to one value per grid point
    return project(x)

def relative_l2(u_pred, u_true):
    # Loss and metric  [arxiv:2109.13459 §3]
    return norm(u_pred - u_true) / norm(u_true)
```

Training settings reported for the MWT runs: Adam, learning rate 1e-3, 500
epochs [arxiv:2109.13459 §3]. Settings reported for the TNO runs: Adam, 900
epochs, learning rate 1e-3 for the first 800 and 1e-4 for the last 100
[arxiv:2609.16066 §3.3].

Where the variants differ:

- **Uncertainty.** The PA variants keep a DeepONet and add a prior over it.
  The Darcy runs use HMC for both PA variants and deep ensembles for
  U-DeepONet [arxiv:2201.07766 §6.5]. The card gives no pseudocode, so write
  this part from its method section [arxiv:2201.07766 §4.3].
- **Scale handling.** MWT replaces the layer with a decomposition, per-scale
  kernels and a reconstruction [arxiv:2109.13459 §2.3]. For the spectral
  alternative, see the [Fourier neural operator](/concepts/fourier-neural-operator)
  page.
- **Branch and trunk.** TNO has no stacked layers. Its branch and trunk are
  combined per query point, and the card's equation 1 passes the products of
  the branch and trunk features into the output network [arxiv:2609.16066 eq. 1].
  The text calls the combination a dot product and also describes a cross
  multiplication [arxiv:2609.16066 §3.1]. See Contradictions.
- **Not stated for TNO.** The card does not give the trunk's number of blocks,
  the embedding, the feed-forward width, the branch depth and width, the
  feature length p, or the number of attention heads. Its 154 thousand
  parameter count can serve as a check on a reimplementation
  [arxiv:2609.16066 tab. 3] [arxiv:2609.16066 §3.1].
- **Grid size.** The card's Pitfalls section records that the released MWT
  code needs grids whose size is a power of two. Check this before relying on
  resolution independence [arxiv:2109.13459 §2.3]. Card:
  [Multiwavelet-based Operator Learning for Differential Equations](/papers/2109.13459).
- **Filter precision.** The paper recommends k between 1 and 6, and reports
  numerical drift in the filters above k = 20 [arxiv:2109.13459 app. C.4].

## Contradictions

- **Size of the gain over FNO.** The paper calls the improvement "nearly an
  order of magnitude". The card's tables show 3.7 times on KdV at s=64, 1.7
  times on Burgers at s=256 and 1.2 times on Navier-Stokes against FNO-3D
  [arxiv:2109.13459 §3.1] [arxiv:2109.13459 tab. 1] [arxiv:2109.13459 tab. 7]
  [arxiv:2109.13459 tab. 3]. The order-of-magnitude figure matches the gap to
  GNO, MGNO and LNO, not to FNO [arxiv:2109.13459 §3.3].
- **Abstract against tables.** The abstract quotes 0.0033 for KdV and 0.0018
  for Burgers. The tables give 0.00338 to 0.00418 and 0.00178 to 0.00199
  [arxiv:2109.13459 tab. 1] [arxiv:2109.13459 tab. 7].
- **Kernel type.** The paper describes the kernels as single-layer
  convolutions [arxiv:2109.13459 §2.3]. The released 1D code uses Fourier-domain
  maps for them. The card's Paper vs code section records this
  [arxiv:2109.13459 §2.3].
- **Baseline numbers.** The Burgers and Navier-Stokes FNO and other baseline
  numbers are taken from earlier published results, not re-run
  [arxiv:2109.13459 §3.3] [arxiv:2109.13459 app. D.1]. Those comparisons are
  not like-for-like. The FNO numbers in the TNO card are for a compressor
  dataset and are not comparable with the MWT card's PDE numbers
  [arxiv:2609.16066 tab. 4] [arxiv:2109.13459 tab. 1].
- **Speed-up over CFD.** The TNO paper states a speed-up of five orders of
  magnitude. Its own figures, about 800 s per CFD run against about 0.9 ms per
  prediction, give a ratio near 8.7 × 10^5, which is closer to six orders
  [arxiv:2609.16066 §2.2] [arxiv:2609.16066 tab. 3] [arxiv:2609.16066 §4.1.1].
  The prediction timing's hardware is not stated.
- **Branch and trunk combination.** Equation 1 passes the p products of branch
  and trunk features into the output network. The text writes a dot product
  and also a cross multiplication [arxiv:2609.16066 eq. 1]
  [arxiv:2609.16066 §3.1]. The card's Pitfalls section records this
  [arxiv:2609.16066 §3.1].
- **Surrogate comparison.** In the TNO card's comparison with conventional
  surrogates, the neural networks predict fields and derive metrics, while the
  conventional methods predict the metrics directly. The paper states this
  asymmetry [arxiv:2609.16066 §4.1.3] [arxiv:2609.16066 tab. 8].

## Papers

- [Multiwavelet-based Operator Learning for Differential Equations](/papers/2109.13459): a neural operator with a fixed multiwavelet split and learned kernels per scale, compared against GNO, MGNO, LNO and FNO on KdV, Burgers, Darcy and Navier-Stokes.
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](/papers/2201.07766): a framework that adds Bayesian methods, ensembles and GAN priors to PINNs and DeepONets, with several calibration-aware metrics and post-training calibration; the Darcy test uses noisy inference data.
- [Compressor-cascade performance from a transformer-based operator](/papers/2609.16066): a branch-trunk operator with Galerkin attention that predicts five flow fields for one transonic compressor rotor and computes performance metrics from them, compared with MLP, UNet, deepONet, FNO and conventional surrogates.