---
kind: concept
slug: score-matching
title: Score matching
summary: >-
  Trains a network to match the score, the gradient of the log density, by regressing onto per-sample targets built from noised data. Two theory papers analyse it in diffusion: a linear model with a closed-form symmetry error, and an online fresh-sample objective with a total-variation bound. Neither has code or benchmarks. Choose by whether you need an exact analysis or a training bound.
tags: [generative-models, diffusion, theory, sample-complexity, symmetry, stochastic-gradient]
papers: ["2610.35720", "2611.01562"]
related: [diffusion-models, noise-schedule, equivariance-constraint, weight-initialization, intrinsic-dimension, langevin-dynamics]
status: draft
updated_at: 2026-11-21
---

## Overview

Score matching fits a network to the score of a distribution, which is the
gradient of its log density. In diffusion models the score is taken at each
noise level, and the reverse process uses it to denoise a sample. The true
score is rarely available, so training regresses the network onto a per-sample
target that is built from a clean data point and fresh noise. The expectation
of that target equals the true score, which is why the regression is valid
even though each individual target is noisy [arxiv:2611.01562 §2]
[arxiv:2610.35720 §2.1].

Both papers in this concept use the same variance-preserving corruption and the
same target form, `-(x_t - alpha x_0) / sigma^2`, written with the noise
coefficients of each paper [arxiv:2611.01562 §2] [arxiv:2610.35720 §2.1]. They
differ in what they analyse. [Linear diffusion with symmetry](/papers/2610.35720)
studies one linear score model trained by gradient flow, which gives closed
forms. [Online conditional diffusion](/papers/2611.01562) studies a general
conditional network trained on fresh minibatches, which gives a bound.

## Variants

### Linear score model with a proxy target

The model is linear, `s(x, t) = W x`, and is trained by gradient flow on the
score-matching loss with an L2 penalty [arxiv:2610.35720 §2.1]. The per-sample
target is a proxy: its mean given `x` is the true score, but its covariance is
non-zero [arxiv:2610.35720 §2.1] [app. B]. The learned weights have a closed
form after training time `tau` [arxiv:2610.35720 §3]. The paper's error measure
is how far the weights break a rotational symmetry of Gaussian data
[arxiv:2610.35720 §2.2]. It splits the expected error into three terms: the
average weight, the random initialisation, and the target noise
[arxiv:2610.35720 §3] [app. C].

The initialisation term decays with training time. It vanishes once the
symmetric sector is learned [arxiv:2610.35720 §4] [§5]. The target-noise term
does not decay in the same way. For fixed ratio `gamma = D/N` below one and
vanishing regularisation, it stays non-zero, and it grows as `gamma` approaches
one [arxiv:2610.35720 §5] [§6.3]. The results are asymptotic and the paper runs
no experiments [arxiv:2610.35720 §7].

### Online objective with a randomized time index

The network is a time-gated sum of blockwise approximants, and its parameters
are updated by SGD on fresh minibatches, with no stored dataset
[arxiv:2611.01562 §2]. Each step draws a clean point, a condition, a grid index
and Gaussian noise. The grid index is sampled with probability proportional to
its step length, so the time average matches the stability argument
[arxiv:2611.01562 §2]. The population loss equals a constant plus the squared
distance to the true score, so its minimum over the class is the best score
the class can reach [arxiv:2611.01562 §2]. The paper then bounds the total
variation of the sampler, with terms for initialisation, time discretisation,
approximation and finite-time optimisation [arxiv:2611.01562 §3.3].

The approximation term decays with the complexity index `M` at a rate set by
the smoothness and the intrinsic dimensions of the response and the condition
[arxiv:2611.01562 §3.1]. The optimisation term is `O(1/N)` under uniform
constants, which gives a sample requirement of order `eps^-2` for a target
error `eps` [arxiv:2611.01562 §3.5]. These rates depend on assumptions the
authors state but do not prove for neural networks [arxiv:2611.01562 §3.2]
[§4].

## When to use which

The concept is used in theory here, not in a tested recipe. Neither card
reports a benchmark, and neither has released code. Pick by the question you
need answered.

| Situation | Use | Why |
|---|---|---|
| You want an exact error breakdown for a linear or Gaussian setting | Linear model with proxy target, [Linear diffusion with symmetry](/papers/2610.35720) | Closed forms for each error term [arxiv:2610.35720 §3] [§5] |
| You are checking whether a symmetry in the data will be learned | Same card | Its split shows which term persists and which decays [arxiv:2610.35720 §4] [§5] |
| You train from a stream of fresh samples and cannot store the data | Online objective, [Online conditional diffusion](/papers/2611.01562) | Each step uses new samples and a randomized time index [arxiv:2611.01562 §2] |
| You need a budget for samples, network size and steps | Same card | It gives orders for `M`, `N`, `K`, `T` under assumptions [arxiv:2611.01562 §3.5] |
| You need a bound on the stopped law rather than the data law | Same card | The bound compares to the law at a positive stopping time; the data-law result needs an extra density assumption [arxiv:2611.01562 §3.6] |

Practical limits. The linear analysis uses Gaussian data and one noise level
[arxiv:2610.35720 §7]. The online bound assumes a localized
Polyak-Lojasiewicz inequality and that every iterate stays in a local region
[arxiv:2611.01562 §3.2]. Its step size needs constants that the paper does not
estimate, so the bound does not give a training recipe with numbers
[arxiv:2611.01562 §3.2].

## Implementation sketch

The core of the denoising variant, using the target both papers share. The
step size, time distribution and sampler follow the cards; this is a sketch
from the formulas, not the authors' code, which does not exist.

```pseudocode
# Denoising score matching with a per-sample target  [arxiv:2611.01562 §2]
# Input:  clean batch x0 (B, d); condition y; grid times t (B,)
# Output: scalar loss for one minibatch

def score_matching_loss(score_net, x0, y, t):
    # Fresh Gaussian noise of the same shape as x0  [arxiv:2611.01562 §2]
    eps = sample_normal(x0.shape)                  # (B, d)
    # Noise coefficients of the variance-preserving process  [§2]
    alpha = exp(-t)                                # (B,)
    sigma2 = 1 - alpha ** 2                        # (B,)
    # Noised state at time t  [arxiv:2611.01562 §2]
    x_t = alpha * x0 + sqrt(sigma2) * eps          # (B, d)
    # Per-sample target: the score of p(x_t | x0)  [arxiv:2610.35720 §2.1]
    target = -(x_t - alpha * x0) / sigma2          # (B, d)
    # Regress the network output onto the target  [arxiv:2611.01562 §2]
    pred = score_net(x_t, y, t)                    # (B, d)
    return mean(sum((pred - target) ** 2, dim=-1))
```

The two variants ask different questions of the same noisy target. The linear
analysis asks how much symmetry error that noise leaves in a linear model
[arxiv:2610.35720 §5]. The online bound asks how far the sampler's distribution
can drift, given the approximation and optimisation errors of the class
[arxiv:2611.01562 §3.3]. The online paper does not split out a target-noise
term.

## Contradictions

None directly. The two cards measure different things: a symmetry error for a
linear model [arxiv:2610.35720 §2.2], and a total-variation bound for a
general conditional class [arxiv:2611.01562 §3.3]. Do not read one as
confirming or refuting the other. The linear paper's target-noise term stays
non-zero at fixed ratio [arxiv:2610.35720 §5]. The online paper states that its
population loss is minimised by the true score, up to a constant
[arxiv:2611.01562 §2]. Whether the noisy target biases a trained network in
practice is not settled by either card.

## Papers

- [Linear diffusion with symmetry](/papers/2610.35720): analytic split of the symmetry error of a linear score model trained by gradient flow.
- [Online conditional diffusion](/papers/2611.01562): online SGD on fresh randomized-time samples, with a total-variation bound split into four terms.
