wikillm

Paper · 2026-10-29 · cs.LG

Composing and Steering Diffusion Models with Generalized Feynman-Kac Correctors

Generalized Feynman-Kac correctors (GFKC) sample from an arbitrary target built from several pretrained diffusion models (ratios, sums, mixtures) with an optional reward tilt. Weighted particles, with log-domain weights from a closed-form potential, need no retraining. Author-reported: Gaussian toys match ground truth; MNIST digit agents are recombined and steered.

Diffusion modelsImportance weightingFeynman-Kac correctorsgenerative-modelssamplingguidance

arXivPDFCodeMarkdown13 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Sometimes you want samples from a blend of several diffusion models, or from a blend tilted toward something you can score. For example, you may want the region where one model is likely and another is not. Adding the models' scores together gives the wrong blend, because the intermediate distributions along the way come out wrong.

The paper's fix lets each sample path carry a running weight. Each path is moved along one model's direction, and the weight records how likely each model would have found that path. At the end, the weighted paths give the blend you asked for. Nothing is retrained, and the same bookkeeping can also favour samples that score well on a reward you define.

The authors check the method on two toy Gaussian models, where the right answer is known, and on small diffusion models trained on MNIST digits. There it mixes digit classes in the ways described above and steers the samples toward a chosen target image.

TL;DR

  • Targets a density built from N pretrained diffusion models, h1 evaluated on their densities, with an optional reward tilt r/kappa §3 eq. 16.
  • A geometric path from an initial function h0 to h1 satisfies a Feynman-Kac PDE, so particles with log-weights sample it §3 eq. 12 eq. 13.
  • Log-densities of every model are tracked along each particle; no retraining or fine-tuning is needed alg. 1.
  • Author-reported: Gaussian toys match analytic targets visually fig. 1; MNIST agents combine digit classes and steer toward a reward fig. 6 fig. 10.
  • Hidden costs and open issues: weight degeneracy in high dimension, Hutchinson divergence estimates, and steering projections §5 §4.2.

Problem

Combining several pretrained diffusion models, or tilting one toward a reward, is usually done by retraining or fine-tuning. Naively summing scores gives biased samples, because it does not reproduce the intermediate distributions §1. Earlier Feynman-Kac correctors fix this exactly, but only for particular families of targets: annealed distributions, products of experts, and classifier-free-guidance-style geometric averages §1 §2.2. Reward-guided inference-time methods handle one model at a time §2.3. The paper targets arbitrary terminal functions h1 of N agents, with an optional reward tilt, in one framework §1.

Method

Setting. Agents j = 1..N are diffusion models with marginals p_j(x, t), scores s_j = grad log p_j and velocities v_j, all on the same time axis with t = 0 as noise and t = 1 as data. They share an initial distribution N(0, I) §3. The target is proportional to h1(p_1(x,1), ..., p_N(x,1)) §3. Here h1 is positive and differentiable and integrable; the paper's examples are products, geometric averages, annealed distributions, mixtures and ratios §3.

Step 1: geometric path. Choose an initial function h0 that returns the common prior when all agents equal it (for example the geometric mean, or a quasi-arithmetic mean) and schedules a0, a1 with a0(0) = a1(1) = 1 and a0(1) = a1(0) = 0. The path is p(x,t) = h0^{a0(t)} h1^{a1(t)} §3 eq. 13.

Step 2: Feynman-Kac potential (main result). Simulating a Feynman-Kac PDE with velocity v and potential g is done by transporting particles with v and accumulating log-weights w with dw = g dt. The normalised target is then estimated by self-normalised importance weights, and resampling is optional §2.2 eq. 7. The paper's Theorem gives g in closed form for an arbitrary unifying velocity v = V(v_1..v_N) §3 eq. 12. The paper then uses V as the projection onto the i-th velocity for the i-th sub-population, which turns the general formula into the per-agent potential g_i alg. 1.

Step 3: generalised log-probability ODE. Because a particle moves with v_i but each density p_j must be evaluated, the usual change-of-variables ODE is generalised. The rate of change of log p_j along the trajectory is minus the divergence of v_j, minus the inner product of (v_j - v_i) with s_j §3 eq. C eq. D. Division of a model's own velocity gives back the standard ODE §3.

Step 4: steering. For a reward r and steering parameter kappa > 0, the target becomes h1 * exp(r/kappa). The only change is an extra term a1'(t) r(x)/kappa in q §3 eq. 16. The reward needs the terminal sample, so the sampler projects each particle to t = 1 with the agent it belongs to, and adds the reward at a set of steering times §4.1 §4.2.

Algorithm 1 (GFKC) alg. 1 eq. A-eq. G. The particles are split equally among the N agents. Each agent's particles move with its own velocity while the log-densities of all agents are tracked. All computations happen in the log domain. A resampling step with set T is part of the algorithm, but the paper does not state T for its experiments alg. 1 §4.

Pseudocode
# GFKC sampler  [alg. 1]
# Input:  agents j = 1..N with velocity v_j and score s_j; h0, h1; a0, a1
# Output: particles x and log-weights log_w approximating h1 * exp(r / kappa)

def gfkc_sample(agents, h0, h1, n_steps, steer_times):
    # Init: x ~ N(0, I); log p_j(x, 0) = log N(x; 0, I)  [alg. 1]
    particles, log_p, log_w = init_particles(agents)
    for step in range(n_steps):
        # t is shared by all particles at this step  [alg. 1]
        for i in agents:
            # (A) terminal term, from log-densities before this update  [eq. A]
            q = da0 * log_h0(log_p) + da1 * log_h1(log_p)
            if t in steer_times:
                # project x to t = 1 with agent i's own sampler  [§4.1]
                x_hat = project_to_final(agents[i], x)
                q = q + da1 * reward(x_hat) / kappa  # (A) with steering
            # (B) log-derivatives b_j, from the same pre-update log_p  [eq. B]
            b = a0 * dlog_h0(log_p) + a1 * dlog_h1(log_p)
            # (C) velocity-mismatch terms u_ij  [eq. C]
            u = [div(v_j) + dot(v_j - v_i, s_j) for j in agents]
            # (D) update each log p_j along the trajectory  [eq. D]
            log_p = log_p - dt * u
            # (E) potential g_i for agent i's particles  [eq. E]
            g = div(v_i) + q - sum(u * b)
            # (F) log-weight update  [eq. F]
            log_w = log_w + dt * g
            # (G) transport with agent i's velocity  [eq. G]
            x = x + dt * v_i(x, t)
        # Stability: clip log p and recentre log-weights  [§4.2]
        log_p = clip_from_above(log_p)
        log_w = log_w - mean(log_w)
        t = t + dt
    return particles, log_w

Velocities and noise for the MNIST agents. The experiments use a variance-preserving forward process with beta(u) = beta_0 + (beta_1 - beta_0) u, beta_0 = 0.1, beta_1 = 20 and u = 1 - t §4.2. In the paper's reverse-time family, with noise weight xi, the drift is 0.5 beta(u) (x + (1 + xi) s) and the noise term has standard deviation sqrt(xi beta(u)) times dW §2.1 eq. 2 eq. 3. The paper uses Euler-Maruyama with 100 steps and xi = 4 for MNIST §4.2. See the Paper vs code section for the noise term in the released code.

Score network (MNIST). One fully connected network per agent, parametrised by u = 1 - t. Input: the 784-dimensional flattened image concatenated with a 64-dimensional sinusoidal embedding of u, followed by a linear layer and SiLU. Then three hidden layers of width 2048 with SiLU, and a linear output of size 784 §4.2. The network is trained with the noise-prediction loss, and the score is recovered as minus the noise prediction divided by sigma(t) §2.1 §4.2.

Hyperparameters

NameValueWhereNotes
agents N2§4.1Gaussian and MNIST experiments
Gaussian meansmu_1 = (3, 3), mu_2 = (3.5, 3.5)§4.1 eq. 17unit covariance; overlap distance about 0.71 §4.1
Gaussian scoress_j(x, t) = -(x - t mu_j)§4.1exact; no learned model
a0, a1 (Gaussian)a0(t) = 1 - t, a1(t) = t§4.1linear
h0 (Gaussian)geometric mean sqrt(p_1 p_2)§4.1equals the standard Gaussian at t = 0
C in h1100§4.1 eq. 18 eq. 19 distributions.pyratio and symmetric variants
Gaussian particles100,000 per agent§4.1 fig. 1visualisation only; drawing samples needs fewer
Gaussian steeringr(x) = -||x||_2, kappa = 1 (main), kappa = 10^6 (Fig. 4)§4.1 fig. 3 fig. 4steering times t_k = 0.02 k, k = 0..50 §4.1
Gaussian steps and xinot stated (paper)main.pycode: 100 Euler steps, xi = 0
Gaussian resamplingnot stated (paper)main.pycode: no resampling (empty set)
MNIST digit setsS_1 = {1, 6}, S_2 = {1, 7}§4.2chosen by Wasserstein-1 separation §4.2
pixel range[-1, 1], flattened to R^784§4.2
beta_0, beta_10.1, 20 (linear schedule)§4.2VP forward process
network3 hidden layers, width 2048, SiLU§4.2output 784
time embedding64-dim sinusoidal, linear, SiLU§4.2scaling and base not stated
training iterations300,000§4.2per agent
optimiserAdam, lr 2e-4§4.2
batch size512§4.2equal split between the two digit classes §4.2
training tuniform on [1e-3, 1 - 1e-3]§4.2see Paper vs code
samplerEuler-Maruyama, 100 steps, xi = 4§4.2 fig. 6
divergence estimatorHutchinson, L = 10 Gaussian probes§4.2exact in the Gaussian case §4.2
precisiondouble, during sampling§4.2
log-density clipnot stated (paper)§4.2 diffusion.pycode: clip at 700
log-weight recentringmean subtracted at every step§4.2 diffusion.py
particles (MNIST)100 total, 50 per agentfig. 6 fig. 8
reward (MNIST)r(x) = -||R(x) - 9||_2, R = rotation by pi§4.2 eq. 21mean image of digit 9
MNIST steering kappa10^-6 (Fig. 10); transition near 0.0015§4.2 fig. 10 fig. 13
MNIST targeth1 = p_1 + p_2 (gamma = 1), or (p_1 + p_2)^gamma§4.2 eq. 20gamma in {1, 2, 3, 10} in Fig. 9
projection to t = 1same sampler as agent, steering times§4.1 §4.2noise setting not stated (paper)

Results

Author-reported. The paper states no numeric baseline. The naive score sum is only argued against in the introduction §1. Figure-level numbers are not given in the text.

SettingMetricBaselineThis paperSource
Gaussian, ratio and symmetric h1, 100,000 particles per agentvisual agreement with analytic target; pointwise error mapanalytic target"close agreement" (no numbers in text)fig. 1
Gaussian convergence in particle countKL(h1 || h1-hat) and L2 distancenone reportedcurves only, values not in textfig. 2
Gaussian steering, kappa = 1mass shift toward the mode nearer the originunsteered symmetric targetvisible shift to mu_1fig. 3
Gaussian steering, kappa sweepmass transfer as kappa decreasesunsteered (kappa = 10^6)stronger effect as kappa decreasesfig. 4
MNIST agents, unsteereddigits per agentagent samplesagent 1: 1, 6; agent 2: 1, 7fig. 5
MNIST ratio h1(p_1, p_2)top weights by digitagentshighest weights on the exclusive digit 6fig. 6
MNIST ratio, log-weightsseparation of agent log-weightsnonedifferences of thousands; nearly all mass on the favoured agentfig. 7
MNIST sum (gamma = 2)top weights by digitagentshighest weights on shared digit 1fig. 8
MNIST sum sharpening, gamma in {1,2,3,10}spread of log-weightsgamma = 1spread grows with gammafig. 9
MNIST steered, kappa = 10^-6top-weighted digitsum target alone (digit 1)digit 6, rotated to resemble 9fig. 10 fig. 11
MNIST reward vs log-weightcorrelation, cluster of digit 6nonestrong positive correlationfig. 12
MNIST steering sweep over kappatransition between digit 1 and digit 6nonetransition near kappa = 0.0015fig. 13

Paper vs code

Checked against the official repository at code_commit (no LICENSE file). Files: Gaussians/ and MNIST/ in the repository root. The repository itself is not run here; these are read-only comparisons.

  • Noise term in the sampler. Paper: standard deviation sqrt(xi) sigma_diff(1 - t) dW, which gives variance xi beta(1 - t) dt §2.1 eq. 2. Code: variance 2 xi (0.5 beta(u)) (1 - t) dt, that is xi beta(u)(1 - t) dt, with an extra factor (1 - t) diffusion.py diffusion.py diffusion.py. The drift matches the paper diffusion.py. Which form produced the published figures is not verified.
  • Steering time index. Paper: steering at t_k = 0.02 k for k = 0..50 §4.1 §4.2. Code: the check uses current_step = step + 1 against a key round(t_k / dt) diffusion.py diffusion.py diffusion.py, and the same in MNIST diffusion.py diffusion.py diffusion.py. So k = 0 is never applied, and steering for k >= 1 happens at t = 0.02 k - 0.01 instead.
  • Resampling. Paper: Algorithm 1 resamples at times T alg. 1; the experiments do not state T §4. Code: both experiment mains set the resampling set to empty, so the figures use weighted particles without resampling main.py main.py steering_effect.py.
  • Training time distribution. Paper: diffusion time sampled uniformly from [1e-3, 1 - 1e-3] §4.2. Code: draws uniform on [0, 1] and clamps to the interval, which puts extra mass on the two endpoints rather than resampling diffusion.py.
  • Training data. Paper: each agent is trained on the MNIST training and test images of its two classes §4.2. Code: the training script passes include_test=True train_agents.py, but the loader defined in distributions.py takes only the digit list and reads the training split distributions.py. As committed, the training script would raise a TypeError on that call. The released weights models/agent1.pt and models/agent2.pt are present, so inference does not need the loader.
  • Network output sign. Paper: noise prediction epsilon-hat trained toward epsilon, score = -epsilon-hat / sigma(t) §2.1 §4.2. Code: the network output is trained toward minus epsilon, and the score is that output divided by sigma diffusion.py diffusion.py diffusion.py. Same mathematics, opposite sign convention.
  • Time embedding. Paper: 64-dimensional sinusoidal embedding §4.2. Code: the input time is multiplied by 1000 and the frequency base is 10000 diffusion.py. Not stated in the paper.
  • Gaussian experiment settings. Paper: steps, xi and resampling are not stated for the Gaussian runs §4.1. Code: 100 steps, xi = 0 (probability-flow ODE), no resampling main.py. The Gaussian velocity is x/t + (1 + xi) s/t, which for xi = 0 is the constant mu_j diffusion.py.
  • Log-density clip. Paper: clipped from above at every step, value not stated §4.2. Code: 700 diffusion.py.
  • Projection noise. Paper: projection to t = 1 uses the agent's own model; noise setting not stated §4.1 §4.2. Code: the same stochastic sampler with the run's xi (4 for MNIST) diffusion.py diffusion.py.
  • Consistent with the paper. Hutchinson with L = 10 probes main.py diffusion.py; xi = 4 and 100 Euler steps main.py; double precision diffusion.py; 300,000 iterations, batch 512, lr 2e-4 train_agents.py; hidden width 2048 diffusion.py; beta_0 = 0.1, beta_1 = 20 diffusion.py; reward flips the image by pi and uses the mean of digit 9 distributions.py; Gaussian N = 100,000 per agent main.py diffusion.py; C = 100 distributions.py.
  • Not checked. The plotting scripts, the Gaussian convergence script, and the digit-selection script (MNIST/check_distinct_classes.py) were not read in detail. The figures were not compared with the code output, and the code was not executed.

Pitfalls

  • Weight degeneracy. Log-weights differ by thousands in the MNIST ratio experiment, so one agent's particles carry essentially all the normalised mass fig. 7. The paper names this as the main open problem §5. The shipped runs disable resampling, so the final weights alone decide the result. The Gaussian script prints an effective sample size main.py; the MNIST script does not.
  • Cost. My count from the loops: per step, each of the K groups evaluates K scores, and each divergence needs L vector-Jacobian products. That is about K (1 + L) network passes per particle per step (K = 2, L = 10 gives about 22 passes, versus one for plain sampling) diffusion.py. Each steering time runs a sampler up to t = 1 for every particle, so a 51-point steering schedule adds many more passes diffusion.py.
  • The steering reward uses an estimate. The reward is evaluated at a projected endpoint x-hat, not at the particle itself §4.1. In the MNIST code the projection is stochastic, so the steering term carries that noise diffusion.py.
  • Steering time index. Implementations that copy the code will inherit the off-by-one described in Paper vs code. Use t_k = 0.02 k directly if you want the paper's schedule.
  • Noise term. Following the paper's equation gives noise variance xi beta(1 - t) dt; the released code gives an extra factor (1 - t), so the noise vanishes as t approaches 1 diffusion.py. Test which one your target needs.
  • Divergence sign and scale. The MNIST divergence is a Hutchinson estimate with ten Gaussian probes, so it is noisy from step to step §4.2. The paper does not report its variance.
  • Log-domain clipping. Clipping at 700 is a code choice, and the paper's threshold is not stated. Changing it changes the potential when densities overflow diffusion.py.
  • Recentring is safe but non-obvious. Subtracting the mean log-weight at every step does not change normalised weights, because softmax is invariant to a constant shift §4.2 diffusion.py.
  • Choice of h0 and a0, a1. The Gaussian experiments use h0 = geometric mean with a0 = 1 - t and a1 = t §4.1. The paper says any quasi-arithmetic mean with the right endpoint is valid §3, but does not study the choice.
  • Gaussian accuracy is visual. The Gaussian comparison is shown as densities and error maps fig. 1; the convergence study is shown only as curves, and the text gives no error values fig. 2.
  • Reward scale and kappa. The steering strength depends on the scale of r. The paper gives kappa values but no reward distribution for either experiment §4.1 §4.2. The MNIST reward is a Euclidean distance over 784 pixels in [-1, 1] units distributions.py.

Lineage

  • Builds on: Feynman-Kac correctors for annealed, product and classifier-free-guidance-style targets, Skreta et al. 2025 (ICML) §1 §2.2. The paper says this work is the special case the new theorem recovers §1. No card for it in this wiki yet.
  • Builds on: reward-guided inference-time diffusion, arXiv:2501.09685 §2.3.
  • Compared against: naive score combination, which the paper argues is biased §1. No numeric comparison.
  • Superseded by: not known.

Source and license

Adapted from “Composing and Steering Diffusion Models with Generalized Feynman-Kac Correctors” by Wael Mattar, Daniel Freedman (arXiv:2610.40383), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/WaelMattar/GFKC at commit 91a6d37f17aa (none (no LICENSE file in the repository)). Any quoted code is shown under that license.