Concept · 4 papers · updated 2026-11-22
Self-conditioning
Feeding a denoiser its own previous prediction as an extra input, so each step sees the last guess. Covers latent video diffusion (W.A.L.T, prediction channels), continuous text diffusion (LangFlow, expected-embedding input) and pure state carry without sampling tokens between steps (LAW). Costs: an extra forward pass on part of the training steps; results depend on the metric.
diffusionself-conditioningtext-generationvideo-generationsampling
Markdown5 min read
AI-compiled overview, updated 2026-11-22, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
Overview
An iterative denoiser usually throws away its prediction after each step and only keeps the noisy input. Self-conditioning keeps the prediction and passes it to the next call as an additional input. In training, the model first runs with an empty self-condition, then runs again with that first estimate, detached from gradients, as input 2312.06662 §4.3 2604.11748 alg. 1. At sampling, the estimate from the previous step is always available 2604.11748 alg. 2. The name is also used for an unrelated mechanism in anomaly detection, where a focus score from a first decoder pass feeds back into the encoder 2201.07284 §3.4. That is not the technique described here.
The invariant all variants share is that the carried quantity is computed from the model's own prediction, so the next step starts from a better-informed guess. The variants differ in what is carried, where it enters the network, and whether tokens are sampled between steps.
Variants
Prediction channels in latent video diffusion
W.A.L.T concatenates the first-pass estimate to the noisy latent along the channel axis. It applies this with probability 0.9 during training and runs the single-pass branch otherwise 2312.06662 §4.3. The paper's own ablation on UCF-101 reports FVD 109.9 without self-conditioning and 61.4 with it at p_sc = 0.9 2312.06662 tab. 3. Card: W.A.L.T. The paper does not state which quantity is fed back or which pass carries the loss, so implementers must choose and document it 2312.06662 §4.3.
Expected-embedding feedback in continuous text diffusion
LangFlow feeds back the probability-weighted average of the embedding table, computed from the predicted token distribution. It enters through a projection that starts at zero, and it is used on about one training step in four 2604.11748 §5 2604.11748 alg. 1. At sampling it is always on, starting from zero at the first step 2604.11748 §4.2 [arxiv:2604.11748 alg. 2]. On LM1B, the paper reports PPL 49.0 without it and 30.0 with it, and notes that the effect is much larger for the continuous model than for MDLM 2604.11748 tab. 1 2604.11748 §4.2. Card: LangFlow.
Pure state carry without token sampling
LAW removes the discrete sample between steps altogether. Each pass sends the expected embedding to the next pass, which is the only input to the denoiser, and tokens are drawn once after the last pass 2610.30068 §3.1. It trains with one or two unrolled passes, p = 0.9 for the two-pass case by default, and a stop-gradient between passes 2610.30068 §3.2. The paper reports that carrying the hidden representation instead drops GSM8K accuracy below 1.5% without temperature regularization, and that removing the stop-gradient drops accuracy from 42.2% to 20.2% at matched FLOPs 2610.30068 §4.3. Card: LAW. The earlier self-conditioning baseline it builds on, Loopholing (arXiv:2510.19304), has no card here, so its content is not used.
When to use which
| Situation | Use | Why |
|---|---|---|
| Latent video or image diffusion, transformer backbone, enough compute for an extra pass on most steps | Prediction channels 2312.06662 §4.3 | Reported FVD gain in the paper's own ablation 2312.06662 tab. 3; the paper uses p_sc = 0.9 |
| Continuous embedding diffusion for text, PPL is the target | Expected-embedding feedback 2604.11748 §4.2 | The paper reports the gap to discrete models closes with it 2604.11748 tab. 1 |
| Discrete masked or uniform diffusion, PPL is the target | Evaluate both ways and report which | Gen. PPL improves but PPL worsens for MDLM with it 2604.11748 tab. 1 |
| Few-step generation, where information lost between steps matters | Pure state carry 2610.30068 §4.3 | Reported gains on GSM8K and Sudoku at matched training FLOPs, small scale only 2610.30068 §6 |
| Training budget is tight | Compare variants at equal FLOPs, not equal steps | The LAW card estimates about 3.9 F per update against 3 F for a baseline, so it trains for fewer steps 2610.30068 app. B.1 2610.30068 app. B |
The choice depends on the metric. LangFlow's gains are measured on PPL and Gen. PPL, and the same self-conditioning raises MDLM's PPL [arxiv:2604.11748 tab. 1]. Any comparison should use the same protocol for every row.
Implementation sketch
The common core is one extra input and, in training, one extra forward pass
without gradient. Shapes are (batch, positions, dim). Variants differ in what
estimate is and where it is concatenated or added.
# Training step with self-conditioning [arxiv:2604.11748 alg. 1]
# p_sc: probability of running the first pass
def train_step(model, x_t, t, target, p_sc):
# Empty self-condition by default [arxiv:2312.06662 §4.3]
prev = zeros_like(x_t)
if random() < p_sc:
# First pass without gradient, its output is the estimate
# [arxiv:2604.11748 alg. 1]
prev = stop_gradient(model(x_t, t, zeros_like(x_t)))
# Second pass receives the estimate as an extra input
pred = model(x_t, t, prev)
return loss(pred, target)# Sampling loop with the previous estimate carried forward [alg. 2]
def sample(model, x, timesteps, to_estimate, step):
# Starts with an empty self-condition [arxiv:2604.11748 alg. 2]
prev = zeros_like(x)
for t_now, t_next in pairs(timesteps):
pred = model(x, t_now, prev)
# Carried quantity: a prediction, or its expected embedding
# [arxiv:2610.30068 eq. 8] [arxiv:2604.11748 alg. 2]
prev = to_estimate(pred)
x = step(x, pred, t_now, t_next)
return xReported settings, for reference:
| Variant | Feedback | Probability | Gradient through first pass | Source |
|---|---|---|---|---|
| Prediction channels | Estimate, concatenated along channels | 0.9 | Detached | 2312.06662 §5.2 |
| LangFlow | Expected embedding through a zero-initialised projection | 0.25 in training, always on at sampling | Detached | 2604.11748 §5 |
| LAW | Expected embedding, sole denoiser input | Two passes with p = 0.9 by default | Stop-gradient between passes | 2610.30068 §3.2 |
Contradictions
- LangFlow's LM1B Gen. PPL. The same model is reported at 81.5 in 2604.11748 tab. 1 and at 92.2 in 2604.11748 tab. 2. The appendix NFE table agrees with 92.2 2604.11748 tab. 2. Both values are cited here, and the card does not settle which one the authors intend.
- W.A.L.T's default ablation row. The default configuration scores 60.7 2312.06662 tab. 3, while the p_sc = 0.9 row scores 61.4 with the same setting 2312.06662 tab. 3. The paper does not say which run is the baseline.
- LAW's stop-gradient. The paper's equation puts the stop-gradient on the whole next state 2610.30068 eq. 10. The released code detaches only the probabilities and keeps gradient through the embedding matrix via the carry dit_loop.py. This comes from reading the code, not from running it.
Papers
- Photorealistic Video Generation with Diffusion Models: prediction-channel self-conditioning in a latent video transformer, with an ablation at p_sc = 0.9.
- LangFlow: expected-embedding self-conditioning in continuous text diffusion, with an LM1B ablation against MDLM.
- LAW: self-conditioning as the only state carry, trained with a stop-gradient.
- TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data: uses the term for a focus-score feedback in anomaly detection; included only to mark the name collision.
Source and license
Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.