---
kind: concept
slug: diffusion-models
title: Diffusion models
summary: >-
  Diffusion models learn to predict the noise added to a sample and run that prediction in reverse, starting from noise. Variants differ in the space they work in (pixels, latents, tokens, graphs, trajectories, speech, motion, robot actions), how conditions enter, and the number of sampling steps. Ninety-nine cards, with author-reported results.
tags: [diffusion, generative-models, text-to-image, video, sampling]
papers: ["2101.02388", "2101.12072", "2104.03670", "2105.06337", "2107.00630", "2108.01368", "2112.00390", "2112.03126", "2112.05744", "2203.02923", "2203.13333", "2203.13777", "2204.00227", "2204.06125", "2205.14217", "2206.09012", "2208.11970", "2209.14734", "2209.14916", "2211.00611", "2211.01324", "2211.10440", "2211.14680", "2212.00774", "2212.03860", "2212.04048", "2212.04493", "2212.05034", "2212.09478", "2212.09748", "2301.07093", "2302.00111", "2302.02913", "2302.08113", "2302.08453", "2302.09778", "2303.01418", "2303.08137", "2303.09472", "2303.14126", "2303.17189", "2304.03373", "2304.05977", "2304.06025", "2304.08466", "2305.07015", "2305.08891", "2305.10431", "2305.10474", "2306.03881", "2306.16928", "2307.01952", "2308.06038", "2308.12964", "2310.07204", "2310.07896", "2310.11868", "2311.07885", "2311.16518", "2311.17042", "2311.18828", "2312.00206", "2312.00858", "2312.02133", "2312.06662", "2312.11243", "2402.17485", "2403.12945", "2404.10667", "2404.13026", "2404.17569", "2405.12213", "2406.00960", "2406.07524", "2406.19464", "2407.01392", "2408.11048", "2409.00588", "2409.05493", "2409.14411", "2411.01349", "2411.12982", "2411.14499", "2411.14786", "2412.02617", "2412.07772", "2412.17226", "2412.20226", "2501.12982", "2502.05855", "2503.23377", "2507.15062", "2509.04018", "2510.12308", "2511.23429", "2512.06963", "2512.07237", "2512.18477", "2603.10448"]
related: [action-chunking, curriculum-learning, classifier-free-guidance, classifier-guidance, attention-mechanism, self-attention, layout-to-image-generation, image-super-resolution, continuous-text-diffusion, controllable-text-generation, time-series-forecasting, image-manipulation-detection, latent-diffusion, discrete-diffusion, knowledge-distillation, kv-cache-management, generative-prior-inverse-problems, multi-view-reconstruction, image-to-3d-generation, semantic-segmentation, imitation-learning, model-predictive-control, masked-language-modeling, equivariance-constraint, diffusion-inpainting, ddim-inversion, data-augmentation, adaptive-instance-normalization, vision-language-action-model, contrastive-learning, machine-unlearning, red-teaming, adversarial-text-generation, text-to-image-personalization, gaussian-splatting, differentiable-simulation, test-time-training, soft-prompt-tuning, entropy-minimization, preference-optimization, rlhf, mixture-of-experts, text-guided-image-editing, flow-matching, text-to-image-generation, vision-transformer, parameter-efficient-fine-tuning, novel-view-synthesis, monocular-depth-estimation, neural-implicit-surface, molecular-representation-learning, masked-image-modeling, multimodal-fusion, positional-encoding, camera-pose-encoding, explicit-feature-grid, feed-forward-reconstruction, differentiable-rendering, few-shot-learning, noise-schedule, self-supervised-learning, reward-modeling, generative-adversarial-network, neural-radiance-field, dataset-curation, dense-correspondence, image-deblurring, generative-model-evaluation, variational-autoencoder, topological-map-navigation, adaptive-loss-weighting]
status: draft
updated_at: 2026-11-01
---

## Overview

A diffusion model is trained to remove noise. A clean sample is mixed with
Gaussian noise at a randomly drawn level, and a network learns to predict the
added noise from the noisy input, the level and any conditions
[arxiv:2301.07093 §4.2] [arxiv:2212.04048 §3.2] [arxiv:2203.13777 §3.2].
Sampling starts from noise and applies the learned prediction step by step, so
the number of steps sets the cost of every sample [arxiv:2203.13777 §4.3]
[arxiv:2212.04048 §4.2].

The shared invariant: the network gets the noisy state, a level and the
conditions, and outputs a correction that moves the state toward the data.
Conditions enter in different ways. They can be extra tokens
[arxiv:2301.07093 §4.1], a concatenated embedding [arxiv:2212.04048 §3.3],
cross-attention [arxiv:2211.01324 §4.2], or a per-sample state from an encoder
[arxiv:2203.13777 §3.5]. Some variants change the target: Grad-TTS predicts a
score and Diffusion-LM predicts the clean sample, as covered under
Contradictions [arxiv:2105.06337 §3.2] [arxiv:2205.14217 §4.2].

The prediction target also differs across the cards. Some predict the noise,
some predict a velocity [arxiv:2312.06662 §3], some predict the clean sample in
planning [arxiv:2407.01392 app. D], and discrete models predict a distribution
over classes [arxiv:2209.14734 §3.2]. Discrete variants replace Gaussian noise
with masking or category switching [arxiv:2406.07524 §3.1]
[arxiv:2209.14734 §3.1].

For a broader map of the field, the 2023 state-of-the-art report on diffusion
models for visual computing explains the SDE and probability-flow view and the
latent design of Stable Diffusion. It runs no experiments of its own, so it is
a reading guide rather than a source of results [arxiv:2310.07204 §3.1]
[arxiv:2310.07204 §3.2]. Card: [State of the art on diffusion models for visual
computing](/papers/2310.07204).

A survey of world models treats video generators as a main current route to
learned simulators, and names weak causal reasoning and physics failures as
their main limits. It runs no experiments [arxiv:2411.14499 §4.1].

The 99 cards cover image, video and 3D generation, grasp synthesis, LiDAR
scenes, humanoid whole-body control, dexterous piano playing, audio-visual
manipulation, physics-based character control, visual navigation, text-to-3D meshes, CLIP-guided textured meshes, 3D materials, sparse-view
3D reconstruction, layouts, text, graphs,
motion, speech, talking faces, trajectories, time series, MRI reconstruction,
robot planning, control, robot demonstration data, generalist robot policies, diffusion action heads with failure checks, joint video and action generation, search over imagined futures, street-view rendering, medical masks, synthetic training data,
reward-model fine-tuning, adversarial one-to-four-step distillation,
test-time adaptation, safety testing, interactive game worlds, a
travelling-salesman search, one theory result, label-efficient segmentation, end-to-end mask diffusion, loss weighting over noise levels, tuning-free multi-subject personalization, metrics for generated designs, semantic super-resolution, goal-masked navigation, molecular
conformations, super-resolution, image restoration, dense correspondence, faster sampling from cached features, camera-controlled video, tactile
manipulation, a tutorial derivation, whether generators copy their training images, mask-guided object inpainting, layout control with and without training, a noise-schedule fix for the brightness range, physics-informed flow field reconstruction, joint audio and video from coupled denoisers, and correlated frame noise for video finetuning. Their results are on
different tasks and metrics, so the numbers below are not comparable across
variants.

## Variants

### One derivation for the noise, clean and score targets

A tutorial treats a diffusion model as a Markovian hierarchical VAE whose
noising steps are fixed Gaussians [arxiv:2208.11970 §3]. Under that view the
ELBO becomes a sum of denoising terms, and each term can be written as
predicting the clean sample, the source noise or the score, each with its own
time-dependent weight [arxiv:2208.11970 §3]. The tutorial also covers the
closed-form ancestral sampler, with the reverse variance fixed to the
posterior's and only the mean learned [arxiv:2208.11970 §3], and it treats
classifier and classifier-free guidance as score combinations
[arxiv:2208.11970 §5]. It runs no experiments, and it names the costs
itself: many sequential sampling steps, latents as large as the data, and a
fixed encoder [arxiv:2208.11970 §6]. Use it to check the algebra behind a
variant, not as a source of numbers. Card: [Understanding Diffusion Models: A
Unified Perspective](/papers/2208.11970).

### Copy detection for generated images

Somepalli et al. ask whether diffusion models reproduce their training images.
They score every generated image against every training image with a
retrieval detector, using DINO ViT-B/16 token features and a split-product
similarity, which keeps the best token pair per image pair [arxiv:2212.03860
§4]. A top-1 score above 0.5 is treated as a copy for Stable Diffusion
[arxiv:2212.03860 §7]. Small DDPMs trained on Celeb-A (300 and 3000 images)
and on Oxford Flowers (100 and 1083 images) produce blatant copies, while the
full-data Celeb-A checkpoint and an ImageNet latent model show none the tool
found [arxiv:2212.03860 §5] [§6]. For Stable Diffusion v1.4, about 1.88% of
9,000 generations pass that cut-off, but only the 12M-image LAION Aesthetics
v2 6+ subset was searched. The authors say the rate is likely an underestimate
[arxiv:2212.03860 §7] [§9]. The matching is object-level, and the paper admits
grey areas in the definition [arxiv:2212.03860 §1] [§3]. Card: [Diffusion Art
or Digital Forgery?](/papers/2212.03860).

### Latent diffusion with a frozen backbone and added conditions

The denoiser works on latents, not on pixels [arxiv:2301.07093 §3] [app. A].
GLIGEN keeps all of the pretrained weights frozen and trains only new layers,
which take extra conditions as tokens and gate their output through a learned
scalar that starts at zero. At initialisation the model therefore reproduces
the pretrained model's output [arxiv:2301.07093 §4.2]. The paper uses this to
add box-grounded text-to-image generation [arxiv:2301.07093 §4.1]. Card:
[GLIGEN](/papers/2301.07093).

Inside that setup, the authors prefer gated self-attention to gated
cross-attention, which performs worse in their ablation [arxiv:2301.07093
app. B]. The attention block itself is covered on [Attention mechanism](/concepts/attention-mechanism).

### Scheduled conditioning at sampling time

The added conditions can be switched off after the first fraction `tau` of the
denoising steps, so that early steps fix the layout and later steps refine
detail with the original model [arxiv:2301.07093 §4.2] [eq. 10]. The paper
reports this only qualitatively, using `tau = 0.2` for one figure [arxiv:2301.07093
§5.4] [fig. 7]. Card: [GLIGEN](/papers/2301.07093).

### Latent diffusion over motion clips

MLD first compresses a motion clip with a transformer VAE. In the best setting
the latent is a single 256-d token [arxiv:2212.04048 §5] [tab. 5]. A transformer
denoiser then runs diffusion in that latent. Text goes through a frozen CLIP
encoder and action labels through a learned embedding, with guidance dropout
0.1 and scale 7.5 [arxiv:2212.04048 §3.3] [tab. 11]. Author-reported FID on
HumanML3D is 0.473, against 0.544 for MDM and 0.630 for MotionDiffuse
[arxiv:2212.04048 tab. 1]. The paper's R-Precision and MM Dist claims do not
match its tables, where MotionDiffuse ranks first on those metrics, so only the
FID lead is clear. Card: [MLD](/papers/2212.04048).

### Expert denoisers split by noise level

eDiff-I is a text-to-image cascade: a 64 px base model, then super-resolution
to 256 px and 1024 px [arxiv:2211.01324 §4.2] [fig. 5]. The base is split by
noise level into experts. One shared model is trained on all levels, then
children are initialised from it and fine-tuned on their own band of levels.
The final base uses three experts: lowest noise, highest noise, and a middle
one [arxiv:2211.01324 §4.1] [tab. 5]. The authors claim the ensemble costs the
same per sampling step as one network, with the numbers shown only in a figure
[arxiv:2211.01324 §5.1] [fig. 13]. Conditioning uses T5-XXL, CLIP text and CLIP
image embeddings, each dropped independently [arxiv:2211.01324 §4.2]. The
training-free paint-with-words control adds a weighted mask bias to the
cross-attention logits [arxiv:2211.01324 §4.3]. Author-reported zero-shot
FID-30K is 6.95 for the best configuration, against 7.23 for Parti and 7.27 for
Imagen [arxiv:2211.01324 tab. 1]. No code or weights are released, and the
training data is about one billion pairs, partly proprietary [arxiv:2211.01324
§5]. Use this variant as a design reference. Card: [eDiff-I](/papers/2211.01324).

### Score-based decoder for speech features

Grad-TTS maps text to mel-spectrograms. An encoder predicts per-token means and
durations, and durations come from a monotonic alignment search [arxiv:2105.06337
§3.2]. The diffusion decoder starts from noise centred on the frame means and
takes N Euler steps of a reverse ODE. A HiFi-GAN vocoder then makes audio
[arxiv:2105.06337 §3.1] [§4]. N is the speed-quality dial. Author-reported
LJSpeech MOS is 4.38 at N = 10 (Tacotron2 4.32, ground truth 4.53), 3.96 at
N = 4, and almost no change from 10 to 1000 steps [arxiv:2105.06337 tab. 2]
[§4.1]. The real-time factor is 0.033 at N = 10, against 0.075 for Tacotron2,
both measured by the authors on a GPU [arxiv:2105.06337 tab. 2]. Card:
[Grad-TTS](/papers/2105.06337).

### Reverse diffusion over future trajectories

MID denoises future pedestrian positions. A Trajectron++ state encoder
summarises the observed history of 8 frames and the neighbours. A Transformer
denoiser predicts the noise over K = 100 steps [arxiv:2203.13777 §3.5] [§5]
[alg. 1]. Each history gets 20 draws (Best-of-20). Author-reported SDD ADE and
FDE are 7.61 and 14.30, the lowest of the trajectory-only rows in its table. On
ETH/UCY the average is comparable rather than best, at 0.21 and 0.38 m
[arxiv:2203.13777 tab. 1] [tab. 2]. The cost is time: 17.4 s against 0.44 s for
Trajectron++ on 512 trajectories [arxiv:2203.13777 §5]. Fewer steps give more
diverse samples and lower accuracy [arxiv:2203.13777 §4.3] [fig. 3]. Card:
[MID](/papers/2203.13777).

### Continuous diffusion over word embeddings

Diffusion-LM denoises a sequence of word-embedding vectors and rounds the result
to words. Its continuous latents can be steered by gradients from a classifier.
Author-reported success on classifier-guided control is 81.2 against 69.9 for
FUDGE on semantic content, and 90.0 against 27.0 on parts of speech
[arxiv:2205.14217 tab. 2]. Its variational bound on E2E is worse than an
autoregressive Transformer's, at 2.28 against 1.77, and decoding takes many
steps [arxiv:2205.14217 §7] [§6.1]. The text-specific details are on
[Continuous text diffusion](/concepts/continuous-text-diffusion). Card:
[Diffusion-LM](/papers/2205.14217).

### Masked diffusion over discrete tokens

MDLM corrupts text by replacing each token with a mask token at a rate set by a
schedule, and a Transformer learns to fill the masks [arxiv:2406.07524 §3.1].
Once a token is unmasked it is carried over unchanged, and the model cannot
output the mask token [arxiv:2406.07524 §3.2.3]. The continuous-time training
loss is then a weighted average of masked-token cross-entropies
[arxiv:2406.07524 §3.5]. Sampling starts from all masks and unmasks tokens over
N steps, and the paper's semi-autoregressive variant writes a 2,048-token text
in blocks [arxiv:2406.07524 §4.2]. Author-reported LM1B perplexity bound is
23.00 at 327B training tokens, against 20.86 for a retrained autoregressive
model [arxiv:2406.07524 tab. 1]. On OpenWebText the bound is 23.21, against
17.54 for the autoregressive model [arxiv:2406.07524 tab. 2]. Diffusion numbers
are upper bounds, not exact likelihoods [arxiv:2406.07524 tab. 1]. Card:
[MDLM](/papers/2406.07524).

### Discrete denoising over graphs

DiGress applies diffusion to graphs with categorical node and edge labels, where
"no edge" is one of the edge classes [arxiv:2209.14734 §3]. Each node and edge
is corrupted by a Markov chain that switches categories, and a graph transformer
predicts the clean classes with cross-entropy [arxiv:2209.14734 §3.1]
[arxiv:2209.14734 §3.2]. Two additions carry most of the reported gain: noise
that keeps the training-set class frequencies [arxiv:2209.14734 §4.1], and cycle
and spectral features computed on the noisy graph [arxiv:2209.14734 §4.2].
Author-reported valid-unique-novel rates are 74% on SBM and 75% on planar graphs
[arxiv:2209.14734 tab. 1]. On QM9 the validity is 99.0% [arxiv:2209.14734
tab. 2]. Card: [DiGress](/papers/2209.14734).

### Discrete diffusion over layout tokens

LayoutDM writes each layout element as a category plus four quantized geometry
tokens, padded to 25 elements [arxiv:2303.08137 §3.2] [§4.1]. Each attribute
has its own corruption matrix, and a token is kept, swapped for another value
of its attribute, or masked [arxiv:2303.08137 §3.2] [eq. 3]. Known fields,
refinement and relation constraints are applied at inference, with no
retraining: known fields are masked back in at every step, and weak priors
shift the reverse distribution in log space [arxiv:2303.08137 §3.3]. Author-
reported unconditional Rico FID is 6.65, against 6.47 for LayoutTrans-fixed,
which uses a fixed element order [arxiv:2303.08137 tab. 2]. On refinement the
paper reports Rico FID 2.77 against 3.23 for the task-specific RUITE, but
PubLayNet FID 6.75 against 6.39 [arxiv:2303.08137 tab. 3]. Card:
[LayoutDM](/papers/2303.08137).

### Distilling a multi-step sampler into one step

The Denoising Student keeps a deterministic DDIM teacher and trains a student to
produce the teacher's final image from the same starting noise, with a plain
regression loss and no adversarial term [arxiv:2101.02388 §3.1]. The student
starts from the teacher's weights [arxiv:2101.02388 §3.2]. The teacher uses 100
DDIM steps for CIFAR-10 and 50 for LSUN [arxiv:2101.02388 §4]. Each training
pair needs one full teacher run, and the paper uses 1.024M pairs per dataset
[arxiv:2101.02388 §3.1]. Author-reported CIFAR-10 FID is 9.36 at one step,
against 4.16 for the 100-step teacher [arxiv:2101.02388 tab. 1]. Generating 50k
CIFAR-10 images takes 51.5 s for the student and 4.96 × 10³ s for the teacher, on
one V100 [arxiv:2101.02388 tab. 2]. The method needs a deterministic teacher, so
score-based and energy-based samplers do not qualify [arxiv:2101.02388 §2.1].
Card: [Denoising Student](/papers/2101.02388).

### Reusing high-level features across sampling steps

DeepCache changes no weights. A full U-Net pass runs every N steps, and the
steps in between reuse the high-level features cached at the last full pass
and recompute only the shallow blocks [arxiv:2312.00858 §3.3]. A non-uniform
spacing of the full passes, with a centre and a power, is also reported.
At N = 10 on ImageNet it gives FID 4.27, against 4.41 for uniform spacing
[arxiv:2312.00858 tab. 1]. Author-reported throughput on Stable Diffusion
v1.5 is 2.15x at N = 5, with a CLIP score of 29.46 against 29.51 for the
50-step sampler on PartiPrompts [arxiv:2312.00858 tab. 3]. On ImageNet with
LDM-4-G the same N gives 4.12x at FID 3.59, against 3.37 for the authors'
regenerated baseline [arxiv:2312.00858 tab. 1]. Quality falls as N grows: on
CIFAR-10, FID is 4.35 at N = 2 and 9.74 at N = 10 [arxiv:2312.00858 tab. 2].
Replacing the cached features with zeros is far worse, raising FID from 9.74
to 192.98 on CIFAR-10 [arxiv:2312.00858 tab. 5]. At equal throughput on
ImageNet, the authors report it matches DDIM with 59 steps, at FID 3.59 for
both [arxiv:2312.00858 tab. 4]. The 2.3x headline in the abstract does not
match any single table row, so check the card's Pitfalls before quoting it.
Card: [DeepCache](/papers/2312.00858).

### Chunked causal video student with a KV cache

CausVid turns a bidirectional video diffusion transformer into a causal one that
generates chunks of frames in order [arxiv:2412.07772 §4.1]. The student is first
initialised by regressing onto teacher ODE trajectories, using 1000 pairs
[arxiv:2412.07772 §4.3] [§5 Training]. It is then trained with asymmetric
distribution matching distillation: the teacher is bidirectional, and the
student sees only past chunks [arxiv:2412.07772 §4.2]. A separately trained fake
score is updated five times per student update [arxiv:2412.07772 §4.2]. At
inference each chunk is denoised in four steps while attending to cached keys and
values of earlier chunks [arxiv:2412.07772 §4.4]. Author-reported latency for a
10 s, 120-frame clip is 1.3 s, against 219.2 s for the bidirectional teacher, at
9.4 frames per second [arxiv:2412.07772 tab. 3]. The few-step student scores
94.7 on temporal quality, against 94.6 for the bidirectional many-step teacher
and 92.4 for a causal many-step model [arxiv:2412.07772 tab. 4]. Card:
[CausVid](/papers/2412.07772).

### Distilled causal world model for interactive video

Hunyuan-GameCraft-2 turns a 14B image-to-video mixture-of-experts model into a
few-step, causal, block-wise generator for interactive game video
[arxiv:2511.23429 §4.2]. Keyboard and mouse inputs become camera-control
signals, and free-form text goes through a multimodal LLM as extra interaction
guidance [arxiv:2511.23429 §4.1]. Distillation follows Self-Forcing, and the
first frame is kept as a sink token in the KV cache [arxiv:2511.23429 §4.2.3].
A long-video tuning stage rolls out clips longer than 10 seconds to limit drift
[arxiv:2511.23429 §4.2.4]. The authors report 16 FPS, with FP8 quantisation,
parallel VAE decoding and sequence parallelism. The GPU model and count are not
stated [arxiv:2511.23429 §4.4]. The InterBench judge is an unnamed VLM, and the
authors report semantic drift beyond about 500 frames [arxiv:2511.23429 §5.2.2]
[§6]. Card: [Hunyuan-GameCraft-2](/papers/2511.23429).

### Per-token noise levels for sequences and planning

Diffusion Forcing gives each token of a sequence its own independent noise
level, and a causal recurrent network carries a latent state over the tokens
[arxiv:2407.01392 §3.2]. Setting different noise levels for different tokens
yields a sampler that can generate long rollouts, guide over future tokens, and
plan over a flexible horizon [arxiv:2407.01392 §3.3] [§3.4]. Author-reported
average reward on the single-task Maze2D benchmarks is 141.7, against 119.5 for
Diffuser with its hand-built controller [arxiv:2407.01392 §4.2]. The paper says
its rollouts stay stable beyond the training horizon, shown qualitatively
[arxiv:2407.01392 §4.1]. Card: [Diffusion Forcing](/papers/2407.01392).

### Text-conditioned video as a planner

UniPi plans with a text-conditioned video diffusion model. The model takes the
observed first frame as conditioning, with the frame repeated over time, and it
is trained with that conditioning rather than only clamped at test time
[arxiv:2302.00111 §3.1] [app. A.1]. An inverse-dynamics model then reads actions
from the generated frames, and the actions are executed open-loop
[arxiv:2302.00111 §3.2]. Author-reported task completion on seen place
instructions is 59.1%, against 19.4% for the best state-based transformer
baseline [arxiv:2302.00111 tab. 1]. With a real robot, pretraining on internet
video raises success from 72.6% to 77.1% [arxiv:2302.00111 tab. 4]. Card:
[UniPi](/papers/2302.00111).

### Video diffusion features for robot actions

DiT4DiT couples a video diffusion transformer (Cosmos-Predict2.5-2B) to an
action diffusion transformer, and trains both with flow-matching losses
[arxiv:2603.10448 §4.2] [§4.3]. The action head reads hidden activations of one
video block at a fixed timestep, so control does not need generated frames
[arxiv:2603.10448 §4.2]. The authors report 98.6% average success on LIBERO,
with the model trained from scratch [arxiv:2603.10448 tab. 1]. Card:
[DiT4DiT](/papers/2603.10448).

### Diffusion action head inside a VLA, with a checker

FPC-VLA uses a diffusion transformer as the action head of a vision-language-
action policy. The head predicts the noise added to a ground-truth action
sequence, conditioned on a token from a Llama2 backbone [arxiv:2509.04018 §3.2]
[eq. 3] [eq. 4]. The paper does not state the number of denoising steps
[arxiv:2509.04018 §3.2]. The diffusion head is not the paper's main addition. A
separate Qwen2.5-VL supervisor is queried only when the gripper state changes,
and it can add a directional correction to the action [arxiv:2509.04018 §3.5]
[alg. 2]. Past predictions are averaged with cosine and time weights before the
supervisor sees them [arxiv:2509.04018 §3.3] [alg. 1]. Author-reported SIMPLER
WidowX average task success is 64.6, against 51.8 for CogACT, with the model not
fine-tuned [arxiv:2509.04018 tab. 3]. Without the supervisor, the ablation's task
average falls from 82.1 to 74.4 [arxiv:2509.04018 tab. 7]. With the paper's
stated fusion constants, the similarity factor changes the pose weights by about
10% at most, by our arithmetic from [arxiv:2509.04018 eq. 8] and [§4.1], so the
fusion is close to a plain average. Card:
[FPC-VLA](/papers/2509.04018).

### Joint video and action denoising in one transformer

VideoVLA starts from the pretrained CogVideoX-5B video diffusion transformer.
One denoising network takes the text, the current observation, the future video
latents and the action tokens as one sequence, and denoises the video and a
6-step action chunk together [arxiv:2512.06963 §3.2] [§3.4]. Both are noised
under the DDPM schedule and trained with the diffusion loss, and the timestep
enters through adaptive LayerNorm [arxiv:2512.06963 §3.4]. Inference uses 50
denoising steps, and the first three actions are executed before the model is
queried again [arxiv:2512.06963 §4.1]. Author-reported SIMPLER average over 12
tasks is 63.0, against 62.6 for CogACT [arxiv:2512.06963 tab. 1]. On novel
objects it reports 65.2 against 42.4 for CogACT, and on new skills 48.6 against
20.4 [arxiv:2512.06963 tab. 2] [tab. 3]. Without the video loss, the same two
generalisation columns fall to 12.7 and 4.4 [arxiv:2512.06963 tab. 9]. Training
from scratch instead of from the video model gives an average of 12.6, against
80.4 for the default, on the three-task ablation [arxiv:2512.06963 tab. 7]. The
authors report about 1.1 s per chunk with 10 steps on one H100, roughly 3 Hz
[arxiv:2512.06963 app. D]. The WidowX tables use 24 trials per task, so one
success is 4.2 points, and the 0.4-point gap to CogACT is within that noise
[arxiv:2512.06963 app. A]. Card:
[VideoVLA](/papers/2512.06963).

### Diffusion proposals under tree search

STORM keeps a diffusion policy as the action proposer, and adds a search loop
around it. The proposer is CogACT-Base, used without further training, and it
returns K = 8 candidate actions with priors [arxiv:2512.18477 §3.1] [§4.1]. Each
candidate is rolled forward in a video world model, and Monte Carlo Tree Search
picks the root action with the most visits [arxiv:2512.18477 §3.2] [§3.3]. The
world model is autoregressive over discrete tokens, not a diffusion model
[arxiv:2512.18477 §3.2]. Author-reported WidowX mean success is 51.0, against
47.9 for CogACT, and the authors call the margin modest [arxiv:2512.18477 tab. 1]
[§4.3]. The paper gives no latency figures for the search [arxiv:2512.18477 §5].
Card: [STORM](/papers/2512.18477).

### Video generation with one transformer for images and clips

W.A.L.T trains one transformer diffusion model on images and videos in a shared
latent space from a causal autoencoder, where an image is a one-frame video
[arxiv:2312.06662 §4.1]. The latent has 8 channels by default [arxiv:2312.06662
§5.2]. Attention is local: spatial windows of one frame and spatiotemporal
windows across frames, alternating [arxiv:2312.06662 §4.2] [§5.2]. Conditioning
uses AdaLN-LoRA with rank 2 and self-conditioning with probability 0.9
[arxiv:2312.06662 §4.3] [§5.2]. Author-reported FVD on Kinetics-600 frame
prediction is 3.3, against 4.3 for MAGVIT-v2 [arxiv:2312.06662 tab. 1]. On
ImageNet 256 the large model reaches FID 2.56 without classifier-free guidance
[arxiv:2312.06662 tab. 2]. Self-conditioning cuts the UCF-101 FVD from 109.9 to
61.4 [arxiv:2312.06662 tab. 3]. A three-model cascade makes text-to-video
[arxiv:2312.06662 §4.5]. Card: [W.A.L.T](/papers/2312.06662).

### Image-to-video from a fine-tuned image model

DreamPose starts from Stable Diffusion, fine-tunes it on fashion videos to
follow a pose sequence, then runs a short per-subject fine-tune on the user's
photos [arxiv:2304.06025 §4.2] [§4.3]. Each frame is sampled on its own,
conditioned on the photo and five consecutive pose maps, so temporal smoothness
comes from the conditioning rather than from a joint video model
[arxiv:2304.06025 §4.1]. Author-reported results on UBC Fashion are state of
the art against MRAA, TPSMM and PIDM [arxiv:2304.06025 tab. 1]. The set-up is
slow for each new subject: about 10 minutes of UNet tuning and 20 minutes of
VAE-decoder tuning, plus 18 s per rendered frame [arxiv:2304.06025 §6]. Card:
[DreamPose](/papers/2304.06025).

### Correlated frame noise when finetuning an image model to video

PYoCo starts from a pretrained text-to-image diffusion model and finetunes it
jointly on images and videos. The change is in the noise: frames from one clip
draw correlated starting noise, not independent noise [arxiv:2305.10474 §4].
Two constructions are given, a mixed one with a shared part and a per-frame part
[arxiv:2305.10474 eq. 3], and a progressive one in which each frame perturbs the
previous frame's noise [arxiv:2305.10474 eq. 4]. Each frame keeps unit variance
[arxiv:2305.10474 §4]. On UCF-101 the authors' strategy table gives FVD 903.37
for training from scratch, 566.67 for finetuning with independent noise, and
337.40 and 339.67 for the mixed and progressive priors [arxiv:2305.10474 tab. 5].
Zero-shot text-to-video on UCF-101 reaches IS 47.76 and FVD 355.19
[arxiv:2305.10474 tab. 2]. The sweep over the correlation strength used only a
69M-parameter model, and the large-scale run uses progressive noise with α = 2
[arxiv:2305.10474 §5.3]. The card records no official code. Card:
[PYoCo](/papers/2305.10474).

### Audio-driven portraits without a skeleton

EMO draws each video frame directly from one photo and one audio clip. It uses
no 3D face model, landmark sequence or pose template [arxiv:2402.17485 §1]
[§3]. A reference network keeps the identity, audio attention drives the
motion, and a face-box mask and a head-speed embedding act as deliberately weak
conditions [arxiv:2402.17485 §3.2]. Long videos are built from 12-frame clips
that share four motion frames with the previous clip [arxiv:2402.17485 §3.2.4]
[§4.1]. Author-reported FID on HDTF is 8.76, the best in its table, and FVD is
67.66 [arxiv:2402.17485 tab. 1]. The authors state that their SyncNet score is
lower than Wav2Lip and SadTalker [arxiv:2402.17485 tab. 1]. Card:
[EMO](/papers/2402.17485).

### Real-time talking faces in a learned face space

VASA-1 learns a disentangled face latent for identity, head pose, facial
dynamics and a 3D appearance volume. A diffusion transformer then generates
pose and dynamics codes from wav2vec 2.0 audio features [arxiv:2404.10667 §3.1].
Because it generates compact codes rather than pixels, rendering is fast and
streams in short overlapping chunks [arxiv:2404.10667 §3.1]. The authors report
512×512 video at up to 40 FPS on one RTX 4090, and the best lip sync and FVD
against MakeItTalk, Audio2Head and SadTalker on two benchmarks they built
[arxiv:2404.10667 §4]. The code is not public, and the paper does not give its
training losses or several architecture details [arxiv:2404.10667 §3.1]. Card:
[VASA-1](/papers/2404.10667).

### Joint audio and video in one diffusion transformer

JavisDiT has a video branch and an audio branch, linked by bidirectional
cross-attention [arxiv:2503.23377 §3.1]. A learned spatio-temporal prior maps
the prompt to 32 spatial and 32 temporal tokens, and cross-attention injects
them into both branches [arxiv:2503.23377 §3.2] [app. E.3]. Training has three
stages, and the last one trains only the cross-attention layers
[arxiv:2503.23377 §3.3]. Author-reported JavisScore is 0.154 against 0.151 for
FoleyCrafter on JavisBench, while UniVerse-1 is better on FVD, KVD and TV-IB
[arxiv:2503.23377 tab. 1]. Card: [JavisDiT](/papers/2503.23377).

### Joint audio and video from two coupled denoisers

MM-Diffusion runs one U-Net for video and one for audio, and denoises both at
each step [arxiv:2212.09478 §3.2] [arxiv:2212.09478 §3.3]. The two streams are
linked by random-shift cross-modal attention: each audio segment attends to a
window of video frames whose start is drawn at random, which is cheaper than
full cross-attention [arxiv:2212.09478 §3.3]. Audio from video, or video from
audio, is generated with no extra training, by replacing the condition with a
noised copy of the real one or by gradient guidance [arxiv:2212.09478 §3.4].
Author-reported FVD is 117.20 on Landscape and 75.71 on AIST++, both with full
1,000-step DDPM sampling. The faster DPM-Solver sampler gives 229.08 on
Landscape [arxiv:2212.09478 tab. 1] [arxiv:2212.09478 tab. 2]. The conditional
generation is shown in figures, not in a table [arxiv:2212.09478 §4.5]. In a
user study, 84.9% of Landscape clips were judged real, against 92.5% for real
clips [arxiv:2212.09478 tab. 6]. The baseline is a two-stage pipeline, with
checkpoints and conditioning not fully stated, and it scores far lower on video
quality, 1.61 against 3.75 on Landscape [arxiv:2212.09478 §4.6] [arxiv:2212.09478
tab. 5]. Card: [MM-Diffusion](/papers/2212.09478).

### Reward-weighted and DPO fine-tuning from an AI judge

The authors fine-tune a 3B text-to-video diffusion model offline, on 120
prompts about object movement [arxiv:2412.02617 §4] [app. A]. The reward is a
binary accept or reject from a vision-language model that reads eight frames
[arxiv:2412.02617 §3.3] [app. D]. Reward-weighted regression (RWR) and DPO are
both derived from one probabilistic objective [arxiv:2412.02617 §3.1]. On the
authors' Gemini "All" column, RWR on AI feedback goes from 52.66% to 56.41%, and
DPO goes to 56.17%. On human "All" accuracy the gains are 19.38% to 31.09%
(RWR) and 34.06% (DPO) [arxiv:2412.02617 tab. 2]. The judge is from the same
model family as the reward, and an independent GPT-4o check gives a smaller
gain of about 2.4 points for RWR [arxiv:2412.02617 tab. 2]. The evaluation has
40 test prompts. Imaging quality in the VBench table is below the pre-trained
model in every row [arxiv:2412.02617 tab. 2]. Card: [AI-feedback
fine-tuning](/papers/2412.02617).

### Learned preference reward used on a late denoising step

ImageReward trains a scorer on pairwise human rankings of generated images
[arxiv:2304.05977 §2.1] [§2.2]. The scorer is a BLIP backbone with a small MLP
head, trained with a pairwise logistic loss on the score gap [arxiv:2304.05977
§2.2] [eq. 1]. Author-reported preference accuracy on 6,399 held-out comparisons
is 65.14%, against 57.76% for the BLIP score and 54.82% for CLIP score
[arxiv:2304.05977 tab. 3]. ReFL uses the same scorer to tune Stable Diffusion
v1.4. Each update draws one late denoising step at random, decodes the result
and back-propagates the reward [arxiv:2304.05977 §3] [alg. 1]. Against the
untuned model, ReFL wins 58.79% of human comparisons on real-user prompts
[arxiv:2304.05977 tab. 4]. The released ReFL code differs from the paper in its
loss, its regulariser and its training length, so see the card's Paper vs code
section before reproducing it. Card: [ImageReward](/papers/2304.05977).

### Adversarial term added to one-step distillation

Adversarial Diffusion Distillation starts a student from the teacher's weights
[arxiv:2311.17042 §3.1]. A discriminator on frozen ViT features, with several
small heads, is trained with a hinge loss, and the student is trained to fool it
[arxiv:2311.17042 §3.2]. A score distillation term, a squared-L2 match to the
frozen teacher's clean estimate on re-noised student samples, is added with
weight 2.5 [arxiv:2311.17042 §3.3] [§4]. One student evaluation gives an image,
with no classifier-free guidance [arxiv:2311.17042 §1] [§4.2]. Author-reported,
one-step ADD-M reaches COCO FID-5k 19.7 and CLIP 0.326, the best in its table
among SD1.5-based few-step methods [arxiv:2311.17042 tab. 2]. In human
preference, one step of ADD-XL beats four steps of LCM-XL, and four steps beat
SDXL at 50 steps in most comparisons [arxiv:2311.17042 §4.2]. The student is
less diverse than its teacher [arxiv:2311.17042 §4.3]. The learning rate, batch
size and iteration count for ADD-M and ADD-XL are not stated, and the released
repository has no training code for them (see the card's Paper vs code section).
The card also records that the released SDXL-Turbo weights carry a
non-commercial licence, so do not assume commercial use of them. Card:
[Adversarial Diffusion Distillation](/papers/2311.17042).

### Fused multi-view tiles lifted to 3D

One-2-3-45++ draws six fixed views in one 3×2 tile, using a fine-tuned Stable
Diffusion 2 conditioned on the input image [arxiv:2311.07885 §3.1]. The tile is
then lifted by two diffusion stages over occupancy and sparse signed-distance
volumes, conditioned on DINOv2 features projected from the six views and max-
pooled [arxiv:2311.07885 §3.2]. A colour field is fitted last [arxiv:2311.07885
§3.3]. Author-reported F-Score on the GSO set is 93.6 and CLIP similarity 81.0,
at about 60 s per object [arxiv:2311.07885 tab. 1]. The paper gives no
implementation details beyond what is above, and training used 8 A100 GPUs for
about 10 days per 3D stage [arxiv:2311.07885 §3.2]. Card:
[One-2-3-45++](/papers/2311.07885).

### Video diffusion as a prior for physical motion

PhysDreamer animates a still 3D Gaussian object with an image-to-video model,
then fits a stiffness value for each region, together with an initial velocity,
so that a differentiable physics simulation matches the generated clip
[arxiv:2404.13026 §4] [fig. 3]. The method assumes elastic materials, no
collisions and one constant Poisson's ratio [arxiv:2404.13026 §3] [§6].
Author-reported 2AFC study on eight real scenes: 80.8% of raters prefer it to
PhysGaussian on motion realism, and 63.5% prefer it to DreamGaussian4D
[arxiv:2404.13026 tab. 1]. The optimisation is slow [arxiv:2404.13026 §6]. Card:
[PhysDreamer](/papers/2404.13026).

### Latent diffusion for 3D shapes with per-input weights

SDFusion compresses truncated signed-distance grids with a 3D VQ-VAE, then runs
diffusion in that latent [arxiv:2212.04493 §3.1] [§3.2]. Partial shapes, images
and text enter through separate encoders and cross-attention. Each input gets a
weight at sampling time, and the paper's rule adds one weighted difference per
input to the unconditional prediction [arxiv:2212.04493 §3.3] [eq. 4].
Author-reported shape-completion UHD on ShapeNet is 0.0557, against 0.0567 for
AutoSDF, with a larger diversity score [arxiv:2212.04493 tab. 1]. On Pix3D the
F-score is 0.432, against 0.415 for AutoSDF [arxiv:2212.04493 tab. 2]. The
per-input rule is implemented only in one code branch, so the released sampler
differs from the equation [arxiv:2212.04493 §3.3]. Card:
[SDFusion](/papers/2212.04493).

### Composable conditions trained by random subsets

Composer splits each training image into eight representations, among them a
caption, a colour histogram, a sketch, instance masks, depth and a grayscale
version [arxiv:2302.09778 §2.2]. Each representation is dropped independently
with probability 0.5, and all are dropped or all kept with probability 0.1 each
[arxiv:2302.09778 §2.3]. Because of this training, chosen representations can be
recombined at inference for layout, palette and style control
[arxiv:2302.09778 §3.2]. The cascade runs at 64, then 256, then 1024 pixels
[arxiv:2302.09778 §2.3]. The base model is trained on about one billion images
[arxiv:2302.09778 §3.1]. Author-reported COCO FID is 9.2 and CLIP score 0.28
[arxiv:2302.09778 §3.5]. The paper names no baseline numbers, so only the
absolute values can be quoted. Card: [Composer](/papers/2302.09778).

### Box tokens with object-aware cross-attention, trained from scratch

Each layout object is a box and a class. A transformer over these tokens lets
the objects see each other, and each image patch carries its own box position
so that cross-attention can match patches to objects [arxiv:2303.17189 §3.2]
[§3.3]. The model is trained from scratch on box-labelled images, so it has no
text prompt and no pretrained image prior [arxiv:2303.17189 §5]. Training uses
classifier-free dropout at 0.2 and sampling uses DPM-Solver in 25 steps
[arxiv:2303.17189 §3.4] [tab. 4]. On COCO-Stuff at 256 px, the author-reported
FID is 15.61, against 29.10 for PLGAN, a GAN baseline [arxiv:2303.17189 tab. 1].
On COCO-Stuff at 128 px, removing both the layout fusion and the object-aware
attention gives FID 29.94, against 16.57 for the full model [arxiv:2303.17189
tab. 2]. The released code differs from the paper in several settings, so
reproduce from the card's Paper vs code notes. Card:
[LayoutDiffusion](/papers/2303.17189).

### Training-free guidance of a pretrained model at sampling time

Semantic diffusion guidance adds language, reference-image or combined guidance
to a pretrained unconditional DDPM [arxiv:2112.05744 §3.1]. At each step the
mean is shifted along the gradient of a CLIP-based score, and the CLIP image
encoder is adapted to noisy images by a contrastive finetune with no text labels
[arxiv:2112.05744 §3.5]. Author-reported FFHQ image-guidance FID is 14.37,
against 17.15 for ILVR [arxiv:2112.05744 tab. 1]. For text guidance FID is
28.38, against 57.45 for StyleGAN+CLIP, but the baseline has higher retrieval
scores [arxiv:2112.05744 tab. 1]. Raising the scale on LSUN cats from 100 to 200
raises top-1 retrieval and lowers LPIPS, so the scale trades match against
diversity [arxiv:2112.05744 tab. 2]. The structure-preserving content loss is
not in the released code [arxiv:2112.05744 §3.3]. Card:
[Semantic diffusion guidance](/papers/2112.05744).

MultiDiffusion uses one pretrained text-to-image model on overlapping windows
and averages the window outputs at each pixel and step, with no training
[arxiv:2302.08113 §3] [alg. 1]. The same averaging, with masks as weights,
handles region prompts [arxiv:2302.08113 §4.2]. Author-reported panorama FID is
10.3, against 18.4 for Blended Latent Diffusion and 45.5 for Stable Inpainting
[arxiv:2302.08113 tab. 1]. On a COCO subset the region IoU is 0.26, against 0.17
and 0.16 for those two baselines [arxiv:2302.08113 tab. 2]. Bootstrapping, which
masks the views in early steps, improves the IoU from 0.18 to 0.26
[arxiv:2302.08113 §4.2] [tab. 2]. Cost grows with the panorama area, since each
step calls the model once per window [arxiv:2302.08113 app. B.1]. Card:
[MultiDiffusion](/papers/2302.08113).

### Backward guidance on cross-attention maps of a frozen model

The model's weights stay fixed. During the first denoising steps, the latent is
moved along the gradient of an energy that is small when each prompt word's
cross-attention mass lies inside its target box [arxiv:2304.03373 §3.2]
[eq. 2] [eq. 3]. The default applies this for the first 10 steps, with up to
five updates per step, on the mid-block and first up-block layers of Stable
Diffusion v1.5 [arxiv:2304.03373 §4.1]. The same method is also applied to a
GLIGEN model, where it raises COCO mAP from 62.8 to 65.1 [arxiv:2304.03373 tab.
3]. Against a variant that edits the attention maps directly, the authors
report better VISOR conditional accuracy at about twice the per-image time
[arxiv:2304.03373 tab. 1]. On VISOR, conditional accuracy is 95.95% for Stable
Diffusion with this guidance, against 62.98% without it [arxiv:2304.03373 tab.
2]. Stable Diffusion's COCO mAP rises from 19.2 to 35.7 [arxiv:2304.03373 tab.
3]. Check the card's Paper vs code notes before using the released code, since
it lacks the attention-map variant and the benchmark scripts. Card:
[Training-Free Layout Control with Cross-Attention Guidance](/papers/2304.03373).

### Attention modulation for dense captions and regions

DenseDiffusion keeps a frozen Stable Diffusion model and changes only its
attention during the first 30% of denoising steps [arxiv:2308.12964 §3.2]. Each
prompt segment comes with a binary region map. Word-to-pixel scores inside a
region are raised and scores outside it are lowered, scaled by the score range
and by the region's area [arxiv:2308.12964 §3.2] [eq. 2]. On 250 MS-COCO
captions the authors report SOA-I 77.61, against 73.08 for Stable Diffusion
[arxiv:2308.12964 tab. 2]. For layout fidelity the IoU is 34.99, against 23.76
for SD-Pww, the only baseline with layout control [arxiv:2308.12964 tab. 3].
Dropping the area term raises IoU to 38.00 but lowers SOA-I to 73.62, so the
terms trade text fidelity against layout [arxiv:2308.12964 tab. 4]. Each
region must be given as a mask, and each region's phrase must appear in the
prompt [arxiv:2308.12964 §3]. Card: [DenseDiffusion](/papers/2308.12964).

### Shared attention for one style across a batch

StyleAligned gives a set of images one shared style without training, on a
pretrained SDXL model [arxiv:2312.02133 §3.2] [§4]. In each self-attention
layer, every image attends to its own tokens and to those of one reference
image, after AdaIN on queries and keys [arxiv:2312.02133 §3.2]. Sharing across
all images gives consistent styles but leaks content and reduces diversity,
which is why only the reference is shared [arxiv:2312.02133 §3.2] [§4.1]. The
paper reports about 29 s for a four-image set on one A100 [arxiv:2312.02133
§4]. The authors' tests found that encoder-based personalisation methods
struggle to separate style from content [arxiv:2312.02133 §4.2]. Card:
[StyleAligned](/papers/2312.02133).

### Tuning-free multi-subject personalization with localized attention

Each subject token in the prompt is replaced by an MLP applied to that token's
text embedding and a CLIP image embedding of one reference photo of the subject
[arxiv:2305.10431 §4.1] [eq. 2]. Other tokens keep their text embedding, so no
training run is needed for each person. During training a localization loss
pushes each subject token's cross-attention inside its segmentation mask, with
weight 0.001 [arxiv:2305.10431 §4.2] [eq. 3] [eq. 4]. The masks and this loss
are used only in training. At sampling time the early steps use the plain text
prompt, and the last fraction alpha uses the subject-augmented prompt. The paper
suggests alpha between 0.6 and 0.8 [arxiv:2305.10431 §4.3] [fig. 5]. Training
pairs come from captioned images whose noun phrases are matched to segments
[arxiv:2305.10431 §5.1]. On the authors' single-subject test, identity
preservation is 51.41% with one reference image, against 43.37% for Custom
Diffusion with five [arxiv:2305.10431 tab. 1]. On the multi-subject test it is
43.11%, against 13.52% for Textual Inversion [arxiv:2305.10431 tab. 2]. Without
the localization loss, identity is 37.66% and prompt consistency is 25.03%,
against 24.25% with it [arxiv:2305.10431 tab. 3]. The released code's
embedding MLP is deeper than the single one written in eq. 2, so the equation
alone does not reproduce the checkpoint. Card:
[FastComposer](/papers/2305.10431).

### Plug-and-play priors and posterior sampling

A diffusion model trained on its own data can serve as a prior for a different
task, with a differentiable constraint added at inference and no further training
[arxiv:2206.09012 §1] [§2.1]. The method minimises the denoising error minus the
log constraint, drawing fresh noise at each step, with the noise level annealed
from high to low [arxiv:2206.09012 §2.2] [alg. 1]. On a 50-node travelling
salesman problem the author-reported gap is 1.28% without beam search
[arxiv:2206.09012 tab. 2]. On land-cover segmentation, the fine-tuned version
reaches 79.8% accuracy on Durham [arxiv:2206.09012 tab. 1]. Each experiment was run
once, and inference is stochastic [arxiv:2206.09012 §2.2]. Card:
[Plug-and-play priors](/papers/2206.09012).

For inverse problems in medical imaging, a score prior replaces the learned
reconstruction network. For MRI, the method draws posterior samples by annealed
Langevin dynamics on the measurement model, and the prior is trained on T2 brain
slices without the sampling pattern [arxiv:2108.01368 §2.2] [§4]. The
released sampler does not implement the stated data-term equation, so the
published numbers come from the code's rule [arxiv:2108.01368 §2.2]. On a
high-resolution brain, the authors report 16 minutes and 0.95 GB on an
RTX 2080Ti [arxiv:2108.01368 §4.1]. A radiologist study ranks it first on knees,
with mean rank 1.17 against 1.87 for MoDL, but the authors call it preliminary
[arxiv:2108.01368 tab. 1] [app. H]. Card: [Robust CS-MRI](/papers/2108.01368).

### Physics-informed reconstruction of fluid fields from coarse or sparse input

A diffusion model trained only on fine two-dimensional flow fields reconstructs
them from coarse or sparse inputs. The input is noised to an intermediate step,
and the reverse chain runs from there instead of from pure noise
[arxiv:2211.14680 §2.3]. Sparse inputs are filled in first and refined over
several rounds, each round using the previous output as its guide
[arxiv:2211.14680 §2.3.1]. An optional term is the gradient of the equation
residual, given to the network as a condition or subtracted in the sampler
[arxiv:2211.14680 §2.3.1] [arxiv:2211.14680 alg. 2]. On Kolmogorov flow at
Reynolds number 1000, the authors report that the diffusion model has a lower
equation loss than a direct-mapping UNet on out-of-distribution inputs, with L2
close to it. For 4x upsampling the values are 0.2178 against 0.5438 for equation
loss, and 0.5622 against 0.6048 for L2 [arxiv:2211.14680 tab. 1]. Card:
[Physics-informed diffusion](/papers/2211.14680).

### Text-guided object inpainting with a mask-precision input

A fine-tuned Stable Diffusion model fills a masked region with the object named
by a short label, and the user chooses how closely the object follows the mask
[arxiv:2212.05034 §4.1] [§4.2]. Training blurs real instance masks to several
precision levels, from the exact outline to the bounding box, and adds an output
that predicts the exact mask so that background pixels inside a coarse box are
kept [arxiv:2212.05034 §4.2] [§4.3]. The number of levels and the blur settings
are not stated [arxiv:2212.05034 §4.2]. On OpenImages with bounding-box masks,
the author-reported local FID is 9.71, against 12.57 for Stable Inpainting
[arxiv:2212.05034 tab. 1]. Card: [SmartBrush](/papers/2212.05034).

### Medical segmentation as mask diffusion

MedSegDiff treats a segmentation mask as the signal to diffuse, conditioned on
the scan [arxiv:2211.00611 §2]. Two additions carry the method. Dynamic
conditional encoding fuses the current noisy mask into the image encoder
[arxiv:2211.00611 §2.1] [eq. 3]. FF-Parser is a learned frequency filter on the
skip features [arxiv:2211.00611 §2.2] [eq. 4]. Inference draws 25 samples of 100
steps and fuses them with STAPLE [arxiv:2211.00611 §3.2]. Author-reported Dice
for MedSegDiff++ is 87.5 on optic cups, 90.5 on brain tumours and 86.6 on thyroid
nodules [arxiv:2211.00611 tab. 1]. Dynamic conditioning adds 2.1, 1.6 and 1.8
Dice points on the three tasks [arxiv:2211.00611 tab. 2]. The released fusion
code differs from the equation [arxiv:2211.00611 §2.1]. Card:
[MedSegDiff](/papers/2211.00611).

### Mask diffusion trained end to end, without a pretrained backbone

SegDiff is close to MedSegDiff above in one respect: the mask is the signal
that gets diffused, and the image conditions the noise predictor
[arxiv:2112.00390 §4]. The difference is in how the image enters. It passes
through a convolutional encoder with residual-in-residual dense blocks, and its
features are summed with the features of the noisy mask before the U-Net
encoder [arxiv:2112.00390 §4] [eq. 17]. The model is trained from random
initialisation, while several of the baselines it is compared with use
ImageNet, PASCAL or COCO pretraining [arxiv:2112.00390 §5]. Author-reported mean
IoU on the eight Cityscapes object classes is 76.44 with a 15% box expansion,
against 73.70 for the best listed baseline [arxiv:2112.00390 tab. 1]. On
Vaihingen the F1 is 95.14 against 94.80 [arxiv:2112.00390 tab. 2], and on
MoNuSeg the Dice is 81.59 against 79.55 [arxiv:2112.00390 tab. 3]. At test time
several reverse chains start from different noise, are averaged and then
thresholded. The paper uses 30 chains in its main experiments
[arxiv:2112.00390 §4.1]. The released code fixes nine chains for every dataset,
so the reported tables cannot be checked against the code alone. Inference time
grows with the number of chains, and the paper says it is slower than Segformer
and STDC [arxiv:2112.00390 §5]. Card: [SegDiff](/papers/2112.00390).

### Reusing one pretrained motion model

Motion diffusion can be extended without retraining the base model.
DoubleTake joins prompted intervals by denoising overlapping intervals together,
then refining each join [arxiv:2303.01418 §3.1]. It uses a one-second overlap, the
handshake, of 20 frames on HumanML3D [arxiv:2303.01418 §4.1]. Author-reported
BABEL transition FID is 1.88, against 3.86 for TEACH at a 70-frame margin
[arxiv:2303.01418 tab. 1]. ComMDM runs two frozen copies of the base model side by
side, with a small coordination block trained on 10 two-person sequences
[arxiv:2303.01418 §3.2] [§4.2]. DiffusionBlending mixes control-tuned models with
equal weights, and fine-tuning zeroes the noise on controlled features
[arxiv:2303.01418 §3.3] [§4.3]. On a composite wrist-and-trajectory control,
the authors report FID 0.22, against 1.18 for inpainting with the base model
[arxiv:2303.01418 tab. 4]. Card: [Human motion diffusion](/papers/2303.01418).

### Forecasting with a diffusion sampler per time step

TimeGrad forecasts several series jointly, one time step at a time. An LSTM
summarises the history, and a conditional diffusion model draws each next value
[arxiv:2101.12072 §3]. Inference uses 100 diffusion steps and 100 sample paths
[arxiv:2101.12072 §4.2]. Author-reported CRPS_sum is the lowest of the compared
methods on five of six datasets, not on Exchange [arxiv:2101.12072 tab. 2].
Each forecast step runs the denoiser once per diffusion step, so sampling is the
slow part [arxiv:2101.12072 §6]. Card: [TimeGrad](/papers/2101.12072).

### Pretrained generator as a data source

CIFAKE does not train a diffusion model. The authors sample 60,000 images from
Stable Diffusion v1.4, 50 steps with the Euler Ancestral sampler, rendered at
512 px and reduced to 32 px, matched to the CIFAR-10 classes [arxiv:2303.14126
§3.2] [§3.5]. They then train a small CNN to separate real from generated images,
and report 93.55% accuracy for the final model [arxiv:2303.14126 tab. 7]. The
guidance scale and per-image seeds are not stated, so the images cannot be
regenerated exactly [arxiv:2303.14126 §3.5]. Only one generator and one source
dataset are tested [arxiv:2303.14126 §5]. Card: [CIFAKE](/papers/2303.14126).

Imagen fine-tuned on ImageNet gives a larger version of the same idea. The
authors fine-tune the 64 px base and the 64-to-256 super-resolution model on
ImageNet-1K training images [arxiv:2304.08466 §4.1], then sample for each class
a number of images matching the real count, scaled from 1x to 10x, with the
class name as the prompt [arxiv:2304.08466 §4.3]. Author-reported FID at 256 px
is 1.76 on the train reference [arxiv:2304.08466 tab. 1]. A ResNet-50 trained on
real images reaches 76.39% top-1, and 78.17% when 1.2M generated images are added
[arxiv:2304.08466 tab. 3]. At 256 px the gain persists to a total of 4.8M
images, and the total falls below the real-only baseline from 6.0M
[arxiv:2304.08466 tab. 4]. Guidance and sampler settings
were chosen on the validation set that is also used for the headline numbers
[arxiv:2304.08466 §4.2]. Card: [Synthetic data from diffusion models](/papers/2304.08466).

### Diffusion views for test-time prompt tuning

DiffTPT tunes four CLIP context tokens on each unlabelled test image, one image
at a time [arxiv:2308.06038 §3.1] [§4.1]. Each image gets 128 views: the
original, 64 crop-and-flip views and 63 Stable Diffusion image variations
[arxiv:2308.06038 §4.1]. For these variations the usual text condition is
replaced by the CLIP image embedding of the test image, because no label is
available [arxiv:2308.06038 §3.3]. A cosine-similarity filter keeps the 80% of
views most similar to the input, and an entropy filter then keeps the 30%
lowest-entropy views [arxiv:2308.06038 §3.4] [§4.1]. Author-reported ten-dataset
averages are 59.85 against 56.98 for TPT with RN50, and 65.47 against 63.99 with
ViT-B/16 [arxiv:2308.06038 tab. 2]. These are gains of 2.87 and 1.48 points,
below the headline figure (see Contradictions). Card:
[DiffTPT](/papers/2308.06038).

### Adversarial prompts against unlearned diffusion models

Safety unlearning changes a text-to-image model so that a concept, a style or
an object is no longer drawn. This paper tests five such unlearned models with a
white-box attack that prepends three to five optimised tokens to the prompt
[arxiv:2310.11868 §3] [§4]. The objective is the victim model's own denoising
error on a target image that contains the forgotten content, so no auxiliary
diffusion or classifier model is needed [arxiv:2310.11868 §4]. On I2P nudity
prompts against ESD, attack success rises from 20.42% with no attack to 76.05%,
against 69.71% for the P4D baseline [arxiv:2310.11868 tab. 2]. Read it as a
stress test of a forgetting fix, and check the card's pitfalls before reusing
its numbers. Card: [Unlearned diffusion models](/papers/2310.11868).

### Metrics for generated designs, with a diffusion topology case

This is a review and guide to scoring generative models in engineering design,
not a new generator. It sorts its metrics into similarity, novelty and
diversity, constraint satisfaction, performance and target achievement, and
conditioning [arxiv:2302.02913 §1] [tab. 1]. Its argument is that a score for
matching the training distribution can rank designs or models wrongly, and it
shows this on simple 2-D data [arxiv:2302.02913 §1] [§11.1]. The only diffusion
model in its experiments is TopoDiff, an external model [arxiv:2302.02913 §10.3].
Two TopoDiff variants are compared on the first 300 conditions of the level-1
test data, with 100 diffusion steps [arxiv:2302.02913 §10.4]. The median
compliance error is 1.99% with FEA fields and 9.61% with kernel fields, and the
mean volume-fraction error is 1.49% and 1.69% [arxiv:2302.02913 tab. 7]. The
mean inference time is 5.87 s and 2.35 s [arxiv:2302.02913 tab. 7]. The
diversity row is bolded in a way that favours the kernel-field variant, so the
direction of each metric has to be taken from [arxiv:2302.02913 tab. 1]. The
repository holds the metric code and the 2-D experiments, not TopoDiff or the
case-study data, so the topology numbers cannot be rerun from it. Card:
[Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](/papers/2302.02913).

### Sampling-step theory for low-dimensional data

Under original DDIM and DDPM coefficients, the paper shows that the total
variation error falls below a target eps after about k / eps steps, where k is an
intrinsic dimension of the data, with no smoothness assumption
[arxiv:2501.12982 §3.1]. The assumptions are a polynomially bounded support and
a covering-number notion of dimension [arxiv:2501.12982 §3.1]. The bounds extend
to learned scores under error conditions, which are stronger for DDIM than for
DDPM [arxiv:2501.12982 §3.2]. A one-step lower bound shows the standard
coefficients are nearly necessary for this adaptation [arxiv:2501.12982 §3.3].
The paper is theory only, with no experiments [arxiv:2501.12982 §1]. Card:
[Low-dimensional adaptation](/papers/2501.12982).

### Likelihood-trained diffusion with a learned noise schedule

The noise schedule is a learned monotone function, trained jointly with a
U-Net on the continuous-time variational bound. The loss depends on the
schedule only through its endpoints, so the interior is free to be chosen for
lower variance [arxiv:2107.00630 §3.2] [§5.1]. Author-reported test bits per
dimension are 2.65 on CIFAR-10 without augmentation and 3.40 on ImageNet 64.
These are bounds, not exact likelihoods [arxiv:2107.00630 tab. 1]. A model
trained in continuous time and evaluated with 10 steps gives 7.54 bits per
dimension, so many steps are needed [arxiv:2107.00630 tab. 2]. Card:
[Variational Diffusion Models](/papers/2107.00630).

### Perception-weighted loss over noise levels

The usual training loss is a simple noise-prediction error summed over noise
levels. This variant multiplies it by one extra factor, 1 / (k + SNR(t))^gamma,
which lowers the weight of the lowest-noise steps
[arxiv:2204.00227 §3.2] [eq. 8]. The authors split the steps into a coarse
stage, a content stage and a clean-up stage, and argue that the high-SNR steps
mainly teach imperceptible detail [arxiv:2204.00227 §3.1]. They use k = 1 and
gamma of 0.5 or 1 [arxiv:2204.00227 §3.4]. The factor is a closed-form function
of the step, so it adds no trainable parameters [arxiv:2204.00227 §3.2].
Author-reported FID-50k falls with this weight: FFHQ from 7.86 to 6.92 at 1000
steps, CUB from 9.60 to 6.95 at 1000 steps, and MetFaces from 44.34 to 36.80 at
250 steps [arxiv:2204.00227 tab. 1]. With 250 uniform steps, a P2 model reaches
FID-10k 8.92, against 9.74 for the baseline at 500 steps. That table does not
name the dataset [arxiv:2204.00227 tab. 4]. The schedule still matters: the
cosine schedule is much worse than the linear one, even with the weight
[arxiv:2204.00227 §3.3] [fig. 4]. Card:
[Perception Prioritized Training of Diffusion Models](/papers/2204.00227).

### Zero terminal SNR schedule with v prediction and rescaled guidance

Common schedules keep some signal at the last step, so the training input there
differs from the pure noise given at sampling time [arxiv:2305.08891 §1]
[§3.1]. The fix rescales the schedule to zero terminal SNR, trains with v
prediction, samples from the last timestep, and rescales classifier-free
guidance [arxiv:2305.08891 §3.1] [§3.2] [§3.3] [§3.4]. Applied to a fine-tuned
Stable Diffusion 2.1-base, with DDIM at 50 steps and 10k COCO captions, the
authors report FID 21.66 against 23.76 for the official model [arxiv:2305.08891
tab. 3]. Their comparison also changes the training data, so the gain on the same
data is from 22.96 to 21.66 [arxiv:2305.08891 tab. 3]. The brightness gain is
shown in pictures only [arxiv:2305.08891 §4.1]. Card: [Common Diffusion Noise
Schedules and Sample Steps are Flawed](/papers/2305.08891).

### Diffusion prior in CLIP image-embedding space, then a diffusion decoder

A prior maps a caption to a CLIP image embedding, and a GLIDE-style diffusion
decoder renders the image from that embedding [arxiv:2204.06125 §2]. The
diffusion prior predicts the clean embedding rather than the noise. It gives a
zero-shot MS-COCO FID of 10.39, against 10.63 for the autoregressive prior
[arxiv:2204.06125 §2.2] [tab. 2]. Because the embedding is a compact summary,
the same model supports variations and interpolations between images
[arxiv:2204.06125 §3]. Card:
[Hierarchical Text-Conditional Image Generation with CLIP Latents](/papers/2204.06125).

### Transformer backbone in place of a U-Net

Latent diffusion's U-Net is replaced by a transformer on patches of the VAE
latent. Timestep and class enter through adaLN-Zero, which regresses a scale,
shift and gate per block and starts each block as the identity [arxiv:2212.09748
§3.2]. Across 12 configurations, FID falls as forward-pass Gflops rise
[arxiv:2212.09748 §5] [tab. 4]. The reported FID is 2.27 for DiT-XL/2 with
classifier-free guidance at 1.50, on class-conditional ImageNet at 256×256
[arxiv:2212.09748 tab. 2]. Card:
[Scalable Diffusion Models with Transformers](/papers/2212.09748).

### Frozen-model adapters for spatial control

A small adapter reads a control map (sketch, segmentation, keypose, depth or
colour) and adds multi-scale features to the encoder of a frozen Stable
Diffusion UNet [arxiv:2302.08453 §3.3]. Several adapters can be combined with
hand-set weights, with no further training [§3.3] [§5]. On COCO validation, the
FID is 16.78 with text plus segmentation, against 24.68 for Stable Diffusion
with text only [arxiv:2302.08453 tab. 1]. Injecting into the decoder as well as
the encoder was worse in the authors' ablation [arxiv:2302.08453 §4.4.1]
[tab. 2]. Card: [T2I-Adapter](/papers/2302.08453).

### Larger latent UNet with size and crop conditioning

The base model's UNet is about three times the size of Stable Diffusion 1.x and
reads the prompt with two text encoders [arxiv:2307.01952 §2.1]. Original size
and crop offsets are embedded as extra conditions, so training images are not
discarded or cropped badly [§2.2]. An optional refiner, run over the first 200
of 1000 noise levels, adds detail [§2.5]. In the authors' four-way human study
the refiner-enhanced model wins 48.44% of votes against 7.91% for SD 1.5
[arxiv:2307.01952 §2.5]. Card: [SDXL](/papers/2307.01952).

### Score distillation on unseen views for sparse-view 3D

A diffusion model supplies a score-distillation loss on random camera views that
the training photos do not cover [arxiv:2312.00206 §3.3.1]. The loss is switched
on after two-thirds of training, once the geometry is stable [§4.1]. On
Mip-NeRF360 with 12 views, the author-reported PSNR is 19.37, against 17.49 for
3D Gaussian Splatting [arxiv:2312.00206 tab. 1]. The ablation gives 1.37 dB from
softmax-scaled depth [§4.3]. Card:
[SparseGS](/papers/2312.00206).

### One-step diffusion repair of Gaussian-splatting renders

The scene stays a plain Gaussian splatting fit, and the diffusion model only
cleans its renders. A single-step enhancer, derived from SD-Turbo and the public
Difix checkpoint, maps each render and its nearest source frame to a cleaner
frame [arxiv:2510.12308 §3.2]. It is fine-tuned on about 36k pairs of corrupted
and clean frames, with dynamic objects masked out of the loss [§3.2]. Feeding
enhanced views back into the fit improved the visuals but not the metric, so it
was left out of the submission [§4]. Author-reported: the enhancer raises the
devset score from 0.423 to 0.462 [arxiv:2510.12308 tab. 1], and the challenge
leaderboard gives 0.432, second of 28 [§1]. The paper does not state the licence
of the Difix checkpoint [§3.2]. Card:
[Hybrid Gaussian Splatting for Novel Urban View Synthesis](/papers/2510.12308).

### Distribution matching into one step

A one-step generator is trained so its outputs match the teacher's distribution.
Its gradient is the difference of two score estimates, one from the frozen
teacher and one from a fake-score denoiser trained alongside, and an LPIPS
regression to teacher outputs is added with weight 0.25 [arxiv:2311.18828 §3.2]
[§3.3]. On class-conditional ImageNet 64×64 it reports FID 2.62 in one forward
pass, against 2.32 for the EDM teacher with 512 [arxiv:2311.18828 tab. 1]. For
zero-shot COCO-30k text-to-image it reports FID 11.49 at 0.09 s per image, against
8.78 at 2.59 s for Stable Diffusion v1.5 [tab. 3]. Card:
[One-step Diffusion with Distribution Matching Distillation](/papers/2311.18828).

### Motion diffusion that predicts the clean motion

A transformer encoder predicts the clean motion at each step, not the noise, so
constraints such as foot contact can be applied to the prediction
[arxiv:2209.14916 §1] [§3]. One model handles text, action and unconditional
generation through classifier-free guidance [§3]. On HumanML3D the author-reported
FID is 0.544, against 1.067 for T2M [arxiv:2209.14916 tab. 1]. On KIT it is
0.497, against 2.770 for T2M [tab. 2]. Card:
[Human Motion Diffusion Model](/papers/2209.14916).

### Painted views lifted to editable procedural material graphs

A segment-conditioned image model renders one view of a 3D shape with labelled
parts [arxiv:2404.17569 §3.1]. Segments are grouped by material [§3.2], and each
group's procedural material graph is chosen from a library and fitted by
differentiable rendering [§3.3]. The output is a graph that a designer can edit,
not a fixed colour texture [§3.5]. Author-reported FID on chairs is 88.3,
against 94.6 for the best baseline [arxiv:2404.17569 tab. 1]. On ABO objects it
is 87.3, against 109.8 [tab. 1]. Timing is 7 minutes per shape, or 33 minutes
without grouping [§4.4]. Card: [MaPa](/papers/2404.17569).

### Per-pixel features from a frozen denoiser for segmentation

Label-efficient segmentation does not draw samples from the model. It reads
the decoder activations of a pretrained DDPM. Each labelled image is noised to
steps 50, 150 and 250, and five decoder blocks are upsampled and concatenated
into an 8448-dimensional vector per pixel [arxiv:2112.03126 §3.2]. An ensemble
of ten small MLPs is trained on the pixels of a few annotated images, and a
majority vote gives each pixel its label [arxiv:2112.03126 §3.2] [app. C].
Author-reported mean IoU is the highest in every column of the main table, for
example 49.4 on Bedroom-28 with 40 training images [arxiv:2112.03126 tab. 2]
[§4]. With 10 training images the same method scores 38.2 on that dataset
[arxiv:2112.03126 tab. 4]. The authors expect the method to need a diffusion
model trained on the target domain, which they think is hard for complex data
such as ImageNet [arxiv:2112.03126 §5]. Card:
[Label-efficient segmentation](/papers/2112.03126).

### Frozen denoiser features for dense correspondence

DIFT runs no sampling loop. A real image is noised to a chosen level t, passed
once through a pretrained denoiser, and the activations of one upsampling block
serve as a per-pixel descriptor. The activations are averaged over several noise
draws [arxiv:2306.03881 §4.2] [app. C]. Points are matched by nearest neighbour
on cosine similarity [arxiv:2306.03881 §3]. The noise level and block are set
per task. A small t keeps detail and a large t keeps semantics [arxiv:2306.03881
§4.2] [app. C]. Author-reported SPair-71k per-image PCK is 52.9 for the Stable
Diffusion features, against 33.9 for DINO and 38.4 for OpenCLIP, and 55.3 for
the supervised SCorrSAN [arxiv:2306.03881 tab. 1]. On PF-WILLOW the diffusion
features score 81.2, against 80.0 for SCorrSAN [arxiv:2306.03881 tab. 3]. On
HPatches they trail the weakly supervised CAPS at 3 and 5 pixels [arxiv:2306.03881
tab. 4]. One denoiser pass takes 203 ms per 768 px image on one A6000 GPU
[arxiv:2306.03881 app. B]. Caveats: the noise level and block were
grid-searched on SPair-71k, which is also the benchmark reported
[arxiv:2306.03881 §5.2]. PF-PASCAL is left out of the main results, because
302 of its 506 test images also appear in training [arxiv:2306.03881 app. D].
Card: [Emergent Correspondence from Image Diffusion](/papers/2306.03881).

### CLIP loss on a subdivision mesh, with a diffusion prior as an optional term

The method optimises the control vertices of a subdivision surface, a texture
and a normal map from a text prompt, with no 3D training data [arxiv:2203.13333
§3]. The main loss is the negative cosine similarity between CLIP embeddings of
random rendered views and the prompt, plus a Laplacian smoothness term
[arxiv:2203.13333 eq. 1] [eq. 2]. A diffusion prior that samples a CLIP image
embedding from the prompt can be added as a weighted term [arxiv:2203.13333 eq.
3] [eq. 4]. The R-precision table leaves that term out, because the prior was
trained on ViT-B/32 embeddings [arxiv:2203.13333 §4.3]. Random backgrounds,
cameras and offsets are the paper's answer to texture-only solutions
[arxiv:2203.13333 §3]. Author-reported CLIP R-precision is 96.7 with ViT-B/16
for both generation and evaluation, against 93.5 for Dreamfields [arxiv:2203.13333
tab. 1]. The authors report about 50 minutes per shape on one 16 GB GPU
[arxiv:2203.13333 §4.1]. Card: [CLIP-Mesh](/papers/2203.13333).

### Score distillation through a volume renderer

Score Jacobian Chaining lifts a 2D diffusion model to 3D without 3D training
data [arxiv:2212.00774 §4]. A voxel radiance field is rendered from random
cameras, and the denoiser's score is passed back through the renderer by the
chain rule [arxiv:2212.00774 §4]. Clean renders lie outside the denoiser's
training distribution, so the score is estimated by adding noise to each render
and averaging the denoised results [arxiv:2212.00774 §4.1]. An emptiness loss
with a two-stage weight suppresses floating density [arxiv:2212.00774 §4.2].
The text-to-3D results use Stable Diffusion v1.5 and are shown only as pictures,
with no quantitative metric [arxiv:2212.00774 §5.2]. The authors report about
25 minutes per object on one A6000 [arxiv:2212.00774 app. A3]. Card:
[Score Jacobian Chaining](/papers/2212.00774).

### Coarse-to-fine score distillation into a textured mesh

Magic3D generates a 3D asset from text in two stages [arxiv:2211.10440 §4]. The
first stage optimises a hash-grid neural field with score distillation against
a 64x64 base diffusion prior [arxiv:2211.10440 §4.1]. The field is then turned
into a tetrahedral mesh, and a 512x512 latent diffusion prior refines its
geometry and texture [arxiv:2211.10440 §4.2] [§4.3]. The authors report about
40 minutes on 8 A100 GPUs, and 61.7% of user-study raters prefer its videos to
DreamFusion's on "more realistic and detailed" [arxiv:2211.10440 §5] [tab. 1].
That comparison uses DreamFusion's published videos, and the DreamFusion runtime
is quoted from that paper, not measured [arxiv:2211.10440 §1]. The text guidance
weight and the angular regularisation weight are not stated [arxiv:2211.10440
§3] [§4.3]. Card: [Magic3D](/papers/2211.10440).

### Single-view mesh from a multi-view diffusion model

One-2-3-45 does not optimise a shape for each object. Zero123, a fine-tuned
Stable Diffusion model, renders 32 posed views around the input
[arxiv:2306.16928 §4.1]. The input's elevation is estimated from four of those
views by feature matching [arxiv:2306.16928 §3.4]. A feed-forward SDF
cost-volume network built on SparseNeuS fuses the views into a mesh
[arxiv:2306.16928 §3.3]. Author-reported F-score averages 83.5 over GSO and
Objaverse, the best in its table, and CLIP similarity is second to Shap-E
[arxiv:2306.16928 tab. 1]. The runtime is about 45 s on an A100, against about
15 minutes for the Zero123 with Stable Diffusion baseline [arxiv:2306.16928
tab. 1]. The authors' ablation finds that training only on ground-truth views
fails at inference [arxiv:2306.16928 §4.3]. Card:
[One-2-3-45](/papers/2306.16928).

### Point-voxel diffusion for point clouds

Point-voxel diffusion runs DDPM-style denoising on clouds of 2048 points, with a
point-voxel CNN as the denoiser [arxiv:2104.03670 §3]. One network does
unconditional generation and completion. In completion, the observed points are
held fixed and only the missing points are denoised [arxiv:2104.03670 §3.2].
Author-reported 1-NN is the best of the compared generators on all six
generation cells [arxiv:2104.03670 tab. 1]. Sampling uses 1000 steps, and the
authors state that 100 steps are not enough [arxiv:2104.03670 app. H.2]. In
completion from single depth views, the method is best on EMD in all three
categories and worse than the baselines on CD in all three [arxiv:2104.03670
tab. 2]. Card: [Point-voxel diffusion](/papers/2104.03670).

### Equivariant diffusion over molecular coordinates

GeoDiff generates 3D atom coordinates for a molecular graph [arxiv:2203.02923
§4]. Its reverse kernels are SE(3)-equivariant, and its starting density has
zero centre of mass, so the sampled density is invariant to rotation and
translation [arxiv:2203.02923 §4.2]. Training reduces to a weighted
noise-prediction loss [arxiv:2203.02923 §4.3]. Author-reported mean COV-R on
GEOM-Drugs is 89.13%, against 67.16% for GeoMol [arxiv:2203.02923 tab. 1]. The
default sampler runs 5000 steps [arxiv:2203.02923 tab. 4], and a 1000-step
variant is weaker [arxiv:2203.02923 tab. 6]. Card: [GeoDiff](/papers/2203.02923).

### Diffusion prior for real-world super-resolution

StableSR keeps a Stable Diffusion 2.1-base model frozen. It trains a
time-aware encoder whose features modulate the U-Net through scale-and-shift
layers [arxiv:2305.07015 §3.1]. An optional fidelity module, weighted by a
user value w, trades realism for faithfulness [arxiv:2305.07015 §3.2]. Large
inputs are processed as overlapping 64×64 latent patches, blended with Gaussian
weights at each step [arxiv:2305.07015 §3.3]. On DIV2K-Valid the author-reported
FID is the best in the table at 24.44, but PSNR is 23.26, below several
baselines [arxiv:2305.07015 tab. 1]. The 200-step sampler takes 15.16 s per
512 px output on one V100, and a 4-step SD-Turbo variant takes 0.83 s
[arxiv:2305.07015 tab. 5]. Card: [StableSR](/papers/2305.07015).

### Diffusion over a compact restoration prior

DiffIR does not diffuse pixels. A pretrained encoder turns the clean image into
a 256-value restoration prior, and a diffusion model learns to predict that
prior from the damaged input [arxiv:2303.09472 §4.1] [§4.2]. Training has two
stages: the encoder first, then the diffusion model and the restorer together
[arxiv:2303.09472 §4.1] [§4.2]. Sampling uses four steps [arxiv:2303.09472
§5.1], and a transformer restorer produces the image from the predicted prior
[arxiv:2303.09472 §4.1]. Author-reported inpainting FID with wide masks is
5.14 on CelebA-HQ and 1.98 on Places, against 5.70 and 2.25 for LaMa, at about
the same parameter count [arxiv:2303.09472 tab. 1]. On x4 super-resolution over
DIV2K100 the PSNR is 29.13, against 28.62 for BebyGAN and 28.58 for SRdiff
[arxiv:2303.09472 tab. 2]. Motion deblurring on GoPro reaches 33.20 PSNR,
against 32.92 for Restormer [arxiv:2303.09472 tab. 3]. Training the two stages
jointly gives CelebA-HQ FID 5.14, against 5.98 with the traditional scheme
[arxiv:2303.09472 tab. 4]. Caveats: the efficiency claim against RePaint is
shown in a figure and is not derivable from the tables [arxiv:2303.09472
fig. 1]. The released code uses a sqrt-linear noise schedule and predicts the
clean prior, not the noise the paper's equations use [arxiv:2303.09472 §5.1]
[eq. 11]. Card: [DiffIR](/papers/2303.09472).

### Semantic tags and a low-resolution start for real-photo upscaling

A tagger is fine-tuned on degraded inputs, so its tags and embeddings on a
damaged photo match what a frozen tagger gives on the clean image
[arxiv:2311.16518 §3.2] [eq. 1]. The tag text goes through the text encoder.
The tagger's embeddings enter through representation cross-attention layers in
a ControlNet-style branch on Stable Diffusion 2-base [arxiv:2311.16518 §3.3]
[eq. 2]. Sampling starts from the low-resolution latent noised to the training
schedule, not from pure noise. The authors argue that pure-noise starts let the
model treat degradation as content to enhance, and they say the low-resolution
start removes false sky texture [arxiv:2311.16518 §3.4] [fig. 3]. Author-reported:
best FID, CLIPIQA and MUSIQ where the table gives those metrics, and PSNR and
SSIM below the best baseline on the three paired sets [arxiv:2311.16518 tab. 2].
In a semantic check on degraded COCO images, a COCO-trained detector and
segmenter score 30.0 panoptic PQ, against 27.2 for DiffBIR [arxiv:2311.16518
§4.3]. Card: [SeeSR](/papers/2311.16518).

### Camera-controlled video with ray encodings

UCPE gives each video token its own viewing ray in a local frame, so lens
distortion and field of view enter the attention layers [arxiv:2512.07237 §3.2].
A latitude-and-up map adds absolute pitch and roll [arxiv:2512.07237 §3.2]. The
encoding enters a Wan2.1 T2V 1.3B model through a parallel attention adapter
whose output projection starts at zero [arxiv:2512.07237 §3.3] [app. D]. The
adapter has 35.6M trainable parameters [arxiv:2512.07237 tab. 1]. On a
synthetic lens dataset the author-reported rotation error is 4.12° against
9.23° for ReCamMaster [arxiv:2512.07237 tab. 1]. Zero-shot on RealEstate10K it
is 0.56° against 1.10° [arxiv:2512.07237 tab. 2]. Card:
[UCPE](/papers/2512.07237).

### Diffusion policy with a visuo-tactile encoder

Touch in the Wild first pretrains a visuo-tactile encoder by reconstructing
masked tactile images, with a CLIP ViT-B/16 on the camera image and
cross-attention between the two embeddings [arxiv:2507.15062 §4.2]. The
encoder then feeds a diffusion policy, which samples an action chunk in 16
denoising steps [arxiv:2507.15062 §4.3] [tab. 5]. The authors report 0.85
whole-task success on the test-tube task with pretraining, 0.70 without it and
0.25 with vision only [arxiv:2507.15062 tab. 1]. Each cell is 20 trials, so one
trial moves a rate by 0.05. Card:
[Touch in the Wild](/papers/2507.15062).

### Diffusion action head on a pretrained robot transformer

Octo tokenizes the language or goal image, the camera views and one readout
slot per time step, and runs one transformer over them [arxiv:2405.12213 §3.1].
The readout embedding goes to a small diffusion head, which denoises an action
chunk over 20 steps on a cosine schedule [arxiv:2405.12213 app. D]. The
transformer runs once per prediction, and only the head loops
[arxiv:2405.12213 §3.3]. The paper reports that this head beats a continuous
MSE head and a discretized action head, at 83% against 35% and 18% aggregate
success on the WidowX ablation [arxiv:2405.12213 tab. 2]. The ablations use
Octo-Small and 40 trials, so the gap is measured on one small setup.

Finetuning updates all parameters with about 100 target trajectories and 50k
steps, and a new sensor or action space gets a new small input encoder or head
[arxiv:2405.12213 §3.3] [§3.1]. The authors report 72% average success over six
real-robot setups, against 20% for training from scratch, with 20 trials per
setup [arxiv:2405.12213 tab. 1]. The paper says the pretrained model was
trained for about 14 hours on a TPU v4-128 for the Base size, and each
finetune took about 5 hours on one A5000 [arxiv:2405.12213 §3.4]. Card:
[Octo](/papers/2405.12213).

### Diffusion expert on a vision-language backbone, trained in stages

DexVLA puts a vision-language model (Qwen2-VL-2B) in front of a separate
diffusion transformer of about 1B parameters, the action expert
[arxiv:2502.05855 §3.1] [app. C.3]. The expert is conditioned on the VLM through
FiLM layers, and the same VLM also writes sub-step text as intermediate
output [arxiv:2502.05855 §3.1] [§3.2]. The loss is the diffusion loss on
actions plus the next-token loss on that text, with weight 1 in all runs
[arxiv:2502.05855 §3.1]. The paper does not state the noise schedule, the
prediction target or the loss form [arxiv:2502.05855 §3.1].

Training has three stages: the expert alone on about 100 hours of
cross-embodiment data, then the whole model on one embodiment, then
task-specific finetuning, which the authors skip for simpler tasks
[arxiv:2502.05855 §3.2] [app. C.4]. In the paper's ablation, skipping stage 1
gives zero shirt-folding success, and stage 3 is needed for laundry folding,
where the score goes from 0.0 to 0.4 [arxiv:2502.05855 §4.4]. The expert size also
matters on shirt folding: 0.17 for a 93M U-Net, 0.63 for a 410M diffusion
transformer and 0.92 for the 1B one [arxiv:2502.05855 §4.4]. These are the
authors' results on their own robots, with 10 trials per task
[arxiv:2502.05855 §4]. Card: [DexVLA](/papers/2502.05855). The official code
is licensed non-commercially, so do not copy it, and see the card's Paper vs
code section for the places where it departs from the paper.

### Diffusion policy co-trained with a large diverse dataset

Co-training fills each training batch half from a task's own demonstrations and
half from a large external dataset [arxiv:2403.12945 §5.1] [app. F.2]. The policy
is a diffusion policy. Two external views at 128x128 go through an
ImageNet-pretrained ResNet-50, a frozen DistilBERT embedding of the instruction
joins the robot state, and a U-Net head denoises 16 action steps, of which eight
run open loop [arxiv:2403.12945 §5.1] [app. F.1]. Author-reported over six
tasks, the co-trained policy beats the next-best method by 22 points
in-distribution and 17 points out-of-distribution, both averaged over tasks
[arxiv:2403.12945 §5.2]. Each task has ten A/B rollouts per method, so the
per-task differences are noisy [§5.2]. The paper's sampler and schedule do not
all match the released configs, so check the card's Paper vs code section
before reproducing the setup. Card: [DROID](/papers/2403.12945).

### Latent diffusion over grasp latents, with a per-task denoiser

Grasps are compressed by a conditional VAE, and a DDPM is trained on the VAE
latents rather than on poses. The VAE is trained first with the ELBO, and its
KL weight is annealed from 1e-7 to 0.1 over the first half of training
[arxiv:2312.11243 §3.1] [arxiv:2312.11243 eq. 2]. The VAE is then frozen, and
the DDPM learns the latent with the noise as the target [arxiv:2312.11243 §3.2]
[arxiv:2312.11243 eq. 6]. The forward process has 1000 steps with a linear
schedule [arxiv:2312.11243 §3.3]. DDIM with 100 steps can replace ancestral
sampling without retraining [arxiv:2312.11243 §4.3]. On the simulated 63C
objects with full point clouds, the median success is 78% [arxiv:2312.11243
§4.1]. The sampler table gives 0.792 for DDPM with 1000 steps, at 7.39 s per 100
grasps, and 0.756 for DDIM with 100 steps, at 0.75 s [arxiv:2312.11243 tab. 1].
A new task only retrains the small denoiser, according to the authors
[arxiv:2312.11243 §4.2]. Card: [GraspLDM](/papers/2312.11243).

### Latent diffusion for hand grasps, with a frozen-prior adaptation step

A hand autoencoder gives a 768-d latent, which the denoiser treats as 256
tokens of 3 values. The denoiser is conditioned on a PointNet embedding of
3000 object surface points [arxiv:2411.14786 §3.2] [arxiv:2411.14786 eq. 8].
Sampling uses DDIM [arxiv:2411.14786 §3.4]. Training happens in two steps.
After the autoencoder and the diffusion model are trained, both are frozen, and
a small MLP predicts an offset to the latent [arxiv:2411.14786 §3.3]
[arxiv:2411.14786 eq. 9]. The authors report that training the diffusion model
directly on physical constraints oscillated and did not converge
[arxiv:2411.14786 §3.3]. On out-of-domain HO-3D, the penetration volume is 5.23
against 7.37 for GraspTTA, at 0.14 s per grasp [arxiv:2411.14786 tab. 2]. The
step count N is not stated in the paper [arxiv:2411.14786 §3.4]. Card:
[FastGrasp](/papers/2411.14786).

### Object-then-scene diffusion for LiDAR

Two diffusion models run in sequence. An object model denoises object point
clouds of 1024 points, voxelised at size 32, and a scene model denoises range
images of height 64 on KITTI-360 [arxiv:2412.17226 app. A]. The object
condition is a CLIP text vector with a Fourier-embedded box [arxiv:2412.17226
eq. 1-3]. The scene model uses ControlNet-style zero-convolution features from
the object range image [arxiv:2412.17226 eq. 7]. The authors' reason is that
foreground points are usually under 10% of a scene's points, so scene-level
models are biased to background [arxiv:2412.17226 §3.2]. Reported KITTI-360
results are FPD 7.60 against 25.1 for UltraLiDAR, and JSD 0.42 against 0.71
[arxiv:2412.17226 tab. 1]. Sampling steps and guidance are not stated
[arxiv:2412.17226 §3.2]. Card: [OLiDM](/papers/2412.17226).

### Diffusion over action chunks, scaled to a billion parameters

The policy is a DDPM over action chunks, with adaptive layer norm in place of
cross-attention, and with the causal mask removed from self-attention
[arxiv:2409.14411 §3.2] [arxiv:2409.14411 fig. 2]. Depth, width and heads are
scaled together, from 10M to 1B parameters [arxiv:2409.14411 tab. 1]. On the
Franka tasks, the average success is 87.50 for the largest model against 51.25
for DP-T [arxiv:2409.14411 tab. 2]. On bimanual UR5, it is 98.33 against 23.33
[arxiv:2409.14411 tab. 3]. Removing the causal mask from ScaleDP-L drops the
average from 90.00 to 66.66 [arxiv:2409.14411 tab. 4]. The optimizer,
learning rate, step count and chunk length are not stated [arxiv:2409.14411
§3]. Card: [ScaleDP](/papers/2409.14411).

### Policy-gradient fine-tuning of the denoising chain

A diffusion policy is first trained by behaviour cloning, then fine-tuned with
PPO. Each denoising step is treated as a Gaussian policy with an analytic
likelihood, and the inner chain is nested inside the environment MDP
[arxiv:2409.00588 §4.1] [arxiv:2409.00588 eq. 6]. Reward arrives only at the
last denoising step [arxiv:2409.00588 eq. 4]. The paper's best practices fine-tune
only the last K' steps, or use a few DDIM steps with stochastic training
and deterministic evaluation [arxiv:2409.00588 §4.3] [arxiv:2409.00588 eq. 12].
Reported results include more than 90% on Robomimic Transport from state
[arxiv:2409.00588 §5.1], and 80% (16 of 20) zero-shot on FurnitureBench One-Leg
hardware [arxiv:2409.00588 §5.4]. Fine-tuning the Gaussian head without BC
regularisation gave 88% in simulation and 0% on hardware, and adding the BC
term gave 53% and 50% [arxiv:2409.00588 §5.4]. Training from scratch with 10
denoising steps was about six times slower than the Gaussian policy
[arxiv:2409.00588 app. D]. Card: [DPPO](/papers/2409.00588).

### Diffusion over action chunks for dexterous manipulation with a VLM planner

A fine-tuned VisualGLM-6B plans the task, and a transformer action-diffusion
policy (GCAD) predicts a chunk of actions at each denoising step
[arxiv:2409.05493 §3.1] [arxiv:2409.05493 §3.2]. The policy is conditioned on
the observation and on an embedded return-to-go [arxiv:2409.05493 §3.2]. It
executes two actions before replanning [arxiv:2409.05493 app. A.5]. In
simulation, the average success is 0.70 for this system and 0.23 for DP-T
[arxiv:2409.05493 tab. 1]. On a real UR5e, it is 0.65 (26 of 40)
[arxiv:2409.05493 tab. 2]. The paper says the transformer backbone beats the
CNN diffusion backbone, and that BC and decision transformer fail to learn the
task [arxiv:2409.05493 §4.2]. Card: [DexDiff](/papers/2409.05493).

### Diffusion as a baseline on a large motion dataset

In the piano benchmark, diffusion policies (a U-Net, DP-U, and a transformer,
DP-T) are baselines trained with DDPM, and no diffusion design is proposed
[arxiv:2408.11048 app. C.1]. With 12 songs, the in-distribution average F1 is
0.539 for DP-U, 0.357 for DP-T and 0.529 for BC-MLP [arxiv:2408.11048 tab. 2].
On 150 songs, evaluated out of distribution, it is 0.256 for DP-U and 0.316 for
DP-T [arxiv:2408.11048 tab. 2]. The paper does not state the step count for the
transformer baseline [arxiv:2408.11048 app. C.1]. Card:
[RP1M](/papers/2408.11048).

### Diffusion policy for humanoid whole-body control

An action-sequence diffusion policy, following DiffuseLoco, controls a simulated
Unitree H1 humanoid [arxiv:2411.01349 §4]. Its training data is generated by a
source RL policy under eight domain-randomization setups, with 0.5M, 2M and 8M
transitions [arxiv:2411.01349 §3] [arxiv:2411.01349 §5]. The best policy, trained
on 8M transitions from all setups, reaches success 1.0 in both the
non-randomized and the randomized evaluation [arxiv:2411.01349 tab. 3]
[arxiv:2411.01349 tab. 5]. The card states that terrain and perturbation
randomization give the most robust policies, and that walking needs far more
data than manipulation [arxiv:2411.01349 §1] [arxiv:2411.01349 §6].
Hyperparameters are deferred to DiffuseLoco and the prediction horizon is not
stated [arxiv:2411.01349 §4]. Card: [The Role of Domain Randomization in Training
Diffusion Policies for Whole-Body Humanoid Control](/papers/2411.01349).

### Hierarchical diffusion with a contact guider

Two conditional diffusion models run at each control step. The Guider denoises
a contact target, which is each finger's position in the object frame plus a
touch flag [arxiv:2411.12982 §3.2]. The Actor denoises an action sequence,
predicting 16 steps and executing 8 in open loop [arxiv:2411.12982 tab. 8]
[arxiv:2411.12982 fig. 2]. The Guider uses 100 denoising steps and the Actor 10
[arxiv:2411.12982 tab. 9]. The Actor loss adds a Q-value term with weight
0.001, and the paper uses a one-step denoising form for it [arxiv:2411.12982
eq. 9] [arxiv:2411.12982 tab. 9] [arxiv:2411.12982 §4.4]. The authors report a
20.8% average success gain over Diffusion Policy across six tasks
[arxiv:2411.12982 §6]. On Move-T stage 2, the success is 0.60 against 0.30 for
Diffusion Policy [arxiv:2411.12982 tab. 5]. The authors state that Diffusion Policy
performs worse on contact-rich, multi-contact tasks than on single-contact
ones [arxiv:2411.12982 §1].
The 20.8% figure is not derived in the text, so it is quoted as reported.
Card: [HDP](/papers/2411.12982).

### Trajectory diffusion for visual navigation

A 1D U-Net denoises a sequence of relative waypoints, conditioned on a
transformer output that combines a depth-CNN, a target-position encoder and a
goal encoder [arxiv:2412.20226 §II-C] [arxiv:2412.20226 §II-D]. Training uses
100 denoising iterations, and deployment uses 5 [arxiv:2412.20226 §III-A]. The
horizon is 32 with 16 waypoints executed, and the goal is masked with
probability 0.7 during training [arxiv:2412.20226 §III-A]. On a Unitree Go2,
with 25 trials per difficulty level, the hard-level success is 84% for this
policy, against 20% for ViNT with a relative-distance goal and 12% for
BC-ConvMLP [arxiv:2412.20226 tab. I]. Only one robot was tested, so transfer
to other embodiments is not shown [arxiv:2412.20226 tab. I]. Card: [ViDEN](/papers/2412.20226).

### Goal-masked diffusion policy for exploration and goal reaching

One Transformer policy serves both jobs. During training the goal token is
masked for about half of the samples. Masking gives undirected exploration at
test time, and an unmasked goal photo gives goal-reaching
[arxiv:2310.07896 §IV-A]. The action head is a diffusion model over short
sequences of 2-D moves, denoised in 10 steps and conditioned on the observation
context [arxiv:2310.07896 §IV-B] [§IV-C]. The training loss adds a
temporal-distance term with weight 1e-4 [arxiv:2310.07896 §IV-C] [eq. 2]. The
sampling update in the paper leaves its coefficients unspecified
[arxiv:2310.07896 eq. 1]. Real-robot tests on a LoCoBot in unseen indoor and
outdoor places give 98% exploration success with 19M parameters and 0.2
collisions, against 77% and 1.7 collisions for Subgoal Diffusion with 335M
parameters [arxiv:2310.07896 tab. 1]. Navigation success is 90%, equal to
Random Subgoals and Subgoal Diffusion in the same table [tab. 1]. The unified
model reaches 98% undirected and 92% goal-conditioned, against 92% for the
goal-conditioned ViNT policy [arxiv:2310.07896 tab. 2]. Trial counts are not
stated [arxiv:2310.07896 §V-A]. Card: [NoMaD](/papers/2310.07896).

### Diffusion policy with audio from a contact microphone

A diffusion policy head is conditioned on a fused vision and audio feature, and
on the last two end-effector poses, at every denoising step [arxiv:2406.19464
§3.2]. The microphone is mounted on the gripper finger [arxiv:2406.19464
§3.1]. Training adds two background overlays to the audio, each with
probability 0.5 [arxiv:2406.19464 §3.2]. On wiping, the success is 85% with
audio and 40% with vision alone [arxiv:2406.19464 §4.2]. On flipping, it is 90%
against 25% [arxiv:2406.19464 §4.1]. The authors name objects that make little
sound, quasi-static tasks and robot motor noise as limits [arxiv:2406.19464
§4.5]. The step count is not stated in the paper [arxiv:2406.19464 §3.2]. Card:
[ManiWAV](/papers/2406.19464).

### Diffusion policy for physics-based character control

A diffusion policy over action sequences is trained by behaviour cloning on
states paired with the clean expert action for the state reached
[arxiv:2406.00960 §3.2] [arxiv:2406.00960 §3.3]. On the perturbation task, the out-of-distribution
success is 96.3 for this policy, against 91.3 for the best C-VAE and 81.0 for
an MLP [arxiv:2406.00960 tab. 3]. Tracking on the AMASS test set gives 97.1 in
the table [arxiv:2406.00960 tab. 2]. The authors name inference cost as the
main limit, since it is K times the cost of an MLP [arxiv:2406.00960 §6].
Card: [PDP](/papers/2406.00960).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Need spatial control (boxes, keypoints) on a pretrained latent text-to-image model, and cannot retrain it | Frozen backbone with gated added layers (GLIGEN style) | Original weights are kept, and the gate starts at zero [arxiv:2301.07093 §4.2] |
| Need the added condition to apply to the whole run | Frozen backbone without scheduling | Scheduling turns the added conditions off after `tau * T` steps [arxiv:2301.07093 eq. 10] |
| Want the layout fixed early but the detail left to the original model | Scheduled conditioning, with the paper's `tau = 0.2` as a starting point | The authors use it for their qualitative figure only [arxiv:2301.07093 §5.4] |
| Need a generator for a short, fixed-size clip (for example motion), where sampling time matters | Latent diffusion with a compact latent | One 256-d token. DDIM-50 takes 10.24 s against 225.28 s for MDM, for 2048 clips [arxiv:2212.04048 tab. 10] |
| Need a per-step speed-quality dial, and have a vocoder or decoder to follow | Score-based decoder with N Euler steps | MOS holds at N = 10 and drops at N = 4 [arxiv:2105.06337 tab. 2] |
| Need several plausible futures from one history, and can pay about 100 denoiser calls per sample | Reverse diffusion over future positions | The paper reports multi-sample metrics. Its sampling cost is about 39 times Trajectron++ [arxiv:2203.13777 §5] |
| Need high-fidelity text-to-image at scale | Expert ensemble, as a design reference only | Best FID reported, but no code or weights and partly proprietary data [arxiv:2211.01324 §5] |
| Need gradient control over whole-sentence structure, and accept a worse likelihood and slow decoding | Continuous text diffusion | Structured-control success is well above FUDGE [arxiv:2205.14217 tab. 2]. Likelihood is worse [arxiv:2205.14217 §7] |
| Need labelled synthetic images for a detector, with no diffusion training | Sample a pretrained generator and document its settings | Paired real and generated data. Accuracy is reported for one generator only [arxiv:2303.14126 tab. 7] |
| Need one-step sampling and can afford one teacher run per training pair | Distil a deterministic teacher by regression | One step gives FID 9.36 on CIFAR-10 against 4.16 for the teacher [arxiv:2101.02388 tab. 1] |
| Need streaming video with frames produced as they are needed | Causal chunked student with few-step sampling and a KV cache | 1.3 s latency for a 120-frame clip, against 219.2 s for the teacher [arxiv:2412.07772 tab. 3] |
| Need one model for both images and video | Shared latent with window attention | Kinetics-600 frame prediction FVD 3.3 against 4.3 for MAGVIT-v2 [arxiv:2312.06662 tab. 1] |
| Need to fill in a fluid field from a coarse or sparse measurement, with only fine simulations for training | Diffusion prior sampled from a noised coarse guide, optionally with a PDE-residual term | Equation loss 0.2178 against 0.5438 for a direct-mapping UNet at 4x [arxiv:2211.14680 tab. 1] |
| Need video and sound from one model, and can pay for a 1,000-step sampler | Two coupled denoisers with random-shift cross-modal attention | FVD 117.20 on Landscape with full DDPM sampling, against 229.08 with DPM-Solver [arxiv:2212.09478 tab. 1] |
| Need to finetune a text-to-image model to video, with few video clips | Correlated per-frame noise instead of independent noise | UCF-101 FVD 337.40 with mixed noise, against 566.67 with independent noise [arxiv:2305.10474 tab. 5] |
| Need planning or long rollouts where each part of a sequence has its own uncertainty | Per-token noise levels with causal recurrence | Maze2D average reward 141.7, against 119.5 for Diffuser [arxiv:2407.01392 §4.2] |
| Need text generation that fills positions in any order, and accept upper-bound likelihoods | Masked diffusion with carry-over | LM1B bound 23.00 against 20.86 for autoregressive at 327B tokens [arxiv:2406.07524 tab. 1] |
| Need one-shot generation of labelled graphs or molecules | Discrete denoising with a graph transformer | Valid-unique-novel 74% on SBM and 75% on planar graphs [arxiv:2209.14734 tab. 1] |
| Need a panorama or a region layout from a pretrained model, with no training | Average overlapping denoising views | Panorama FID 10.3, against 18.4 for Blended Latent Diffusion [arxiv:2302.08113 tab. 1] |
| Need image or reference-image steering of an unconditional model, with no caption data | CLIP-based sampling guidance with a finetuned encoder | FFHQ image-guidance FID 14.37, against 17.15 for ILVR [arxiv:2112.05744 tab. 1] |
| Need an inverse problem solved from a prior trained on clean data, and minutes per image are acceptable | Posterior sampling with annealed Langevin | 16 minutes and 0.95 GB for one high-resolution brain [arxiv:2108.01368 §4.1] |
| Need a constrained point estimate from a fixed generator | Plug-and-play point estimate with annealed noise | 1.28% tour gap at N = 50 without beam search [arxiv:2206.09012 tab. 2] |
| Need a 3D object from one image in about a minute | Multi-view tile, then sparse-volume diffusion | F-Score 93.6 on GSO at about 60 s [arxiv:2311.07885 tab. 1] |
| Need 3D shapes from mixed inputs, with a weight per input | Latent VQ diffusion with classifier-free weights | ShapeNet UHD 0.0557, against 0.0567 for AutoSDF [arxiv:2212.04493 tab. 1] |
| Need a robot plan that a person can inspect before it runs | Text-conditioned video planner with inverse dynamics, run open-loop | 59.1% task completion on seen-place instructions, against 19.4% [arxiv:2302.00111 tab. 1] |
| Need a segmentation mask from a low-contrast medical image | Mask diffusion with image-gated features | Dice 87.5 on optic cups for MedSegDiff++ [arxiv:2211.00611 tab. 1] |
| Need long or two-person motion from a single-person model | Compose intervals, and add a small coordination block | BABEL transition FID 1.88, against 3.86 for TEACH [arxiv:2303.01418 tab. 1] |
| Need a classifier's training data from a text-to-image model | Fine-tune the generator on the target set, and match per-class counts | ResNet-50 rises from 76.39% to 78.17% with 1.2M generated images [arxiv:2304.08466 tab. 3] |
| Need many controls at once, with one model | Train on random subsets of representations | Composer reports COCO FID 9.2 but names no baseline [arxiv:2302.09778 §3.5] |
| Need a robot policy to start from a pretrained video model, and can pay about 1 s per action chunk | Joint video and action denoising on the video backbone | SIMPLER novel-object average 65.2 against 42.4 for CogACT [arxiv:2512.06963 tab. 2] |
| Need a VLA with a diffusion action head, and a cheap check at moments when a grasp can fail | Diffusion head with a VLM supervisor that runs on gripper changes | SIMPLER WidowX average task success 64.6 against 51.8 for CogACT [arxiv:2509.04018 tab. 3] |
| Need to compare candidate actions before moving, and can pay for many policy and world-model calls | Diffusion proposals under MCTS with a world model | WidowX mean 51.0 against 47.9 for CogACT, with no latency reported [arxiv:2512.18477 tab. 1] |
| Need a layout generated, completed or refined by one model, with conditions applied at inference | Discrete diffusion over quantized layout tokens | Rico refinement FID 2.77 against 3.23 for RUITE, but PubLayNet 6.75 against 6.39 [arxiv:2303.08137 tab. 3] |
| Need a region layout on a frozen Stable Diffusion model, and masks are available | Attention modulation in the first 30% of steps | SOA-I 77.61 against 73.08 for Stable Diffusion [arxiv:2308.12964 tab. 2] |
| Need a set of images in one style, with no per-style training | Shared self-attention with one reference image | About 29 s for four images on one A100 [arxiv:2312.02133 §4]. Sharing across all images leaks content [§3.2] |
| Need a CLIP classifier to adapt on single unlabelled test images | Diffusion views filtered by similarity, then entropy | 59.85 against 56.98 for TPT with RN50, ten-dataset average [arxiv:2308.06038 tab. 2] |
| Need to check whether an unlearning fix holds against crafted prompts | Prompt-level white-box attack on the unlearned model | Nudity attack success 20.42% to 76.05% against ESD [arxiv:2310.11868 tab. 2] |
| Need a fashion or portrait video from one photo, and a per-subject set-up is acceptable | Fine-tuned image model, frames sampled independently | About 30 minutes of tuning and 18 s per frame [arxiv:2304.06025 §6] |
| Need a talking head from one photo and audio, with no skeleton | Audio-conditioned frame diffusion with a reference network | HDTF FID 8.76, the best in the authors' table [arxiv:2402.17485 tab. 1] |
| Need talking faces in real time from one image | Diffusion transformer over a learned face latent | Up to 40 FPS at 512×512 on one RTX 4090 [arxiv:2404.10667 §4] |
| Need sound and picture generated together and aligned in space and time | Joint audio-video diffusion transformer with prior tokens | JavisScore 0.154 against 0.151 for FoleyCrafter [arxiv:2503.23377 tab. 1] |
| Need a video model to move toward a judged property, and an independent check of the result | Offline RWR or DPO on accept or reject labels | Gemini All 52.66% to 56.41% with RWR; GPT-4o gain about 2.4 points [arxiv:2412.02617 tab. 2] |
| Need elastic, force-responsive 3D objects without measured material data | Video-derived stiffness field with a physics simulation | 80.8% prefer it over PhysGaussian on motion realism [arxiv:2404.13026 tab. 1] |
| Need a robot policy that reuses a video model's learned dynamics | Video features at a fixed timestep feeding an action diffusion transformer | 98.6% average success on LIBERO, author-reported [arxiv:2603.10448 tab. 1] |
| Need an interactive video that responds to keys, mouse and text in near real time | Causal block-wise distilled generator with a sink-token KV cache | 16 FPS, hardware not stated [arxiv:2511.23429 §4.4] |
| Need a likelihood-style score in bits per dimension, and can afford long training runs | Learned-schedule diffusion trained on the variational bound | 2.65 bits per dimension on CIFAR-10, a bound, and 7.54 at 10 evaluation steps [arxiv:2107.00630 tab. 1] [tab. 2] |
| Need text-to-image with diverse outputs, and a second model that maps text to an image embedding is acceptable | Diffusion prior in CLIP embedding space, then a decoder | Zero-shot MS-COCO FID 10.39, against 10.63 for the autoregressive prior [arxiv:2204.06125 tab. 2] |
| Need a class-conditional generator whose quality scales with compute, and the team can drop the U-Net | Transformer over VAE latent patches with adaLN-Zero | FID 2.27 at 256×256 with guidance 1.50, author-reported [arxiv:2212.09748 tab. 2] |
| Need spatial control of a frozen Stable Diffusion model with a small training run, and several controls to combine | Frozen-model adapters added to encoder features | COCO FID 16.78 with segmentation, against 24.68 for text only [arxiv:2302.08453 tab. 1] |
| Need the highest human-rated quality in the Stable Diffusion family, and can run two large models | Larger UNet with size and crop conditioning, plus an optional refiner | Refiner-enhanced model wins 48.44% of votes, against 7.91% for SD 1.5 [arxiv:2307.01952 §2.5] |
| Need a 3D scene from about 12 photos, and a diffusion model can be called during training | Score distillation on unseen views, plus depth-correlation losses | PSNR 19.37 on Mip-NeRF360 with 12 views, against 17.49 for 3DGS [arxiv:2312.00206 tab. 1] |
| Need held-out street-view frames from a driving log, and can build about 36k training pairs for a cleanup model | Gaussian splatting on densified points, then a one-step diffusion pass on each render | Devset score 0.423 pretrained, 0.462 fine-tuned [arxiv:2510.12308 tab. 1]; leaderboard 0.432, second of 28 [§1] |
| Need one-step sampling without pairs of noise and teacher images, and can afford a second score network | Distribution matching with a fake-score model, plus LPIPS regression | ImageNet 64 FID 2.62 in one pass, against 2.32 for EDM at 512 steps [arxiv:2311.18828 tab. 1] |
| Need a human motion model that can enforce constraints on the predicted clean motion | Transformer encoder that predicts clean motion, with classifier-free guidance | HumanML3D FID 0.544, against 1.067 for T2M [arxiv:2209.14916 tab. 1] |
| Need an editable material for a 3D shape from a text prompt | Painted views fitted to a procedural material graph | Chair FID 88.3, against 94.6 for the best baseline [arxiv:2404.17569 tab. 1] |
| Need segmentation from a few annotated images, and a diffusion model trained on the target domain | Frozen DDPM decoder features with a pixel-MLP ensemble | Mean IoU 49.4 on Bedroom-28 with 40 images, against 47.9 for DatasetDDPM [arxiv:2112.03126 tab. 2] |
| Need a 3D asset from text with no 3D training data, and about 25 minutes per object is acceptable | Score distillation through a volume renderer | Qualitative results only; no metric is reported [arxiv:2212.00774 §5.2] |
| Need a mesh from one photo in under a minute, without per-shape optimisation | Zero123 views, then a feed-forward SDF cost volume | F-score 83.5 averaged over GSO and Objaverse, about 45 s on an A100 [arxiv:2306.16928 tab. 1] |
| Need point clouds or completions with several plausible samples from one partial input | Point-voxel denoiser, with noise on the free points only | 1-NN best on all six generation cells; 1000 sampling steps [arxiv:2104.03670 tab. 1] [app. H.2] |
| Need 3D molecular conformations from a graph, and the sampling cost is acceptable | Equivariant noise network with a zero-centre-of-mass prior | COV-R mean 89.13% on GEOM-Drugs, against 67.16% for GeoMol [arxiv:2203.02923 tab. 1] |
| Need to upscale real photos with a frozen text-to-image model | Trainable time-aware encoder with modulation, plus an optional fidelity weight | FID 24.44 on DIV2K-Valid, but PSNR 23.26 [arxiv:2305.07015 tab. 1]; 15.16 s per output with 200 steps [tab. 5] |
| Need camera control in generated video that handles lens distortion, pitch and roll | Per-token ray encoding through a zero-initialised parallel adapter | Rotation error 4.12° against 9.23° for ReCamMaster, on the authors' synthetic set [arxiv:2512.07237 tab. 1] |
| Need a manipulation policy that uses touch, learned from demonstrations | Masked tactile pretraining, then a diffusion policy with 16 inference steps | Test-tube whole-task success 0.85 with pretraining, 0.70 without, 0.25 vision-only [arxiv:2507.15062 tab. 1] |
| Need a generalist robot policy that a small team can finetune on about 100 demonstrations, with a new camera or action space | Pretrained tokenized transformer with a small diffusion action head, all weights finetuned | 72% against 20% from scratch, over six setups, author-reported [arxiv:2405.12213 tab. 1]. Action-head ablation: 83% diffusion, 35% MSE, 18% discretized [arxiv:2405.12213 tab. 2] |
| Need one model to handle a long, multi-step task by itself, with its own text for each sub-step, and can train a 2B VLM and a 1B expert in three stages | VLM with a diffusion action expert, trained in three stages, and sub-step text as an intermediate output | Laundry folding 0.4 against 0.2 for pi0 with direct prompting, 10 trials per task [arxiv:2502.05855 §4.3]. Sub-steps beat SayCan on bussing 0.70 against 0.58 [arxiv:2502.05855 app. D.2] |
| Need a step count for a sampler in theory | Use the k / eps scaling, with k the intrinsic dimension | Bound holds for original DDIM and DDPM coefficients [arxiv:2501.12982 §3.1] |
| Need to know which target (noise, clean sample or score) to regress, and why | Read the unified derivation, then pick by the cards' results | The three targets give one objective up to weights, and the weights are not given in practice [arxiv:2208.11970 §3] |
| Need to check whether a generator reproduces its training images, with the training set available to search | Retrieval detector on copy-detection features, flagging top-1 matches above a threshold | About 1.88% of Stable Diffusion v1.4 samples pass 0.5 on a 12M-image subset [arxiv:2212.03860 §7] |
| Need a text-to-3D asset with a textured mesh, and about 40 minutes on 8 A100 GPUs is acceptable | Coarse neural field with a 64 px prior, then a mesh refined with a 512 px latent prior | 61.7% of raters prefer it to DreamFusion, a human study on DreamFusion's videos [arxiv:2211.10440 tab. 1] |
| Need a text-to-image model tuned toward human preference, with a scorer trained on pairwise rankings and a late-step reward | Preference reward on the BLIP backbone, back-propagated through one late denoising step | ReFL wins 58.79% of human comparisons against SD v1.4 on real-user prompts [arxiv:2304.05977 tab. 4] |
| Need a one- to four-step text-to-image sampler from a large teacher, and a discriminator and training code are acceptable to build | Student from teacher weights, with a hinge discriminator and score distillation | ADD-XL one step beats LCM-XL four steps in human preference [arxiv:2311.17042 §4.2]. Training recipe not stated. Released SDXL-Turbo weights are non-commercial |
| Need a robot policy that uses a few task demonstrations plus a large, diverse demonstration set | Diffusion policy with a 50/50 batch mix from the large set | +22 points in-distribution and +17 out-of-distribution over the next-best method, averaged over tasks [arxiv:2403.12945 §5.2] |
| Need inpainting, super-resolution or deblurring from a diffusion-style restorer with few sampling steps, and a two-stage training run is acceptable | Diffuse a 256-value restoration prior in four steps, then restore with a transformer | Places wide-mask inpainting FID 1.98, against 2.25 for LaMa at about the same parameter count [arxiv:2303.09472 tab. 1] |
| Need point matches between two images with no labels, and a pretrained diffusion model is available | Nearest-neighbour matching on one denoiser pass's features, with the noise level and block tuned per task | SPair-71k per-image PCK 52.9, against 33.9 for DINO, but 55.3 for the supervised SCorrSAN [arxiv:2306.03881 tab. 1] |
| Need faster sampling of a pretrained U-Net diffusion model without retraining, and a small quality loss is acceptable | Full U-Net pass every N steps, with cached high-level features reused in between | 2.15x throughput on Stable Diffusion v1.5 at N = 5, with a 0.05 CLIP drop [arxiv:2312.00858 tab. 3]. Quality falls as N grows [tab. 2] |
| Need a grasp generator that is fast at sampling and cheap to retrain per task | Latent DDPM over a conditional VAE, sampled with DDIM at 100 steps | Simulated success 0.756 with DDIM against 0.792 with 1000-step DDPM, and 0.75 s against 7.39 s per 100 grasps [arxiv:2312.11243 tab. 1] |
| Need a hand grasp that avoids object penetration on an unseen dataset, and can accept a two-stage training | Latent DDIM with a frozen-model adaptation MLP | Out-of-domain HO-3D penetration 5.23 against 7.37 for GraspTTA, at 0.14 s [arxiv:2411.14786 tab. 2] |
| Need 3D object or scene generation for LiDAR, with rare foreground objects | Object-level diffusion first, then a range-image scene diffusion with a controller | KITTI-360 FPD 7.60 against 25.1 for UltraLiDAR [arxiv:2412.17226 tab. 1] |
| Need a visuomotor policy that scales with more demonstrations, and can afford a large transformer | Diffusion over action chunks with adaptive layer norm and no causal mask | Franka average 87.50 for the largest model against 51.25 for DP-T [arxiv:2409.14411 tab. 2] |
| Need to improve a behaviour-cloned diffusion policy with reward, and have a simulator with dense task reward | PPO on each denoising step, with BC regularisation if hardware is in the loop | Simulation success 88% without BC regularisation and 53% with it, against 0% and 50% on hardware [arxiv:2409.00588 §5.4] |
| Need a dexterous manipulation policy driven by a language planner, with few demonstrations per task | Transformer action-diffusion policy after a VLM planner | Simulated average success 0.70 against 0.23 for DP-T [arxiv:2409.05493 tab. 1] |
| Need a visuomotor policy where contact is hard to see, and you can record audio on the gripper | Diffusion head on fused vision and audio | Wiping 85% with audio against 40% with vision only [arxiv:2406.19464 §4.2] |
| Need a whole-body humanoid policy from RL data, with domain randomization | Action-sequence diffusion trained on randomized RL rollouts | 8M transitions from all setups: success 1.0 on both randomized and non-randomized targets [arxiv:2411.01349 tab. 3] |
| Need a dexterous hand policy that uses contact targets before the action | Guider diffusion over contact, then an Actor with a Q term | Move-T stage-2 success 0.60 against 0.30 for Diffusion Policy [arxiv:2411.12982 tab. 5] |
| Need a navigation policy from visual demonstrations, with relative waypoints and one robot | 1D U-Net over relative waypoints, 5 denoising steps at deployment | Hard-level success 84% against 20% for ViNT with relative-distance goal [arxiv:2412.20226 tab. I] |
| Need a physics-based character that tracks motion under perturbation, and inference cost is acceptable | Diffusion policy trained by behaviour cloning on noisy states | Out-of-distribution perturbation success 96.3 against 81.0 for an MLP [arxiv:2406.00960 tab. 3] |
| Need a segmentation model trained from scratch on the target data, with no pretrained backbone, and several reverse chains per mask are affordable | Image-conditioned mask diffusion, with chains averaged and thresholded | Cityscapes per-object mIoU 76.44, against 73.70 for the best listed baseline [arxiv:2112.00390 tab. 1]. Inference is slower than Segformer and STDC [arxiv:2112.00390 §5] |
| Need better sample quality from the standard DDPM recipe, with the same network and no extra model | Perception-weighted simple loss over noise levels | FFHQ FID 7.86 to 6.92 at 1000 steps [arxiv:2204.00227 tab. 1]. The authors report no extra models or heavy computation [arxiv:2204.00227 §5.1] |
| Need several named people in one image, with one photo each, and no training run per person | Subject embeddings in text tokens, with delayed subject conditioning | Multi-subject identity 43.11% against 13.52% for Textual Inversion, which uses five images per subject [arxiv:2305.10431 tab. 2] |
| Need one small navigation policy that explores unseen places and also reaches a goal photo, on a real robot | Goal-masked diffusion policy with a 10-step denoiser | Exploration success 98% with 19M parameters, against 77% for Subgoal Diffusion with 335M [arxiv:2310.07896 tab. 1] |
| Need 4x upscaling of real photos where recognisable objects matter, and about 2.3 billion parameters and roughly 7 s per 512 px output on one V100 are acceptable | Tag-prompted ControlNet with a low-resolution start | Best FID, CLIPIQA and MUSIQ where the table reports them, but lower PSNR and SSIM than the best baseline on the paired sets [arxiv:2311.16518 tab. 2]. Model size and timing come from the supplement [arxiv:2311.16518 suppl.] |
| Need an object placed into a masked region from a short label, and control over how closely it follows the mask | Mask-precision-conditioned inpainting, fine-tuned from Stable Diffusion | OpenImages bounding-box local FID 9.71, against 12.57 for Stable Inpainting [arxiv:2212.05034 tab. 1] |
| Need box-level layout control, and training from scratch on box-labelled images is acceptable | Layout tokens with object-aware cross-attention, trained from scratch | COCO-Stuff 256 FID 15.61, against 29.10 for PLGAN, a GAN [arxiv:2303.17189 tab. 1] |
| Need layout control on a frozen Stable Diffusion model with no training, and about twice the sampling time is acceptable | Backward cross-attention guidance on the first 10 steps | VISOR conditional accuracy 95.95%, against 62.98% for Stable Diffusion [arxiv:2304.03373 tab. 2] |
| Need a textured mesh from text with no 3D data, and about 50 minutes on one 16 GB GPU is acceptable | Subdivision mesh with a CLIP loss on random views | R-precision 96.7 with ViT-B/16, against 93.5 for Dreamfields [arxiv:2203.13333 tab. 1]. The prior term is not in these numbers [arxiv:2203.13333 §4.3] |
| Need very dark or very bright images from a Stable Diffusion-style model, and can fine-tune with v prediction | Zero terminal SNR schedule, v prediction, trailing steps and rescaled guidance | FID 21.66, against 23.76 for the official SD 2.1-base; the brightness gain is qualitative [arxiv:2305.08891 tab. 3] [§4.1] |

Step count is model-specific. In MLD's table, DDIM-50 gives FID 0.473 and
DDPM-1000 gives 0.568 [arxiv:2212.04048 tab. 10]. For MDM, the same two samplers
give 7.334 and 0.544 in that table. Measure the step count on your own model
instead of copying a sampler setting. The theory in [arxiv:2501.12982 §3.1] gives
a scaling law in the target error, not a number of steps to use.

Guidance also trades metrics against each other. MLD's sweep has its
best FID at scale 2.5 (0.350), while R-precision rises from 0.648 at 1.5 to
0.772 at 7.5 [arxiv:2212.04048 tab. 11]. Choose the scale against the metric you
need. The guidance mechanism is on [Classifier-free guidance](/concepts/classifier-free-guidance).
Scales differ by model: the Imagen base uses 1.25 [arxiv:2304.08466 §4.2],
Composer uses 3.0 for its prior and base models [arxiv:2302.09778 §3.5].
Neither transfers to another model without a sweep.

Budget note: the GLIGEN larger runs use 500k iterations at batch 32 for Stable
Diffusion, and 400k iterations for the LDM run [arxiv:2301.07093 app. A]. Training
compute varies widely across the cards: Grad-TTS trains on one RTX 2080 Ti
[arxiv:2105.06337 §4], the MLD VAE and denoiser take about 8 h and 4 h on eight
V100s [arxiv:2212.04048 §4.2], the eDiff-I base uses 256 A100s
[arxiv:2211.01324 §5], and the Imagen fine-tune uses 256 TPU-v4 chips for 210k and
490k steps [arxiv:2304.08466 §4.1]. Inference cost of the GLIGEN grounding tokens is not stated
[arxiv:2301.07093 app. A].

## Implementation sketch

Common core: noise a clean sample at a random level and regress the noise.
GLIGEN, MLD and MID use this objective, with their own noise schedules
[arxiv:2301.07093 §4.2] [arxiv:2212.04048 §3.2] [arxiv:2203.13777 §3.2]. The
comments cite the card section that each step comes from, with the arXiv id
short form.

```pseudocode
# One training step, noise-prediction objective  [2301.07093 §4.2] [2203.13777 eq. 10]
# Input:  clean samples x0 (B, ...), conditions c (may be empty)
# Output: scalar loss

def training_step(x0, c, model):
    # Draw one noise level per sample; the schedule is variant-specific
    t = sample_timestep(batch=B)
    eps = sample_normal_like(x0)
    # Corrupt x0 to level t with the variant's forward process  [2212.04048 §3.2]
    x_t = add_noise(x0, eps, t)
    # Drop the condition for some samples, so the model also learns the
    # unconditional branch used by guidance  [2212.04048 §3.3]
    c = drop_with_probability(c, p=0.1)
    # Predict the noise from the noisy sample, the level and the condition
    eps_hat = model(x_t, t, c)
    return mean((eps - eps_hat) ** 2)
```

```pseudocode
# Sampling: start from noise and apply the learned step  [2212.04048 §4.2]
# Input:  shape of the working space, conditions c, number of steps
# Output: one sample in the working space

def sample(shape, c, steps, model):
    # Start from Gaussian noise in the working space (latents or data)
    x = sample_normal(shape)
    for t in timesteps(steps):
        # Noise estimate under the condition  [2212.04048 §3.3]
        eps_hat = model(x, t, c)
        # Variant-specific update: DDIM, DDPM posterior or an ODE step
        x = update(x, eps_hat, t)
    return x
```

Where variants differ: the prediction target (noise in GLIGEN, MLD and MID;
score in Grad-TTS; clean sample in Diffusion-LM, in eDiff-I's
preconditioned denoiser and in MDLM's per-token rule [arxiv:2406.07524 §3.2.3];
velocity in W.A.L.T [arxiv:2312.06662 §3]), the sampler (DDIM with eta 0 in MLD, Euler steps in
Grad-TTS, DEIS in eDiff-I),
and how conditions enter (see Variants).

For the frozen-backbone variant, the added layer attends over visual and
grounding tokens, and the gate starts at zero. Its scheduled weight switches
the grounding off after the first fraction of steps
[arxiv:2301.07093 §4.2] [eq. 8] [eq. 10].

```pseudocode
# Grounding token for one entity  [2301.07093 §4.1] [app. A]
# Input:  text_feature (768,)  CLIP text feature of the phrase
#         box (4,)             (x_min, y_min, x_max, y_max)
# Output: grounding_token (768,)

def grounding_token(text_feature, box):
    # Fourier-embed each of the four coordinates  [app. A]
    box_feature = fourier_embedding(box)          # (64,)
    # Concatenate the two inputs and run the MLP  [eq. 5] [app. A]
    joined = concat(text_feature, box_feature)    # (832,)
    return mlp(joined)                            # (768,)
```

```pseudocode
# Gated self-attention, inserted between the frozen self-attention and
# cross-attention of each transformer block  [2301.07093 §4.2] [eq. 6-8]
# Input:  visual (B, M, d), grounding (B, N, 768), gamma (learned, init 0)
# Output: visual (B, M, d)

def gated_self_attention(visual, grounding, gamma, beta):
    num_visual = visual.shape[1]                    # M
    # Project grounding tokens to the visual width d  [app. A]
    grounding = linear_projection(grounding)        # (B, N, d)
    # Joint self-attention over visual and grounding tokens  [eq. 8]
    joint = concat(visual, grounding, axis=1)       # (B, M + N, d)
    attended = self_attention(joint)[:, :num_visual]
    # Gated residual; tanh(0) = 0 at initialization  [eq. 8]
    return visual + beta * tanh(gamma) * attended
```

```pseudocode
# Scheduled sampling at inference  [2301.07093 §4.2] [eq. 10] [§5.4]
# tau in [0, 1]: fraction of denoising steps that use grounding

def grounding_weight(step, total_steps, tau):
    # beta = 1 for the first tau * T steps, then 0  [eq. 10]
    if step <= tau * total_steps:
        return 1.0
    return 0.0
```

## Contradictions

- **Prediction target.** Diffusion-LM predicts the clean embedding and reports
  that predicting the noise collapsed as the embedding dimension grew
  [arxiv:2205.14217 §7.4] [fig. 4]. The reported evidence is one figure, with no
  table. GLIGEN, MLD and MID train the noise prediction and report results
  [arxiv:2301.07093 eq. 1] [arxiv:2212.04048 eq. 2] [arxiv:2203.13777 eq. 10].
  Grad-TTS predicts a score under its own weighting [arxiv:2105.06337 eq. 17].
  MDLM predicts clean tokens in a masked setting [arxiv:2406.07524 §3.2.3], which
  is a different space from the continuous cases above. The papers work in
  different spaces, so none tests the other's claim. Both sides are stated here,
  not resolved.
- **Clean target versus noise target.** MDM predicts the clean motion
  [arxiv:2209.14916 §3], and the unCLIP diffusion prior predicts the clean
  embedding [arxiv:2204.06125 §2.2]. DiT predicts noise with a covariance
  [arxiv:2212.09748 §3.1], and VDM predicts noise under its learned schedule
  [arxiv:2107.00630 §3.4]. Point-voxel diffusion [arxiv:2104.03670 eq. 6] and
  GeoDiff [arxiv:2203.02923 eq. 9] also predict noise. The cards work on
  different data and report no head-to-head comparison, so neither choice is
  shown to be better.
- **Equivalent targets versus practice.** The tutorial states that predicting
  the clean sample, the noise or the score gives the same objective up to
  time-dependent weights, and that predicting the noise has been found to work
  better in practice, citing other work [arxiv:2208.11970 §3]. It does not
  test this. The cards cited here train noise prediction [arxiv:2212.04048 eq.
  2] [arxiv:2203.13777 eq. 10] and report results for their own tasks. This
  page found no head-to-head comparison of the two targets, so the practical
  preference is stated here, not measured.
- **Diffusion policy against simple baselines.** On the piano benchmark, the
  transformer diffusion baseline is below BC-MLP with 12 songs in distribution,
  0.357 against 0.529 [arxiv:2408.11048 tab. 2]. On Franka tasks, the
  transformer diffusion baseline is below the larger ScaleDP model, 51.25
  against 87.50 [arxiv:2409.14411 tab. 2]. The two papers use different tasks,
  data and baselines, and neither tests the other's setup, so the ranking is
  not resolved here.
- **Human preference versus FID.** SDXL has the worst FID of the three models
  it compares, but its human study favours it [arxiv:2307.01952 app. On FID
  Assessment]. The DiT and unCLIP cards report FID as their main metric
  [arxiv:2212.09748 tab. 2] [arxiv:2204.06125 tab. 2]. ImageReward makes the
  same argument: on its six-model ranking of 100 prompts, FID has a Spearman
  correlation of 0.09 with the human order, and CLIP 0.60 [arxiv:2304.05977
  tab. 1]. That table uses the authors as raters and six models, so it is
  small. Both views are stated here, not resolved.
- **Teacher gap for distillation.** The one-step DMD student trails SD v1.5 by
  2.71 FID at guidance 3, with the teacher's number taken from the paper
  [arxiv:2311.18828 tab. 3]. At guidance 8, with the teacher rerun by the
  authors, the gap is 1.48 [arxiv:2311.18828 tab. 4]. The two rows use
  different guidance scales and different teacher numbers.
- **Cross-attention versus concatenation.** MLD reports that concatenating its
  condition beats cross-attention, with FID 0.473 against 1.922
  [arxiv:2212.04048 tab. 5]. eDiff-I conditions its text through cross-attention
  [arxiv:2211.01324 §4.2], and the card here has no comparison against
  concatenation. GLIGEN finds gated cross-attention worse than gated
  self-attention for its boxes, with AP 16.6 against 21.7 [arxiv:2301.07093 app.
  B]. The results are for different conditions and models.
- **Step count.** MLD's DDIM-50 beats its DDPM-1000 on FID, while MDM's DDIM-50
  is far worse than its DDPM-1000 [arxiv:2212.04048 tab. 10]. Grad-TTS finds that
  10 steps are near saturation [arxiv:2105.06337 tab. 2]. Diffusion-LM uses 200
  steps for control and 2000 for ROCStories, because 200 hurt quality there
  [arxiv:2205.14217 §6.1]. These are model-specific results, not a conflict over
  one setting.
- **Autoregressive gap for discrete diffusion.** MDLM argues that much of the
  gap to autoregressive models came from training recipes, not from the
  framework [arxiv:2406.07524 §1] [§6]. Its own table gives a bound of 23.00
  against 20.86 for a retrained autoregressive model [arxiv:2406.07524 tab. 1].
  Diffusion-LM reports its E2E bound at 2.28, against 1.77 for an autoregressive
  Transformer [arxiv:2205.14217 §7]. The two comparisons use different datasets
  and bound types, so the gaps are not comparable.
- **Time-series baselines.** TimeGrad reports the lowest CRPS_sum on five of six
  datasets. On Exchange its value is 0.006, against 0.005 for three baselines
  [arxiv:2101.12072 tab. 2]. The Diffusion Forcing card reports 0.003 for its own
  method on Exchange [arxiv:2407.01392 §4.5]. It also lists a diffusion baseline,
  ScoreGrad, with CRPS_sum no higher than TimeGrad's on every column it reports
  [arxiv:2407.01392 §4.5]. The TimeGrad card does not include ScoreGrad, so its
  "best on five of six" claim is relative to a smaller baseline set. The two papers
  use different baselines and seeds, so their rankings cannot be merged.

- **Headline gain of DiffTPT.** The abstract and conclusion state an average
  gain of 5.13% over TPT. Section 4.2 gives 5.1% for RN50 and 2.3% for
  ViT-B/16 on the cross-dataset scenario [arxiv:2308.06038 §4.2]. The printed
  averages in that scenario give 2.87 and 1.48 points [arxiv:2308.06038 tab.
  2]. The paper does not say how the headline is computed, so both are stated
  here.
- **Which algorithm wins in video RL.** On the Gemini "All" column RWR on AI
  feedback (56.41%) is above DPO (56.17%), but on human "All" DPO (34.06%) is
  above RWR (31.09%) [arxiv:2412.02617 tab. 2]. The authors also say DPO
  generalises better to held-out prompts [arxiv:2412.02617 §4.2] [§4.3], while
  RWR keeps improving over two rounds and DPO saturates [arxiv:2412.02617 §4.3]. The ranking depends on the
  evaluator and the number of rounds.
- **CogACT on SimplerEnv WidowX.** Three cards report the same CogACT policy on
  this benchmark, with different averages: 47.9 in STORM [arxiv:2512.18477 tab. 1],
  51.3 in VideoVLA [arxiv:2512.06963 tab. 1] and 51.8 in FPC-VLA
  [arxiv:2509.04018 tab. 3]. The per-task values differ as well. None of the cards
  says whether the baseline was re-run or copied, so the gain each paper reports
  over CogACT is not comparable across papers. Both sides are stated here, not
  resolved.
- **Talking-face sync scores.** EMO reports that its SyncNet score is below
  Wav2Lip and SadTalker [arxiv:2402.17485 tab. 1]. VASA-1 reports the best lip
  sync against SadTalker and two other baselines [arxiv:2404.10667 §4]. The two
  papers use different benchmarks, so their sync numbers cannot be merged.

- **FID for StableSR on DIV2K.** The StableSR entry above reports FID 24.44 on
  DIV2K-Valid, the best value in its table [arxiv:2305.07015 tab. 1]. SeeSR's
  table reports StableSR at 36.57 on its own DIV2K-Val set, where SeeSR scores
  31.93 [arxiv:2311.16518 tab. 2]. SeeSR builds that set with its own
  degradation pipeline [arxiv:2311.16518 §4.1]. The StableSR entry does not
  state its test degradation, so the two StableSR numbers cannot be checked
  against each other. Both are stated here, not resolved.

Within a card, the released code departs from the paper in places, for example
the Grad-TTS sampling temperature, the MID diffused variable and the
Denoising Student's teacher start index. Those are on the cards and are not
repeated here.

## Papers

- [Variational Diffusion Models](/papers/2107.00630): a diffusion model with a learned noise schedule, trained on the variational bound.
- [Understanding Diffusion Models: A Unified Perspective](/papers/2208.11970): a tutorial derivation that links the ELBO, the three training targets and guidance, with no experiments.
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](/papers/2212.03860): a retrieval detector that counts generated images copying training images, for small DDPMs and Stable Diffusion.
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](/papers/2204.06125): a CLIP-embedding prior with a diffusion image decoder, for text-to-image and image variations.
- [Scalable Diffusion Models with Transformers](/papers/2212.09748): a transformer backbone on latent patches, conditioned by adaLN-Zero, with FID scaling in Gflops.
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](/papers/2302.08453): small adapters that add control-map features to a frozen Stable Diffusion encoder.
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](/papers/2307.01952): a larger latent UNet with size and crop conditioning, and an optional refiner.
- [SparseGS: Sparse View Synthesis using 3D Gaussian Splatting](/papers/2312.00206): Gaussian splatting from few views, with score distillation on unseen camera poses.
- [Hybrid Gaussian Splatting for Novel Urban View Synthesis](/papers/2510.12308): Gaussian splatting on COLMAP-densified points, with a one-step diffusion enhancer that cleans its renders.
- [One-step Diffusion with Distribution Matching Distillation](/papers/2311.18828): a one-step generator trained to match a teacher's output distribution, with a regression term.
- [Human Motion Diffusion Model](/papers/2209.14916): a transformer diffusion model for human motion that predicts the clean motion.
- [MaPa: Text-driven Photorealistic Material Painting for 3D Shapes](/papers/2404.17569): painted part views fitted to editable procedural material graphs for 3D shapes.
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](/papers/2411.14499): a survey of world models, including video generators as simulators. It reports no experiments.

- [GLIGEN: Open-Set Grounded Text-to-Image Generation](/papers/2301.07093): adds box grounding to a frozen latent text-to-image diffusion model with gated attention layers.
- [Executing your Commands via Motion Diffusion in Latent Space](/papers/2212.04048): latent diffusion for text- and action-conditioned human motion, with a single-token transformer VAE latent.
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](/papers/2211.01324): a text-to-image cascade whose base is an ensemble of noise-level experts, with paint-with-words control.
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](/papers/2105.06337): a score-based diffusion decoder for mel-spectrograms, where the number of Euler steps trades speed for quality.
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](/papers/2203.13777): reverse diffusion over future pedestrian positions, sampled to give several plausible futures.
- [Diffusion-LM Improves Controllable Text Generation](/papers/2205.14217): diffusion over word embeddings for text, with classifier-guided control on the latents.
- [Diffusion models as plug-and-play priors](/papers/2206.09012): an independently trained diffusion model as a prior, with a differentiable constraint for search over solutions.
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](/papers/2211.00611): mask diffusion conditioned on the scan, with image-gated features and a learned Fourier filter.
- [Robust Compressed Sensing MRI with Deep Generative Priors](/papers/2108.01368): posterior sampling for MRI with a score-based prior trained on brain slices.
- [Learning Universal Policies via Text-Guided Video Generation](/papers/2302.00111): a text-conditioned video diffusion planner with an inverse-dynamics policy, executed open-loop.
- [Photorealistic Video Generation with Diffusion Models](/papers/2312.06662): one transformer diffusion model for images and video, in a shared causal-autoencoder latent.
- [One-2-3-45++](/papers/2311.07885): a multi-view tile from a fine-tuned image diffusion model, lifted to 3D by two volume diffusion stages.
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](/papers/2212.04493): latent diffusion over 3D shapes with per-input classifier-free weights.
- [Composer: Creative and Controllable Image Synthesis with Composable Conditions](/papers/2302.09778): a cascade trained to rebuild images from random subsets of eight representations.
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](/papers/2302.08113): overlapping denoising windows averaged at each step, for panoramas and region layouts.
- [Semantic diffusion guidance (More Control for Free!)](/papers/2112.05744): CLIP-based guidance of an unconditional DDPM at sampling time, with a finetuned CLIP encoder.
- [Human Motion Diffusion as a Generative Prior](/papers/2303.01418): composes a pretrained motion model into long, two-person and controlled motion without retraining it.
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](/papers/2407.01392): per-token noise levels with a causal recurrent denoiser, for rollouts, guidance and planning.
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](/papers/2412.07772): a causal few-step video student distilled from a bidirectional teacher, with a KV cache.
- [Simple and Effective Masked Diffusion Language Models](/papers/2406.07524): masked diffusion over text tokens with a simplified continuous-time objective.
- [DiGress: Discrete Denoising diffusion for graph generation](/papers/2209.14734): discrete denoising over labelled graphs with a graph transformer.
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](/papers/2101.02388): regresses a one-step student onto a deterministic multi-step DDIM teacher.
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](/papers/2101.12072): an LSTM-conditioned diffusion sampler for multivariate forecasts, one time step at a time.
- [Low-dimensional adaptation of diffusion models: Convergence in total variation](/papers/2501.12982): sampling-step bounds that scale with intrinsic dimension, for original DDIM and DDPM coefficients (theory only).
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](/papers/2304.08466): fine-tunes an Imagen cascade on ImageNet and uses its samples as extra classifier training data.
- [CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images](/papers/2303.14126): uses Stable Diffusion v1.4 outputs as a labelled fake class for a real-versus-generated classifier.
- [LayoutDM: Discrete Diffusion Model for Controllable Layout Generation](/papers/2303.08137): discrete diffusion over quantized layout tokens, with conditions applied at inference.
- [Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning](/papers/2308.06038): Stable Diffusion image variations as extra views for test-time prompt tuning of CLIP.
- [Dense Text-to-Image Generation with Attention Modulation](/papers/2308.12964): attention modulation of a frozen Stable Diffusion model for dense captions and region layouts.
- [State of the Art on Diffusion Models for Visual Computing](/papers/2310.07204): a 2023 survey of diffusion models for images, video, 3D and 4D, with no experiments of its own.
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images](/papers/2310.11868): a prompt-level attack that tests whether concept unlearning holds in text-to-image models.
- [StyleAligned: Style Aligned Image Generation via Shared Attention](/papers/2312.02133): shared self-attention with a reference image, for one style across a batch with no training.
- [DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion](/papers/2304.06025): a fine-tuned Stable Diffusion for pose-guided fashion video from one photo.
- [EMO: Emote Portrait Alive](/papers/2402.17485): audio-to-video diffusion for portrait videos, with no skeleton or 3D face model.
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](/papers/2404.10667): a diffusion transformer over a learned face latent, for talking faces that render in real time.
- [JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization](/papers/2503.23377): one diffusion transformer for video and audio, linked by cross-attention and shared prior tokens.
- [Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback](/papers/2412.02617): offline reward-weighted and DPO fine-tuning of a text-to-video diffusion model, with accept or reject labels from a VLM.
- [PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation](/papers/2404.13026): a video model's motion used to fit a material field for physics simulation of 3D Gaussian objects.
- [DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control](/papers/2603.10448): video diffusion transformer features feeding an action diffusion transformer for robot control.
- [Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model](/papers/2511.23429): a distilled causal generator for interactive game video, with a sink-token KV cache.
- [Label-Efficient Semantic Segmentation with Diffusion Models](/papers/2112.03126): per-pixel features from a frozen DDPM decoder, classified by an MLP ensemble trained on a few labels.
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](/papers/2212.00774): a 2D denoiser's score back-propagated through a voxel renderer for text-to-3D.
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](/papers/2306.16928): Zero123 views fused in one feed-forward pass into a textured mesh.
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](/papers/2203.13333): a subdivision mesh, texture and normal map optimised from a text prompt through CLIP loss, with an optional diffusion-prior term.
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](/papers/2212.05034): inpaints a named object into a mask whose precision the user sets, with a predicted-mask output that keeps the background.
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](/papers/2303.17189): a layout-conditioned diffusion model trained from scratch on box tokens, with object-aware cross-attention.
- [Training-Free Layout Control with Cross-Attention Guidance](/papers/2304.03373): gradient guidance of cross-attention maps during the first denoising steps, steering a frozen Stable Diffusion to a layout.
- [Common Diffusion Noise Schedules and Sample Steps are Flawed](/papers/2305.08891): zero terminal SNR, v prediction, trailing steps and rescaled guidance, for a wider brightness range.
- [3D Shape Generation and Completion through Point-Voxel Diffusion](/papers/2104.03670): DDPM-style denoising on point clouds, for unconditional generation and completion.
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](/papers/2203.02923): equivariant diffusion over atom coordinates, conditioned on the molecular graph.
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](/papers/2305.07015): a frozen Stable Diffusion 2.1 with a trained time-aware encoder, for blind super-resolution.
- [Unified Camera Positional Encoding for Controlled Video Generation](/papers/2512.07237): per-token ray encodings with a latitude-and-up map, added to a video diffusion transformer through a parallel adapter.
- [Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper](/papers/2507.15062): a visuo-tactile encoder pretrained by masked reconstruction, feeding a diffusion policy.
- [Octo: An Open-Source Generalist Robot Policy](/papers/2405.12213): a pretrained tokenized transformer policy with a small diffusion action head, finetuned to new sensors and action spaces.
- [DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control](/papers/2502.05855): a VLM with a 1B diffusion action expert, trained in three stages, that writes sub-step text for long tasks.
- [Magic3D: High-Resolution Text-to-3D Content Creation](/papers/2211.10440): a two-stage text-to-3D method, a hash-grid field with a 64 px prior, then a textured mesh with a 512 px latent prior.
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](/papers/2304.05977): a BLIP-based preference reward model, and ReFL, which back-propagates it through one late denoising step of Stable Diffusion v1.4.
- [Adversarial Diffusion Distillation](/papers/2311.17042): a one- to four-step student from a frozen diffusion teacher, trained with a hinge-loss discriminator and score distillation.
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](/papers/2403.12945): a large multi-lab robot dataset, with a diffusion-policy co-training study on six tasks.
- [DiffIR: Efficient Diffusion Model for Image Restoration](/papers/2303.09472): a diffusion model over a 256-value restoration prior, in four steps, with a two-stage training run.
- [Emergent Correspondence from Image Diffusion](/papers/2306.03881): frozen diffusion features with nearest-neighbour matching for dense correspondence, with no training.
- [FPC-VLA: A Vision-Language-Action Framework with a Supervisor for Failure Prediction and Correction](/papers/2509.04018): a diffusion action head in a VLA, with a VLM supervisor that corrects actions at gripper changes.
- [VideoVLA: Video Generators Can Be Generalizable Robot Manipulators](/papers/2512.06963): a video diffusion transformer that denoises future frames and a 6-step action chunk together.
- [STORM: Search-Guided Generative World Models for Robotic Manipulation](/papers/2512.18477): a diffusion policy's candidate actions searched with MCTS over a video world model.
- [DeepCache: Accelerating Diffusion Models for Free](/papers/2312.00858): a training-free U-Net sampler that reuses cached high-level features between full passes.
- [GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Models](/papers/2312.11243): a DDPM in the latent of a conditional VAE over grasps, with DDIM sampling.
- [PDP: Physics-Based Character Animation via Diffusion Policy](/papers/2406.00960): a diffusion policy over action sequences for physics-based character control.
- [ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data](/papers/2406.19464): a diffusion policy head on fused vision and contact-microphone audio.
- [RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands](/papers/2408.11048): a piano motion benchmark that uses DDPM diffusion policies as baselines.
- [Diffusion Policy Policy Optimization](/papers/2409.00588): PPO fine-tuning of a pretrained diffusion policy, with each denoising step treated as a Gaussian policy.
- [DexDiff: Towards Extrinsic Dexterity Manipulation of Ungraspable Objects in Unrestricted Environments](/papers/2409.05493): a VLM planner with a transformer action-diffusion policy for dexterous manipulation.
- [Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation](/papers/2409.14411): a transformer diffusion policy with adaptive layer norm, scaled from 10M to 1B parameters.
- [The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control](/papers/2411.01349): an action-sequence diffusion policy for a simulated humanoid, trained on randomized RL data.
- [Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance](/papers/2411.12982): a contact-target Guider and an action Actor, both diffusion models, with a Q-value term.
- [FastGrasp: Efficient Grasp Synthesis with Diffusion](/papers/2411.14786): latent DDIM over a hand latent, with a frozen-model adaptation MLP.
- [OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving](/papers/2412.17226): object-level diffusion, then range-image scene diffusion with a controller.
- [Embodiment-Agnostic Navigation Policy Trained with Visual Demonstrations](/papers/2412.20226): a 1D U-Net diffusion policy over relative waypoints for visual navigation.

- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](/papers/2112.00390): a DDPM over masks that adds image features to the noisy-mask features, trained from scratch and averaged over several chains.
- [Perception Prioritized Training of Diffusion Models](/papers/2204.00227): reweights the simple diffusion loss by 1/(k+SNR)^gamma, down-weighting the lowest-noise steps.
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](/papers/2302.02913): a guide to metrics for design generators, with a TopoDiff topology case study.
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](/papers/2305.10431): personalizes Stable Diffusion to several people from one photo each, with a cross-attention localization loss in training.
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](/papers/2310.07896): one diffusion policy that explores when the goal is masked and heads to the goal when it is shown.
- [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](/papers/2311.16518): Stable Diffusion super-resolution guided by tags from a degradation-aware tagger, with a low-resolution start for sampling.
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](/papers/2211.14680): a diffusion prior trained only on fine flow fields, sampled from a noised coarse or sparse guide, with an optional PDE-residual term.
- [MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation](/papers/2212.09478): two coupled denoisers for video and audio, linked by random-shift cross-modal attention.
- [Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models](/papers/2305.10474): correlated frame noise that lets a text-to-image diffusion model be finetuned to video.