Diffusion models

Overview

A diffusion model is trained to remove noise. A clean sample is mixed with Gaussian noise at a randomly drawn level, and a network learns to predict the added noise from the noisy input, the level and any conditions 2301.07093 §4.2 2212.04048 §3.2 2203.13777 §3.2. Sampling starts from noise and applies the learned prediction step by step, so the number of steps sets the cost of every sample 2203.13777 §4.3 2212.04048 §4.2.

The shared invariant: the network gets the noisy state, a level and the conditions, and outputs a correction that moves the state toward the data. Conditions enter in different ways. They can be extra tokens 2301.07093 §4.1, a concatenated embedding 2212.04048 §3.3, cross-attention 2211.01324 §4.2, or a per-sample state from an encoder 2203.13777 §3.5. Some variants change the target: Grad-TTS predicts a score and Diffusion-LM predicts the clean sample, as covered under Contradictions 2105.06337 §3.2 2205.14217 §4.2.

The prediction target also differs across the cards. Some predict the noise, some predict a velocity 2312.06662 §3, some predict the clean sample in planning 2407.01392 app. D, and discrete models predict a distribution over classes 2209.14734 §3.2. Discrete variants replace Gaussian noise with masking or category switching 2406.07524 §3.1 2209.14734 §3.1.

For a broader map of the field, the 2023 state-of-the-art report on diffusion models for visual computing explains the SDE and probability-flow view and the latent design of Stable Diffusion. It runs no experiments of its own, so it is a reading guide rather than a source of results 2310.07204 §3.1 2310.07204 §3.2. Card: State of the art on diffusion models for visual computing.

A survey of world models treats video generators as a main current route to learned simulators, and names weak causal reasoning and physics failures as their main limits. It runs no experiments 2411.14499 §4.1.

The 99 cards cover image, video and 3D generation, grasp synthesis, LiDAR scenes, humanoid whole-body control, dexterous piano playing, audio-visual manipulation, physics-based character control, visual navigation, text-to-3D meshes, CLIP-guided textured meshes, 3D materials, sparse-view 3D reconstruction, layouts, text, graphs, motion, speech, talking faces, trajectories, time series, MRI reconstruction, robot planning, control, robot demonstration data, generalist robot policies, diffusion action heads with failure checks, joint video and action generation, search over imagined futures, street-view rendering, medical masks, synthetic training data, reward-model fine-tuning, adversarial one-to-four-step distillation, test-time adaptation, safety testing, interactive game worlds, a travelling-salesman search, one theory result, label-efficient segmentation, end-to-end mask diffusion, loss weighting over noise levels, tuning-free multi-subject personalization, metrics for generated designs, semantic super-resolution, goal-masked navigation, molecular conformations, super-resolution, image restoration, dense correspondence, faster sampling from cached features, camera-controlled video, tactile manipulation, a tutorial derivation, whether generators copy their training images, mask-guided object inpainting, layout control with and without training, a noise-schedule fix for the brightness range, physics-informed flow field reconstruction, joint audio and video from coupled denoisers, and correlated frame noise for video finetuning. Their results are on different tasks and metrics, so the numbers below are not comparable across variants.

Variants

One derivation for the noise, clean and score targets

A tutorial treats a diffusion model as a Markovian hierarchical VAE whose noising steps are fixed Gaussians 2208.11970 §3. Under that view the ELBO becomes a sum of denoising terms, and each term can be written as predicting the clean sample, the source noise or the score, each with its own time-dependent weight 2208.11970 §3. The tutorial also covers the closed-form ancestral sampler, with the reverse variance fixed to the posterior's and only the mean learned 2208.11970 §3, and it treats classifier and classifier-free guidance as score combinations 2208.11970 §5. It runs no experiments, and it names the costs itself: many sequential sampling steps, latents as large as the data, and a fixed encoder 2208.11970 §6. Use it to check the algebra behind a variant, not as a source of numbers. Card: Understanding Diffusion Models: A Unified Perspective.

Copy detection for generated images

Somepalli et al. ask whether diffusion models reproduce their training images. They score every generated image against every training image with a retrieval detector, using DINO ViT-B/16 token features and a split-product similarity, which keeps the best token pair per image pair [arxiv:2212.03860 §4]. A top-1 score above 0.5 is treated as a copy for Stable Diffusion 2212.03860 §7. Small DDPMs trained on Celeb-A (300 and 3000 images) and on Oxford Flowers (100 and 1083 images) produce blatant copies, while the full-data Celeb-A checkpoint and an ImageNet latent model show none the tool found 2212.03860 §5 §6. For Stable Diffusion v1.4, about 1.88% of 9,000 generations pass that cut-off, but only the 12M-image LAION Aesthetics v2 6+ subset was searched. The authors say the rate is likely an underestimate 2212.03860 §7 §9. The matching is object-level, and the paper admits grey areas in the definition 2212.03860 §1 §3. Card: Diffusion Art or Digital Forgery?.

Latent diffusion with a frozen backbone and added conditions

The denoiser works on latents, not on pixels 2301.07093 §3 app. A. GLIGEN keeps all of the pretrained weights frozen and trains only new layers, which take extra conditions as tokens and gate their output through a learned scalar that starts at zero. At initialisation the model therefore reproduces the pretrained model's output 2301.07093 §4.2. The paper uses this to add box-grounded text-to-image generation 2301.07093 §4.1. Card: GLIGEN.

Inside that setup, the authors prefer gated self-attention to gated cross-attention, which performs worse in their ablation [arxiv:2301.07093 app. B]. The attention block itself is covered on Attention mechanism.

Scheduled conditioning at sampling time

The added conditions can be switched off after the first fraction tau of the denoising steps, so that early steps fix the layout and later steps refine detail with the original model 2301.07093 §4.2 eq. 10. The paper reports this only qualitatively, using tau = 0.2 for one figure [arxiv:2301.07093 §5.4] fig. 7. Card: GLIGEN.

Latent diffusion over motion clips

MLD first compresses a motion clip with a transformer VAE. In the best setting the latent is a single 256-d token 2212.04048 §5 tab. 5. A transformer denoiser then runs diffusion in that latent. Text goes through a frozen CLIP encoder and action labels through a learned embedding, with guidance dropout 0.1 and scale 7.5 2212.04048 §3.3 tab. 11. Author-reported FID on HumanML3D is 0.473, against 0.544 for MDM and 0.630 for MotionDiffuse 2212.04048 tab. 1. The paper's R-Precision and MM Dist claims do not match its tables, where MotionDiffuse ranks first on those metrics, so only the FID lead is clear. Card: MLD.

Expert denoisers split by noise level

eDiff-I is a text-to-image cascade: a 64 px base model, then super-resolution to 256 px and 1024 px 2211.01324 §4.2 fig. 5. The base is split by noise level into experts. One shared model is trained on all levels, then children are initialised from it and fine-tuned on their own band of levels. The final base uses three experts: lowest noise, highest noise, and a middle one 2211.01324 §4.1 tab. 5. The authors claim the ensemble costs the same per sampling step as one network, with the numbers shown only in a figure 2211.01324 §5.1 fig. 13. Conditioning uses T5-XXL, CLIP text and CLIP image embeddings, each dropped independently 2211.01324 §4.2. The training-free paint-with-words control adds a weighted mask bias to the cross-attention logits 2211.01324 §4.3. Author-reported zero-shot FID-30K is 6.95 for the best configuration, against 7.23 for Parti and 7.27 for Imagen 2211.01324 tab. 1. No code or weights are released, and the training data is about one billion pairs, partly proprietary [arxiv:2211.01324 §5]. Use this variant as a design reference. Card: eDiff-I.

Score-based decoder for speech features

Grad-TTS maps text to mel-spectrograms. An encoder predicts per-token means and durations, and durations come from a monotonic alignment search [arxiv:2105.06337 §3.2]. The diffusion decoder starts from noise centred on the frame means and takes N Euler steps of a reverse ODE. A HiFi-GAN vocoder then makes audio 2105.06337 §3.1 §4. N is the speed-quality dial. Author-reported LJSpeech MOS is 4.38 at N = 10 (Tacotron2 4.32, ground truth 4.53), 3.96 at N = 4, and almost no change from 10 to 1000 steps 2105.06337 tab. 2 §4.1. The real-time factor is 0.033 at N = 10, against 0.075 for Tacotron2, both measured by the authors on a GPU 2105.06337 tab. 2. Card: Grad-TTS.

Reverse diffusion over future trajectories

MID denoises future pedestrian positions. A Trajectron++ state encoder summarises the observed history of 8 frames and the neighbours. A Transformer denoiser predicts the noise over K = 100 steps 2203.13777 §3.5 §5 alg. 1. Each history gets 20 draws (Best-of-20). Author-reported SDD ADE and FDE are 7.61 and 14.30, the lowest of the trajectory-only rows in its table. On ETH/UCY the average is comparable rather than best, at 0.21 and 0.38 m 2203.13777 tab. 1 tab. 2. The cost is time: 17.4 s against 0.44 s for Trajectron++ on 512 trajectories 2203.13777 §5. Fewer steps give more diverse samples and lower accuracy 2203.13777 §4.3 fig. 3. Card: MID.

Continuous diffusion over word embeddings

Diffusion-LM denoises a sequence of word-embedding vectors and rounds the result to words. Its continuous latents can be steered by gradients from a classifier. Author-reported success on classifier-guided control is 81.2 against 69.9 for FUDGE on semantic content, and 90.0 against 27.0 on parts of speech 2205.14217 tab. 2. Its variational bound on E2E is worse than an autoregressive Transformer's, at 2.28 against 1.77, and decoding takes many steps 2205.14217 §7 §6.1. The text-specific details are on Continuous text diffusion. Card: Diffusion-LM.

Masked diffusion over discrete tokens

MDLM corrupts text by replacing each token with a mask token at a rate set by a schedule, and a Transformer learns to fill the masks 2406.07524 §3.1. Once a token is unmasked it is carried over unchanged, and the model cannot output the mask token 2406.07524 §3.2.3. The continuous-time training loss is then a weighted average of masked-token cross-entropies 2406.07524 §3.5. Sampling starts from all masks and unmasks tokens over N steps, and the paper's semi-autoregressive variant writes a 2,048-token text in blocks 2406.07524 §4.2. Author-reported LM1B perplexity bound is 23.00 at 327B training tokens, against 20.86 for a retrained autoregressive model 2406.07524 tab. 1. On OpenWebText the bound is 23.21, against 17.54 for the autoregressive model 2406.07524 tab. 2. Diffusion numbers are upper bounds, not exact likelihoods 2406.07524 tab. 1. Card: MDLM.

Discrete denoising over graphs

DiGress applies diffusion to graphs with categorical node and edge labels, where "no edge" is one of the edge classes 2209.14734 §3. Each node and edge is corrupted by a Markov chain that switches categories, and a graph transformer predicts the clean classes with cross-entropy 2209.14734 §3.1 2209.14734 §3.2. Two additions carry most of the reported gain: noise that keeps the training-set class frequencies 2209.14734 §4.1, and cycle and spectral features computed on the noisy graph 2209.14734 §4.2. Author-reported valid-unique-novel rates are 74% on SBM and 75% on planar graphs 2209.14734 tab. 1. On QM9 the validity is 99.0% [arxiv:2209.14734 tab. 2]. Card: DiGress.

Discrete diffusion over layout tokens

LayoutDM writes each layout element as a category plus four quantized geometry tokens, padded to 25 elements 2303.08137 §3.2 §4.1. Each attribute has its own corruption matrix, and a token is kept, swapped for another value of its attribute, or masked 2303.08137 §3.2 eq. 3. Known fields, refinement and relation constraints are applied at inference, with no retraining: known fields are masked back in at every step, and weak priors shift the reverse distribution in log space 2303.08137 §3.3. Author- reported unconditional Rico FID is 6.65, against 6.47 for LayoutTrans-fixed, which uses a fixed element order 2303.08137 tab. 2. On refinement the paper reports Rico FID 2.77 against 3.23 for the task-specific RUITE, but PubLayNet FID 6.75 against 6.39 2303.08137 tab. 3. Card: LayoutDM.

Distilling a multi-step sampler into one step

The Denoising Student keeps a deterministic DDIM teacher and trains a student to produce the teacher's final image from the same starting noise, with a plain regression loss and no adversarial term 2101.02388 §3.1. The student starts from the teacher's weights 2101.02388 §3.2. The teacher uses 100 DDIM steps for CIFAR-10 and 50 for LSUN 2101.02388 §4. Each training pair needs one full teacher run, and the paper uses 1.024M pairs per dataset 2101.02388 §3.1. Author-reported CIFAR-10 FID is 9.36 at one step, against 4.16 for the 100-step teacher 2101.02388 tab. 1. Generating 50k CIFAR-10 images takes 51.5 s for the student and 4.96 × 10³ s for the teacher, on one V100 2101.02388 tab. 2. The method needs a deterministic teacher, so score-based and energy-based samplers do not qualify 2101.02388 §2.1. Card: Denoising Student.

Reusing high-level features across sampling steps

DeepCache changes no weights. A full U-Net pass runs every N steps, and the steps in between reuse the high-level features cached at the last full pass and recompute only the shallow blocks 2312.00858 §3.3. A non-uniform spacing of the full passes, with a centre and a power, is also reported. At N = 10 on ImageNet it gives FID 4.27, against 4.41 for uniform spacing 2312.00858 tab. 1. Author-reported throughput on Stable Diffusion v1.5 is 2.15x at N = 5, with a CLIP score of 29.46 against 29.51 for the 50-step sampler on PartiPrompts 2312.00858 tab. 3. On ImageNet with LDM-4-G the same N gives 4.12x at FID 3.59, against 3.37 for the authors' regenerated baseline 2312.00858 tab. 1. Quality falls as N grows: on CIFAR-10, FID is 4.35 at N = 2 and 9.74 at N = 10 2312.00858 tab. 2. Replacing the cached features with zeros is far worse, raising FID from 9.74 to 192.98 on CIFAR-10 2312.00858 tab. 5. At equal throughput on ImageNet, the authors report it matches DDIM with 59 steps, at FID 3.59 for both 2312.00858 tab. 4. The 2.3x headline in the abstract does not match any single table row, so check the card's Pitfalls before quoting it. Card: DeepCache.

Chunked causal video student with a KV cache

CausVid turns a bidirectional video diffusion transformer into a causal one that generates chunks of frames in order 2412.07772 §4.1. The student is first initialised by regressing onto teacher ODE trajectories, using 1000 pairs 2412.07772 §4.3 §5 Training. It is then trained with asymmetric distribution matching distillation: the teacher is bidirectional, and the student sees only past chunks 2412.07772 §4.2. A separately trained fake score is updated five times per student update 2412.07772 §4.2. At inference each chunk is denoised in four steps while attending to cached keys and values of earlier chunks 2412.07772 §4.4. Author-reported latency for a 10 s, 120-frame clip is 1.3 s, against 219.2 s for the bidirectional teacher, at 9.4 frames per second 2412.07772 tab. 3. The few-step student scores 94.7 on temporal quality, against 94.6 for the bidirectional many-step teacher and 92.4 for a causal many-step model 2412.07772 tab. 4. Card: CausVid.

Distilled causal world model for interactive video

Hunyuan-GameCraft-2 turns a 14B image-to-video mixture-of-experts model into a few-step, causal, block-wise generator for interactive game video 2511.23429 §4.2. Keyboard and mouse inputs become camera-control signals, and free-form text goes through a multimodal LLM as extra interaction guidance 2511.23429 §4.1. Distillation follows Self-Forcing, and the first frame is kept as a sink token in the KV cache 2511.23429 §4.2.3. A long-video tuning stage rolls out clips longer than 10 seconds to limit drift 2511.23429 §4.2.4. The authors report 16 FPS, with FP8 quantisation, parallel VAE decoding and sequence parallelism. The GPU model and count are not stated 2511.23429 §4.4. The InterBench judge is an unnamed VLM, and the authors report semantic drift beyond about 500 frames 2511.23429 §5.2.2 §6. Card: Hunyuan-GameCraft-2.

Per-token noise levels for sequences and planning

Diffusion Forcing gives each token of a sequence its own independent noise level, and a causal recurrent network carries a latent state over the tokens 2407.01392 §3.2. Setting different noise levels for different tokens yields a sampler that can generate long rollouts, guide over future tokens, and plan over a flexible horizon 2407.01392 §3.3 §3.4. Author-reported average reward on the single-task Maze2D benchmarks is 141.7, against 119.5 for Diffuser with its hand-built controller 2407.01392 §4.2. The paper says its rollouts stay stable beyond the training horizon, shown qualitatively 2407.01392 §4.1. Card: Diffusion Forcing.

Text-conditioned video as a planner

UniPi plans with a text-conditioned video diffusion model. The model takes the observed first frame as conditioning, with the frame repeated over time, and it is trained with that conditioning rather than only clamped at test time 2302.00111 §3.1 app. A.1. An inverse-dynamics model then reads actions from the generated frames, and the actions are executed open-loop 2302.00111 §3.2. Author-reported task completion on seen place instructions is 59.1%, against 19.4% for the best state-based transformer baseline 2302.00111 tab. 1. With a real robot, pretraining on internet video raises success from 72.6% to 77.1% 2302.00111 tab. 4. Card: UniPi.

Video diffusion features for robot actions

DiT4DiT couples a video diffusion transformer (Cosmos-Predict2.5-2B) to an action diffusion transformer, and trains both with flow-matching losses 2603.10448 §4.2 §4.3. The action head reads hidden activations of one video block at a fixed timestep, so control does not need generated frames 2603.10448 §4.2. The authors report 98.6% average success on LIBERO, with the model trained from scratch 2603.10448 tab. 1. Card: DiT4DiT.

Diffusion action head inside a VLA, with a checker

FPC-VLA uses a diffusion transformer as the action head of a vision-language- action policy. The head predicts the noise added to a ground-truth action sequence, conditioned on a token from a Llama2 backbone 2509.04018 §3.2 eq. 3 eq. 4. The paper does not state the number of denoising steps 2509.04018 §3.2. The diffusion head is not the paper's main addition. A separate Qwen2.5-VL supervisor is queried only when the gripper state changes, and it can add a directional correction to the action 2509.04018 §3.5 alg. 2. Past predictions are averaged with cosine and time weights before the supervisor sees them 2509.04018 §3.3 alg. 1. Author-reported SIMPLER WidowX average task success is 64.6, against 51.8 for CogACT, with the model not fine-tuned 2509.04018 tab. 3. Without the supervisor, the ablation's task average falls from 82.1 to 74.4 2509.04018 tab. 7. With the paper's stated fusion constants, the similarity factor changes the pose weights by about 10% at most, by our arithmetic from 2509.04018 eq. 8 and §4.1, so the fusion is close to a plain average. Card: FPC-VLA.

Joint video and action denoising in one transformer

VideoVLA starts from the pretrained CogVideoX-5B video diffusion transformer. One denoising network takes the text, the current observation, the future video latents and the action tokens as one sequence, and denoises the video and a 6-step action chunk together 2512.06963 §3.2 §3.4. Both are noised under the DDPM schedule and trained with the diffusion loss, and the timestep enters through adaptive LayerNorm 2512.06963 §3.4. Inference uses 50 denoising steps, and the first three actions are executed before the model is queried again 2512.06963 §4.1. Author-reported SIMPLER average over 12 tasks is 63.0, against 62.6 for CogACT 2512.06963 tab. 1. On novel objects it reports 65.2 against 42.4 for CogACT, and on new skills 48.6 against 20.4 2512.06963 tab. 2 tab. 3. Without the video loss, the same two generalisation columns fall to 12.7 and 4.4 2512.06963 tab. 9. Training from scratch instead of from the video model gives an average of 12.6, against 80.4 for the default, on the three-task ablation 2512.06963 tab. 7. The authors report about 1.1 s per chunk with 10 steps on one H100, roughly 3 Hz 2512.06963 app. D. The WidowX tables use 24 trials per task, so one success is 4.2 points, and the 0.4-point gap to CogACT is within that noise 2512.06963 app. A. Card: VideoVLA.

Diffusion proposals under tree search

STORM keeps a diffusion policy as the action proposer, and adds a search loop around it. The proposer is CogACT-Base, used without further training, and it returns K = 8 candidate actions with priors 2512.18477 §3.1 §4.1. Each candidate is rolled forward in a video world model, and Monte Carlo Tree Search picks the root action with the most visits 2512.18477 §3.2 §3.3. The world model is autoregressive over discrete tokens, not a diffusion model 2512.18477 §3.2. Author-reported WidowX mean success is 51.0, against 47.9 for CogACT, and the authors call the margin modest 2512.18477 tab. 1 §4.3. The paper gives no latency figures for the search 2512.18477 §5. Card: STORM.

Video generation with one transformer for images and clips

W.A.L.T trains one transformer diffusion model on images and videos in a shared latent space from a causal autoencoder, where an image is a one-frame video 2312.06662 §4.1. The latent has 8 channels by default [arxiv:2312.06662 §5.2]. Attention is local: spatial windows of one frame and spatiotemporal windows across frames, alternating 2312.06662 §4.2 §5.2. Conditioning uses AdaLN-LoRA with rank 2 and self-conditioning with probability 0.9 2312.06662 §4.3 §5.2. Author-reported FVD on Kinetics-600 frame prediction is 3.3, against 4.3 for MAGVIT-v2 2312.06662 tab. 1. On ImageNet 256 the large model reaches FID 2.56 without classifier-free guidance 2312.06662 tab. 2. Self-conditioning cuts the UCF-101 FVD from 109.9 to 61.4 2312.06662 tab. 3. A three-model cascade makes text-to-video 2312.06662 §4.5. Card: W.A.L.T.

Image-to-video from a fine-tuned image model

DreamPose starts from Stable Diffusion, fine-tunes it on fashion videos to follow a pose sequence, then runs a short per-subject fine-tune on the user's photos 2304.06025 §4.2 §4.3. Each frame is sampled on its own, conditioned on the photo and five consecutive pose maps, so temporal smoothness comes from the conditioning rather than from a joint video model 2304.06025 §4.1. Author-reported results on UBC Fashion are state of the art against MRAA, TPSMM and PIDM 2304.06025 tab. 1. The set-up is slow for each new subject: about 10 minutes of UNet tuning and 20 minutes of VAE-decoder tuning, plus 18 s per rendered frame 2304.06025 §6. Card: DreamPose.

Correlated frame noise when finetuning an image model to video

PYoCo starts from a pretrained text-to-image diffusion model and finetunes it jointly on images and videos. The change is in the noise: frames from one clip draw correlated starting noise, not independent noise 2305.10474 §4. Two constructions are given, a mixed one with a shared part and a per-frame part 2305.10474 eq. 3, and a progressive one in which each frame perturbs the previous frame's noise 2305.10474 eq. 4. Each frame keeps unit variance 2305.10474 §4. On UCF-101 the authors' strategy table gives FVD 903.37 for training from scratch, 566.67 for finetuning with independent noise, and 337.40 and 339.67 for the mixed and progressive priors 2305.10474 tab. 5. Zero-shot text-to-video on UCF-101 reaches IS 47.76 and FVD 355.19 2305.10474 tab. 2. The sweep over the correlation strength used only a 69M-parameter model, and the large-scale run uses progressive noise with α = 2 2305.10474 §5.3. The card records no official code. Card: PYoCo.

Audio-driven portraits without a skeleton

EMO draws each video frame directly from one photo and one audio clip. It uses no 3D face model, landmark sequence or pose template 2402.17485 §1 §3. A reference network keeps the identity, audio attention drives the motion, and a face-box mask and a head-speed embedding act as deliberately weak conditions 2402.17485 §3.2. Long videos are built from 12-frame clips that share four motion frames with the previous clip 2402.17485 §3.2.4 §4.1. Author-reported FID on HDTF is 8.76, the best in its table, and FVD is 67.66 2402.17485 tab. 1. The authors state that their SyncNet score is lower than Wav2Lip and SadTalker 2402.17485 tab. 1. Card: EMO.

Real-time talking faces in a learned face space

VASA-1 learns a disentangled face latent for identity, head pose, facial dynamics and a 3D appearance volume. A diffusion transformer then generates pose and dynamics codes from wav2vec 2.0 audio features 2404.10667 §3.1. Because it generates compact codes rather than pixels, rendering is fast and streams in short overlapping chunks 2404.10667 §3.1. The authors report 512×512 video at up to 40 FPS on one RTX 4090, and the best lip sync and FVD against MakeItTalk, Audio2Head and SadTalker on two benchmarks they built 2404.10667 §4. The code is not public, and the paper does not give its training losses or several architecture details 2404.10667 §3.1. Card: VASA-1.

Joint audio and video in one diffusion transformer

JavisDiT has a video branch and an audio branch, linked by bidirectional cross-attention 2503.23377 §3.1. A learned spatio-temporal prior maps the prompt to 32 spatial and 32 temporal tokens, and cross-attention injects them into both branches 2503.23377 §3.2 app. E.3. Training has three stages, and the last one trains only the cross-attention layers 2503.23377 §3.3. Author-reported JavisScore is 0.154 against 0.151 for FoleyCrafter on JavisBench, while UniVerse-1 is better on FVD, KVD and TV-IB 2503.23377 tab. 1. Card: JavisDiT.

Joint audio and video from two coupled denoisers

MM-Diffusion runs one U-Net for video and one for audio, and denoises both at each step 2212.09478 §3.2 2212.09478 §3.3. The two streams are linked by random-shift cross-modal attention: each audio segment attends to a window of video frames whose start is drawn at random, which is cheaper than full cross-attention 2212.09478 §3.3. Audio from video, or video from audio, is generated with no extra training, by replacing the condition with a noised copy of the real one or by gradient guidance 2212.09478 §3.4. Author-reported FVD is 117.20 on Landscape and 75.71 on AIST++, both with full 1,000-step DDPM sampling. The faster DPM-Solver sampler gives 229.08 on Landscape 2212.09478 tab. 1 2212.09478 tab. 2. The conditional generation is shown in figures, not in a table 2212.09478 §4.5. In a user study, 84.9% of Landscape clips were judged real, against 92.5% for real clips 2212.09478 tab. 6. The baseline is a two-stage pipeline, with checkpoints and conditioning not fully stated, and it scores far lower on video quality, 1.61 against 3.75 on Landscape 2212.09478 §4.6 [arxiv:2212.09478 tab. 5]. Card: MM-Diffusion.

Reward-weighted and DPO fine-tuning from an AI judge

The authors fine-tune a 3B text-to-video diffusion model offline, on 120 prompts about object movement 2412.02617 §4 app. A. The reward is a binary accept or reject from a vision-language model that reads eight frames 2412.02617 §3.3 app. D. Reward-weighted regression (RWR) and DPO are both derived from one probabilistic objective 2412.02617 §3.1. On the authors' Gemini "All" column, RWR on AI feedback goes from 52.66% to 56.41%, and DPO goes to 56.17%. On human "All" accuracy the gains are 19.38% to 31.09% (RWR) and 34.06% (DPO) 2412.02617 tab. 2. The judge is from the same model family as the reward, and an independent GPT-4o check gives a smaller gain of about 2.4 points for RWR 2412.02617 tab. 2. The evaluation has 40 test prompts. Imaging quality in the VBench table is below the pre-trained model in every row 2412.02617 tab. 2. Card: AI-feedback fine-tuning.

Learned preference reward used on a late denoising step

ImageReward trains a scorer on pairwise human rankings of generated images 2304.05977 §2.1 §2.2. The scorer is a BLIP backbone with a small MLP head, trained with a pairwise logistic loss on the score gap [arxiv:2304.05977 §2.2] eq. 1. Author-reported preference accuracy on 6,399 held-out comparisons is 65.14%, against 57.76% for the BLIP score and 54.82% for CLIP score 2304.05977 tab. 3. ReFL uses the same scorer to tune Stable Diffusion v1.4. Each update draws one late denoising step at random, decodes the result and back-propagates the reward 2304.05977 §3 alg. 1. Against the untuned model, ReFL wins 58.79% of human comparisons on real-user prompts 2304.05977 tab. 4. The released ReFL code differs from the paper in its loss, its regulariser and its training length, so see the card's Paper vs code section before reproducing it. Card: ImageReward.

Adversarial term added to one-step distillation

Adversarial Diffusion Distillation starts a student from the teacher's weights 2311.17042 §3.1. A discriminator on frozen ViT features, with several small heads, is trained with a hinge loss, and the student is trained to fool it 2311.17042 §3.2. A score distillation term, a squared-L2 match to the frozen teacher's clean estimate on re-noised student samples, is added with weight 2.5 2311.17042 §3.3 §4. One student evaluation gives an image, with no classifier-free guidance 2311.17042 §1 §4.2. Author-reported, one-step ADD-M reaches COCO FID-5k 19.7 and CLIP 0.326, the best in its table among SD1.5-based few-step methods 2311.17042 tab. 2. In human preference, one step of ADD-XL beats four steps of LCM-XL, and four steps beat SDXL at 50 steps in most comparisons 2311.17042 §4.2. The student is less diverse than its teacher 2311.17042 §4.3. The learning rate, batch size and iteration count for ADD-M and ADD-XL are not stated, and the released repository has no training code for them (see the card's Paper vs code section). The card also records that the released SDXL-Turbo weights carry a non-commercial licence, so do not assume commercial use of them. Card: Adversarial Diffusion Distillation.

Fused multi-view tiles lifted to 3D

One-2-3-45++ draws six fixed views in one 3×2 tile, using a fine-tuned Stable Diffusion 2 conditioned on the input image 2311.07885 §3.1. The tile is then lifted by two diffusion stages over occupancy and sparse signed-distance volumes, conditioned on DINOv2 features projected from the six views and max- pooled 2311.07885 §3.2. A colour field is fitted last [arxiv:2311.07885 §3.3]. Author-reported F-Score on the GSO set is 93.6 and CLIP similarity 81.0, at about 60 s per object 2311.07885 tab. 1. The paper gives no implementation details beyond what is above, and training used 8 A100 GPUs for about 10 days per 3D stage 2311.07885 §3.2. Card: One-2-3-45++.

Video diffusion as a prior for physical motion

PhysDreamer animates a still 3D Gaussian object with an image-to-video model, then fits a stiffness value for each region, together with an initial velocity, so that a differentiable physics simulation matches the generated clip 2404.13026 §4 fig. 3. The method assumes elastic materials, no collisions and one constant Poisson's ratio 2404.13026 §3 §6. Author-reported 2AFC study on eight real scenes: 80.8% of raters prefer it to PhysGaussian on motion realism, and 63.5% prefer it to DreamGaussian4D 2404.13026 tab. 1. The optimisation is slow 2404.13026 §6. Card: PhysDreamer.

Latent diffusion for 3D shapes with per-input weights

SDFusion compresses truncated signed-distance grids with a 3D VQ-VAE, then runs diffusion in that latent 2212.04493 §3.1 §3.2. Partial shapes, images and text enter through separate encoders and cross-attention. Each input gets a weight at sampling time, and the paper's rule adds one weighted difference per input to the unconditional prediction 2212.04493 §3.3 eq. 4. Author-reported shape-completion UHD on ShapeNet is 0.0557, against 0.0567 for AutoSDF, with a larger diversity score 2212.04493 tab. 1. On Pix3D the F-score is 0.432, against 0.415 for AutoSDF 2212.04493 tab. 2. The per-input rule is implemented only in one code branch, so the released sampler differs from the equation 2212.04493 §3.3. Card: SDFusion.

Composable conditions trained by random subsets

Composer splits each training image into eight representations, among them a caption, a colour histogram, a sketch, instance masks, depth and a grayscale version 2302.09778 §2.2. Each representation is dropped independently with probability 0.5, and all are dropped or all kept with probability 0.1 each 2302.09778 §2.3. Because of this training, chosen representations can be recombined at inference for layout, palette and style control 2302.09778 §3.2. The cascade runs at 64, then 256, then 1024 pixels 2302.09778 §2.3. The base model is trained on about one billion images 2302.09778 §3.1. Author-reported COCO FID is 9.2 and CLIP score 0.28 2302.09778 §3.5. The paper names no baseline numbers, so only the absolute values can be quoted. Card: Composer.

Box tokens with object-aware cross-attention, trained from scratch

Each layout object is a box and a class. A transformer over these tokens lets the objects see each other, and each image patch carries its own box position so that cross-attention can match patches to objects 2303.17189 §3.2 §3.3. The model is trained from scratch on box-labelled images, so it has no text prompt and no pretrained image prior 2303.17189 §5. Training uses classifier-free dropout at 0.2 and sampling uses DPM-Solver in 25 steps 2303.17189 §3.4 tab. 4. On COCO-Stuff at 256 px, the author-reported FID is 15.61, against 29.10 for PLGAN, a GAN baseline 2303.17189 tab. 1. On COCO-Stuff at 128 px, removing both the layout fusion and the object-aware attention gives FID 29.94, against 16.57 for the full model [arxiv:2303.17189 tab. 2]. The released code differs from the paper in several settings, so reproduce from the card's Paper vs code notes. Card: LayoutDiffusion.

Training-free guidance of a pretrained model at sampling time

Semantic diffusion guidance adds language, reference-image or combined guidance to a pretrained unconditional DDPM 2112.05744 §3.1. At each step the mean is shifted along the gradient of a CLIP-based score, and the CLIP image encoder is adapted to noisy images by a contrastive finetune with no text labels 2112.05744 §3.5. Author-reported FFHQ image-guidance FID is 14.37, against 17.15 for ILVR 2112.05744 tab. 1. For text guidance FID is 28.38, against 57.45 for StyleGAN+CLIP, but the baseline has higher retrieval scores 2112.05744 tab. 1. Raising the scale on LSUN cats from 100 to 200 raises top-1 retrieval and lowers LPIPS, so the scale trades match against diversity 2112.05744 tab. 2. The structure-preserving content loss is not in the released code 2112.05744 §3.3. Card: Semantic diffusion guidance.

MultiDiffusion uses one pretrained text-to-image model on overlapping windows and averages the window outputs at each pixel and step, with no training 2302.08113 §3 alg. 1. The same averaging, with masks as weights, handles region prompts 2302.08113 §4.2. Author-reported panorama FID is 10.3, against 18.4 for Blended Latent Diffusion and 45.5 for Stable Inpainting 2302.08113 tab. 1. On a COCO subset the region IoU is 0.26, against 0.17 and 0.16 for those two baselines 2302.08113 tab. 2. Bootstrapping, which masks the views in early steps, improves the IoU from 0.18 to 0.26 2302.08113 §4.2 tab. 2. Cost grows with the panorama area, since each step calls the model once per window 2302.08113 app. B.1. Card: MultiDiffusion.

Backward guidance on cross-attention maps of a frozen model

The model's weights stay fixed. During the first denoising steps, the latent is moved along the gradient of an energy that is small when each prompt word's cross-attention mass lies inside its target box 2304.03373 §3.2 eq. 2 eq. 3. The default applies this for the first 10 steps, with up to five updates per step, on the mid-block and first up-block layers of Stable Diffusion v1.5 2304.03373 §4.1. The same method is also applied to a GLIGEN model, where it raises COCO mAP from 62.8 to 65.1 [arxiv:2304.03373 tab. 3]. Against a variant that edits the attention maps directly, the authors report better VISOR conditional accuracy at about twice the per-image time 2304.03373 tab. 1. On VISOR, conditional accuracy is 95.95% for Stable Diffusion with this guidance, against 62.98% without it [arxiv:2304.03373 tab. 2]. Stable Diffusion's COCO mAP rises from 19.2 to 35.7 [arxiv:2304.03373 tab. 3]. Check the card's Paper vs code notes before using the released code, since it lacks the attention-map variant and the benchmark scripts. Card: Training-Free Layout Control with Cross-Attention Guidance.

Attention modulation for dense captions and regions

DenseDiffusion keeps a frozen Stable Diffusion model and changes only its attention during the first 30% of denoising steps 2308.12964 §3.2. Each prompt segment comes with a binary region map. Word-to-pixel scores inside a region are raised and scores outside it are lowered, scaled by the score range and by the region's area 2308.12964 §3.2 eq. 2. On 250 MS-COCO captions the authors report SOA-I 77.61, against 73.08 for Stable Diffusion 2308.12964 tab. 2. For layout fidelity the IoU is 34.99, against 23.76 for SD-Pww, the only baseline with layout control 2308.12964 tab. 3. Dropping the area term raises IoU to 38.00 but lowers SOA-I to 73.62, so the terms trade text fidelity against layout 2308.12964 tab. 4. Each region must be given as a mask, and each region's phrase must appear in the prompt 2308.12964 §3. Card: DenseDiffusion.

Shared attention for one style across a batch

StyleAligned gives a set of images one shared style without training, on a pretrained SDXL model 2312.02133 §3.2 §4. In each self-attention layer, every image attends to its own tokens and to those of one reference image, after AdaIN on queries and keys 2312.02133 §3.2. Sharing across all images gives consistent styles but leaks content and reduces diversity, which is why only the reference is shared 2312.02133 §3.2 §4.1. The paper reports about 29 s for a four-image set on one A100 [arxiv:2312.02133 §4]. The authors' tests found that encoder-based personalisation methods struggle to separate style from content 2312.02133 §4.2. Card: StyleAligned.

Tuning-free multi-subject personalization with localized attention

Each subject token in the prompt is replaced by an MLP applied to that token's text embedding and a CLIP image embedding of one reference photo of the subject 2305.10431 §4.1 eq. 2. Other tokens keep their text embedding, so no training run is needed for each person. During training a localization loss pushes each subject token's cross-attention inside its segmentation mask, with weight 0.001 2305.10431 §4.2 eq. 3 eq. 4. The masks and this loss are used only in training. At sampling time the early steps use the plain text prompt, and the last fraction alpha uses the subject-augmented prompt. The paper suggests alpha between 0.6 and 0.8 2305.10431 §4.3 fig. 5. Training pairs come from captioned images whose noun phrases are matched to segments 2305.10431 §5.1. On the authors' single-subject test, identity preservation is 51.41% with one reference image, against 43.37% for Custom Diffusion with five 2305.10431 tab. 1. On the multi-subject test it is 43.11%, against 13.52% for Textual Inversion 2305.10431 tab. 2. Without the localization loss, identity is 37.66% and prompt consistency is 25.03%, against 24.25% with it 2305.10431 tab. 3. The released code's embedding MLP is deeper than the single one written in eq. 2, so the equation alone does not reproduce the checkpoint. Card: FastComposer.

Plug-and-play priors and posterior sampling

A diffusion model trained on its own data can serve as a prior for a different task, with a differentiable constraint added at inference and no further training 2206.09012 §1 §2.1. The method minimises the denoising error minus the log constraint, drawing fresh noise at each step, with the noise level annealed from high to low 2206.09012 §2.2 alg. 1. On a 50-node travelling salesman problem the author-reported gap is 1.28% without beam search 2206.09012 tab. 2. On land-cover segmentation, the fine-tuned version reaches 79.8% accuracy on Durham 2206.09012 tab. 1. Each experiment was run once, and inference is stochastic 2206.09012 §2.2. Card: Plug-and-play priors.

For inverse problems in medical imaging, a score prior replaces the learned reconstruction network. For MRI, the method draws posterior samples by annealed Langevin dynamics on the measurement model, and the prior is trained on T2 brain slices without the sampling pattern 2108.01368 §2.2 §4. The released sampler does not implement the stated data-term equation, so the published numbers come from the code's rule 2108.01368 §2.2. On a high-resolution brain, the authors report 16 minutes and 0.95 GB on an RTX 2080Ti 2108.01368 §4.1. A radiologist study ranks it first on knees, with mean rank 1.17 against 1.87 for MoDL, but the authors call it preliminary 2108.01368 tab. 1 app. H. Card: Robust CS-MRI.

Physics-informed reconstruction of fluid fields from coarse or sparse input

A diffusion model trained only on fine two-dimensional flow fields reconstructs them from coarse or sparse inputs. The input is noised to an intermediate step, and the reverse chain runs from there instead of from pure noise 2211.14680 §2.3. Sparse inputs are filled in first and refined over several rounds, each round using the previous output as its guide 2211.14680 §2.3.1. An optional term is the gradient of the equation residual, given to the network as a condition or subtracted in the sampler 2211.14680 §2.3.1 2211.14680 alg. 2. On Kolmogorov flow at Reynolds number 1000, the authors report that the diffusion model has a lower equation loss than a direct-mapping UNet on out-of-distribution inputs, with L2 close to it. For 4x upsampling the values are 0.2178 against 0.5438 for equation loss, and 0.5622 against 0.6048 for L2 2211.14680 tab. 1. Card: Physics-informed diffusion.

Text-guided object inpainting with a mask-precision input

A fine-tuned Stable Diffusion model fills a masked region with the object named by a short label, and the user chooses how closely the object follows the mask 2212.05034 §4.1 §4.2. Training blurs real instance masks to several precision levels, from the exact outline to the bounding box, and adds an output that predicts the exact mask so that background pixels inside a coarse box are kept 2212.05034 §4.2 §4.3. The number of levels and the blur settings are not stated 2212.05034 §4.2. On OpenImages with bounding-box masks, the author-reported local FID is 9.71, against 12.57 for Stable Inpainting 2212.05034 tab. 1. Card: SmartBrush.

Medical segmentation as mask diffusion

MedSegDiff treats a segmentation mask as the signal to diffuse, conditioned on the scan 2211.00611 §2. Two additions carry the method. Dynamic conditional encoding fuses the current noisy mask into the image encoder 2211.00611 §2.1 eq. 3. FF-Parser is a learned frequency filter on the skip features 2211.00611 §2.2 eq. 4. Inference draws 25 samples of 100 steps and fuses them with STAPLE 2211.00611 §3.2. Author-reported Dice for MedSegDiff++ is 87.5 on optic cups, 90.5 on brain tumours and 86.6 on thyroid nodules 2211.00611 tab. 1. Dynamic conditioning adds 2.1, 1.6 and 1.8 Dice points on the three tasks 2211.00611 tab. 2. The released fusion code differs from the equation 2211.00611 §2.1. Card: MedSegDiff.

Mask diffusion trained end to end, without a pretrained backbone

SegDiff is close to MedSegDiff above in one respect: the mask is the signal that gets diffused, and the image conditions the noise predictor 2112.00390 §4. The difference is in how the image enters. It passes through a convolutional encoder with residual-in-residual dense blocks, and its features are summed with the features of the noisy mask before the U-Net encoder 2112.00390 §4 eq. 17. The model is trained from random initialisation, while several of the baselines it is compared with use ImageNet, PASCAL or COCO pretraining 2112.00390 §5. Author-reported mean IoU on the eight Cityscapes object classes is 76.44 with a 15% box expansion, against 73.70 for the best listed baseline 2112.00390 tab. 1. On Vaihingen the F1 is 95.14 against 94.80 2112.00390 tab. 2, and on MoNuSeg the Dice is 81.59 against 79.55 2112.00390 tab. 3. At test time several reverse chains start from different noise, are averaged and then thresholded. The paper uses 30 chains in its main experiments 2112.00390 §4.1. The released code fixes nine chains for every dataset, so the reported tables cannot be checked against the code alone. Inference time grows with the number of chains, and the paper says it is slower than Segformer and STDC 2112.00390 §5. Card: SegDiff.

Reusing one pretrained motion model

Motion diffusion can be extended without retraining the base model. DoubleTake joins prompted intervals by denoising overlapping intervals together, then refining each join 2303.01418 §3.1. It uses a one-second overlap, the handshake, of 20 frames on HumanML3D 2303.01418 §4.1. Author-reported BABEL transition FID is 1.88, against 3.86 for TEACH at a 70-frame margin 2303.01418 tab. 1. ComMDM runs two frozen copies of the base model side by side, with a small coordination block trained on 10 two-person sequences 2303.01418 §3.2 §4.2. DiffusionBlending mixes control-tuned models with equal weights, and fine-tuning zeroes the noise on controlled features 2303.01418 §3.3 §4.3. On a composite wrist-and-trajectory control, the authors report FID 0.22, against 1.18 for inpainting with the base model 2303.01418 tab. 4. Card: Human motion diffusion.

Forecasting with a diffusion sampler per time step

TimeGrad forecasts several series jointly, one time step at a time. An LSTM summarises the history, and a conditional diffusion model draws each next value 2101.12072 §3. Inference uses 100 diffusion steps and 100 sample paths 2101.12072 §4.2. Author-reported CRPS_sum is the lowest of the compared methods on five of six datasets, not on Exchange 2101.12072 tab. 2. Each forecast step runs the denoiser once per diffusion step, so sampling is the slow part 2101.12072 §6. Card: TimeGrad.

Pretrained generator as a data source

CIFAKE does not train a diffusion model. The authors sample 60,000 images from Stable Diffusion v1.4, 50 steps with the Euler Ancestral sampler, rendered at 512 px and reduced to 32 px, matched to the CIFAR-10 classes [arxiv:2303.14126 §3.2] §3.5. They then train a small CNN to separate real from generated images, and report 93.55% accuracy for the final model 2303.14126 tab. 7. The guidance scale and per-image seeds are not stated, so the images cannot be regenerated exactly 2303.14126 §3.5. Only one generator and one source dataset are tested 2303.14126 §5. Card: CIFAKE.

Imagen fine-tuned on ImageNet gives a larger version of the same idea. The authors fine-tune the 64 px base and the 64-to-256 super-resolution model on ImageNet-1K training images 2304.08466 §4.1, then sample for each class a number of images matching the real count, scaled from 1x to 10x, with the class name as the prompt 2304.08466 §4.3. Author-reported FID at 256 px is 1.76 on the train reference 2304.08466 tab. 1. A ResNet-50 trained on real images reaches 76.39% top-1, and 78.17% when 1.2M generated images are added 2304.08466 tab. 3. At 256 px the gain persists to a total of 4.8M images, and the total falls below the real-only baseline from 6.0M 2304.08466 tab. 4. Guidance and sampler settings were chosen on the validation set that is also used for the headline numbers 2304.08466 §4.2. Card: Synthetic data from diffusion models.

Diffusion views for test-time prompt tuning

DiffTPT tunes four CLIP context tokens on each unlabelled test image, one image at a time 2308.06038 §3.1 §4.1. Each image gets 128 views: the original, 64 crop-and-flip views and 63 Stable Diffusion image variations 2308.06038 §4.1. For these variations the usual text condition is replaced by the CLIP image embedding of the test image, because no label is available 2308.06038 §3.3. A cosine-similarity filter keeps the 80% of views most similar to the input, and an entropy filter then keeps the 30% lowest-entropy views 2308.06038 §3.4 §4.1. Author-reported ten-dataset averages are 59.85 against 56.98 for TPT with RN50, and 65.47 against 63.99 with ViT-B/16 2308.06038 tab. 2. These are gains of 2.87 and 1.48 points, below the headline figure (see Contradictions). Card: DiffTPT.

Adversarial prompts against unlearned diffusion models

Safety unlearning changes a text-to-image model so that a concept, a style or an object is no longer drawn. This paper tests five such unlearned models with a white-box attack that prepends three to five optimised tokens to the prompt 2310.11868 §3 §4. The objective is the victim model's own denoising error on a target image that contains the forgotten content, so no auxiliary diffusion or classifier model is needed 2310.11868 §4. On I2P nudity prompts against ESD, attack success rises from 20.42% with no attack to 76.05%, against 69.71% for the P4D baseline 2310.11868 tab. 2. Read it as a stress test of a forgetting fix, and check the card's pitfalls before reusing its numbers. Card: Unlearned diffusion models.

Metrics for generated designs, with a diffusion topology case

This is a review and guide to scoring generative models in engineering design, not a new generator. It sorts its metrics into similarity, novelty and diversity, constraint satisfaction, performance and target achievement, and conditioning 2302.02913 §1 tab. 1. Its argument is that a score for matching the training distribution can rank designs or models wrongly, and it shows this on simple 2-D data 2302.02913 §1 §11.1. The only diffusion model in its experiments is TopoDiff, an external model 2302.02913 §10.3. Two TopoDiff variants are compared on the first 300 conditions of the level-1 test data, with 100 diffusion steps 2302.02913 §10.4. The median compliance error is 1.99% with FEA fields and 9.61% with kernel fields, and the mean volume-fraction error is 1.49% and 1.69% 2302.02913 tab. 7. The mean inference time is 5.87 s and 2.35 s 2302.02913 tab. 7. The diversity row is bolded in a way that favours the kernel-field variant, so the direction of each metric has to be taken from 2302.02913 tab. 1. The repository holds the metric code and the 2-D experiments, not TopoDiff or the case-study data, so the topology numbers cannot be rerun from it. Card: Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design.

Sampling-step theory for low-dimensional data

Under original DDIM and DDPM coefficients, the paper shows that the total variation error falls below a target eps after about k / eps steps, where k is an intrinsic dimension of the data, with no smoothness assumption 2501.12982 §3.1. The assumptions are a polynomially bounded support and a covering-number notion of dimension 2501.12982 §3.1. The bounds extend to learned scores under error conditions, which are stronger for DDIM than for DDPM 2501.12982 §3.2. A one-step lower bound shows the standard coefficients are nearly necessary for this adaptation 2501.12982 §3.3. The paper is theory only, with no experiments 2501.12982 §1. Card: Low-dimensional adaptation.

Likelihood-trained diffusion with a learned noise schedule

The noise schedule is a learned monotone function, trained jointly with a U-Net on the continuous-time variational bound. The loss depends on the schedule only through its endpoints, so the interior is free to be chosen for lower variance 2107.00630 §3.2 §5.1. Author-reported test bits per dimension are 2.65 on CIFAR-10 without augmentation and 3.40 on ImageNet 64. These are bounds, not exact likelihoods 2107.00630 tab. 1. A model trained in continuous time and evaluated with 10 steps gives 7.54 bits per dimension, so many steps are needed 2107.00630 tab. 2. Card: Variational Diffusion Models.

Perception-weighted loss over noise levels

The usual training loss is a simple noise-prediction error summed over noise levels. This variant multiplies it by one extra factor, 1 / (k + SNR(t))^gamma, which lowers the weight of the lowest-noise steps 2204.00227 §3.2 eq. 8. The authors split the steps into a coarse stage, a content stage and a clean-up stage, and argue that the high-SNR steps mainly teach imperceptible detail 2204.00227 §3.1. They use k = 1 and gamma of 0.5 or 1 2204.00227 §3.4. The factor is a closed-form function of the step, so it adds no trainable parameters 2204.00227 §3.2. Author-reported FID-50k falls with this weight: FFHQ from 7.86 to 6.92 at 1000 steps, CUB from 9.60 to 6.95 at 1000 steps, and MetFaces from 44.34 to 36.80 at 250 steps 2204.00227 tab. 1. With 250 uniform steps, a P2 model reaches FID-10k 8.92, against 9.74 for the baseline at 500 steps. That table does not name the dataset 2204.00227 tab. 4. The schedule still matters: the cosine schedule is much worse than the linear one, even with the weight 2204.00227 §3.3 fig. 4. Card: Perception Prioritized Training of Diffusion Models.

Zero terminal SNR schedule with v prediction and rescaled guidance

Common schedules keep some signal at the last step, so the training input there differs from the pure noise given at sampling time 2305.08891 §1 §3.1. The fix rescales the schedule to zero terminal SNR, trains with v prediction, samples from the last timestep, and rescales classifier-free guidance 2305.08891 §3.1 §3.2 §3.3 §3.4. Applied to a fine-tuned Stable Diffusion 2.1-base, with DDIM at 50 steps and 10k COCO captions, the authors report FID 21.66 against 23.76 for the official model [arxiv:2305.08891 tab. 3]. Their comparison also changes the training data, so the gain on the same data is from 22.96 to 21.66 2305.08891 tab. 3. The brightness gain is shown in pictures only 2305.08891 §4.1. Card: Common Diffusion Noise Schedules and Sample Steps are Flawed.

Diffusion prior in CLIP image-embedding space, then a diffusion decoder

A prior maps a caption to a CLIP image embedding, and a GLIDE-style diffusion decoder renders the image from that embedding 2204.06125 §2. The diffusion prior predicts the clean embedding rather than the noise. It gives a zero-shot MS-COCO FID of 10.39, against 10.63 for the autoregressive prior 2204.06125 §2.2 tab. 2. Because the embedding is a compact summary, the same model supports variations and interpolations between images 2204.06125 §3. Card: Hierarchical Text-Conditional Image Generation with CLIP Latents.

Transformer backbone in place of a U-Net

Latent diffusion's U-Net is replaced by a transformer on patches of the VAE latent. Timestep and class enter through adaLN-Zero, which regresses a scale, shift and gate per block and starts each block as the identity [arxiv:2212.09748 §3.2]. Across 12 configurations, FID falls as forward-pass Gflops rise 2212.09748 §5 tab. 4. The reported FID is 2.27 for DiT-XL/2 with classifier-free guidance at 1.50, on class-conditional ImageNet at 256×256 2212.09748 tab. 2. Card: Scalable Diffusion Models with Transformers.

Frozen-model adapters for spatial control

A small adapter reads a control map (sketch, segmentation, keypose, depth or colour) and adds multi-scale features to the encoder of a frozen Stable Diffusion UNet 2302.08453 §3.3. Several adapters can be combined with hand-set weights, with no further training §3.3 §5. On COCO validation, the FID is 16.78 with text plus segmentation, against 24.68 for Stable Diffusion with text only 2302.08453 tab. 1. Injecting into the decoder as well as the encoder was worse in the authors' ablation 2302.08453 §4.4.1 tab. 2. Card: T2I-Adapter.

Larger latent UNet with size and crop conditioning

The base model's UNet is about three times the size of Stable Diffusion 1.x and reads the prompt with two text encoders 2307.01952 §2.1. Original size and crop offsets are embedded as extra conditions, so training images are not discarded or cropped badly §2.2. An optional refiner, run over the first 200 of 1000 noise levels, adds detail §2.5. In the authors' four-way human study the refiner-enhanced model wins 48.44% of votes against 7.91% for SD 1.5 2307.01952 §2.5. Card: SDXL.

Score distillation on unseen views for sparse-view 3D

A diffusion model supplies a score-distillation loss on random camera views that the training photos do not cover 2312.00206 §3.3.1. The loss is switched on after two-thirds of training, once the geometry is stable §4.1. On Mip-NeRF360 with 12 views, the author-reported PSNR is 19.37, against 17.49 for 3D Gaussian Splatting 2312.00206 tab. 1. The ablation gives 1.37 dB from softmax-scaled depth §4.3. Card: SparseGS.

One-step diffusion repair of Gaussian-splatting renders

The scene stays a plain Gaussian splatting fit, and the diffusion model only cleans its renders. A single-step enhancer, derived from SD-Turbo and the public Difix checkpoint, maps each render and its nearest source frame to a cleaner frame 2510.12308 §3.2. It is fine-tuned on about 36k pairs of corrupted and clean frames, with dynamic objects masked out of the loss §3.2. Feeding enhanced views back into the fit improved the visuals but not the metric, so it was left out of the submission §4. Author-reported: the enhancer raises the devset score from 0.423 to 0.462 2510.12308 tab. 1, and the challenge leaderboard gives 0.432, second of 28 §1. The paper does not state the licence of the Difix checkpoint §3.2. Card: Hybrid Gaussian Splatting for Novel Urban View Synthesis.

Distribution matching into one step

A one-step generator is trained so its outputs match the teacher's distribution. Its gradient is the difference of two score estimates, one from the frozen teacher and one from a fake-score denoiser trained alongside, and an LPIPS regression to teacher outputs is added with weight 0.25 2311.18828 §3.2 §3.3. On class-conditional ImageNet 64×64 it reports FID 2.62 in one forward pass, against 2.32 for the EDM teacher with 512 2311.18828 tab. 1. For zero-shot COCO-30k text-to-image it reports FID 11.49 at 0.09 s per image, against 8.78 at 2.59 s for Stable Diffusion v1.5 tab. 3. Card: One-step Diffusion with Distribution Matching Distillation.

Motion diffusion that predicts the clean motion

A transformer encoder predicts the clean motion at each step, not the noise, so constraints such as foot contact can be applied to the prediction 2209.14916 §1 §3. One model handles text, action and unconditional generation through classifier-free guidance §3. On HumanML3D the author-reported FID is 0.544, against 1.067 for T2M 2209.14916 tab. 1. On KIT it is 0.497, against 2.770 for T2M tab. 2. Card: Human Motion Diffusion Model.

Painted views lifted to editable procedural material graphs

A segment-conditioned image model renders one view of a 3D shape with labelled parts 2404.17569 §3.1. Segments are grouped by material §3.2, and each group's procedural material graph is chosen from a library and fitted by differentiable rendering §3.3. The output is a graph that a designer can edit, not a fixed colour texture §3.5. Author-reported FID on chairs is 88.3, against 94.6 for the best baseline 2404.17569 tab. 1. On ABO objects it is 87.3, against 109.8 tab. 1. Timing is 7 minutes per shape, or 33 minutes without grouping §4.4. Card: MaPa.

Per-pixel features from a frozen denoiser for segmentation

Label-efficient segmentation does not draw samples from the model. It reads the decoder activations of a pretrained DDPM. Each labelled image is noised to steps 50, 150 and 250, and five decoder blocks are upsampled and concatenated into an 8448-dimensional vector per pixel 2112.03126 §3.2. An ensemble of ten small MLPs is trained on the pixels of a few annotated images, and a majority vote gives each pixel its label 2112.03126 §3.2 app. C. Author-reported mean IoU is the highest in every column of the main table, for example 49.4 on Bedroom-28 with 40 training images 2112.03126 tab. 2 §4. With 10 training images the same method scores 38.2 on that dataset 2112.03126 tab. 4. The authors expect the method to need a diffusion model trained on the target domain, which they think is hard for complex data such as ImageNet 2112.03126 §5. Card: Label-efficient segmentation.

Frozen denoiser features for dense correspondence

DIFT runs no sampling loop. A real image is noised to a chosen level t, passed once through a pretrained denoiser, and the activations of one upsampling block serve as a per-pixel descriptor. The activations are averaged over several noise draws 2306.03881 §4.2 app. C. Points are matched by nearest neighbour on cosine similarity 2306.03881 §3. The noise level and block are set per task. A small t keeps detail and a large t keeps semantics [arxiv:2306.03881 §4.2] app. C. Author-reported SPair-71k per-image PCK is 52.9 for the Stable Diffusion features, against 33.9 for DINO and 38.4 for OpenCLIP, and 55.3 for the supervised SCorrSAN 2306.03881 tab. 1. On PF-WILLOW the diffusion features score 81.2, against 80.0 for SCorrSAN 2306.03881 tab. 3. On HPatches they trail the weakly supervised CAPS at 3 and 5 pixels [arxiv:2306.03881 tab. 4]. One denoiser pass takes 203 ms per 768 px image on one A6000 GPU 2306.03881 app. B. Caveats: the noise level and block were grid-searched on SPair-71k, which is also the benchmark reported 2306.03881 §5.2. PF-PASCAL is left out of the main results, because 302 of its 506 test images also appear in training 2306.03881 app. D. Card: Emergent Correspondence from Image Diffusion.

CLIP loss on a subdivision mesh, with a diffusion prior as an optional term

The method optimises the control vertices of a subdivision surface, a texture and a normal map from a text prompt, with no 3D training data [arxiv:2203.13333 §3]. The main loss is the negative cosine similarity between CLIP embeddings of random rendered views and the prompt, plus a Laplacian smoothness term 2203.13333 eq. 1 eq. 2. A diffusion prior that samples a CLIP image embedding from the prompt can be added as a weighted term [arxiv:2203.13333 eq. 3] eq. 4. The R-precision table leaves that term out, because the prior was trained on ViT-B/32 embeddings 2203.13333 §4.3. Random backgrounds, cameras and offsets are the paper's answer to texture-only solutions 2203.13333 §3. Author-reported CLIP R-precision is 96.7 with ViT-B/16 for both generation and evaluation, against 93.5 for Dreamfields [arxiv:2203.13333 tab. 1]. The authors report about 50 minutes per shape on one 16 GB GPU 2203.13333 §4.1. Card: CLIP-Mesh.

Score distillation through a volume renderer

Score Jacobian Chaining lifts a 2D diffusion model to 3D without 3D training data 2212.00774 §4. A voxel radiance field is rendered from random cameras, and the denoiser's score is passed back through the renderer by the chain rule 2212.00774 §4. Clean renders lie outside the denoiser's training distribution, so the score is estimated by adding noise to each render and averaging the denoised results 2212.00774 §4.1. An emptiness loss with a two-stage weight suppresses floating density 2212.00774 §4.2. The text-to-3D results use Stable Diffusion v1.5 and are shown only as pictures, with no quantitative metric 2212.00774 §5.2. The authors report about 25 minutes per object on one A6000 2212.00774 app. A3. Card: Score Jacobian Chaining.

Coarse-to-fine score distillation into a textured mesh

Magic3D generates a 3D asset from text in two stages 2211.10440 §4. The first stage optimises a hash-grid neural field with score distillation against a 64x64 base diffusion prior 2211.10440 §4.1. The field is then turned into a tetrahedral mesh, and a 512x512 latent diffusion prior refines its geometry and texture 2211.10440 §4.2 §4.3. The authors report about 40 minutes on 8 A100 GPUs, and 61.7% of user-study raters prefer its videos to DreamFusion's on "more realistic and detailed" 2211.10440 §5 tab. 1. That comparison uses DreamFusion's published videos, and the DreamFusion runtime is quoted from that paper, not measured 2211.10440 §1. The text guidance weight and the angular regularisation weight are not stated [arxiv:2211.10440 §3] §4.3. Card: Magic3D.

Single-view mesh from a multi-view diffusion model

One-2-3-45 does not optimise a shape for each object. Zero123, a fine-tuned Stable Diffusion model, renders 32 posed views around the input 2306.16928 §4.1. The input's elevation is estimated from four of those views by feature matching 2306.16928 §3.4. A feed-forward SDF cost-volume network built on SparseNeuS fuses the views into a mesh 2306.16928 §3.3. Author-reported F-score averages 83.5 over GSO and Objaverse, the best in its table, and CLIP similarity is second to Shap-E 2306.16928 tab. 1. The runtime is about 45 s on an A100, against about 15 minutes for the Zero123 with Stable Diffusion baseline [arxiv:2306.16928 tab. 1]. The authors' ablation finds that training only on ground-truth views fails at inference 2306.16928 §4.3. Card: One-2-3-45.

Point-voxel diffusion for point clouds

Point-voxel diffusion runs DDPM-style denoising on clouds of 2048 points, with a point-voxel CNN as the denoiser 2104.03670 §3. One network does unconditional generation and completion. In completion, the observed points are held fixed and only the missing points are denoised 2104.03670 §3.2. Author-reported 1-NN is the best of the compared generators on all six generation cells 2104.03670 tab. 1. Sampling uses 1000 steps, and the authors state that 100 steps are not enough 2104.03670 app. H.2. In completion from single depth views, the method is best on EMD in all three categories and worse than the baselines on CD in all three [arxiv:2104.03670 tab. 2]. Card: Point-voxel diffusion.

Equivariant diffusion over molecular coordinates

GeoDiff generates 3D atom coordinates for a molecular graph [arxiv:2203.02923 §4]. Its reverse kernels are SE(3)-equivariant, and its starting density has zero centre of mass, so the sampled density is invariant to rotation and translation 2203.02923 §4.2. Training reduces to a weighted noise-prediction loss 2203.02923 §4.3. Author-reported mean COV-R on GEOM-Drugs is 89.13%, against 67.16% for GeoMol 2203.02923 tab. 1. The default sampler runs 5000 steps 2203.02923 tab. 4, and a 1000-step variant is weaker 2203.02923 tab. 6. Card: GeoDiff.

Diffusion prior for real-world super-resolution

StableSR keeps a Stable Diffusion 2.1-base model frozen. It trains a time-aware encoder whose features modulate the U-Net through scale-and-shift layers 2305.07015 §3.1. An optional fidelity module, weighted by a user value w, trades realism for faithfulness 2305.07015 §3.2. Large inputs are processed as overlapping 64×64 latent patches, blended with Gaussian weights at each step 2305.07015 §3.3. On DIV2K-Valid the author-reported FID is the best in the table at 24.44, but PSNR is 23.26, below several baselines 2305.07015 tab. 1. The 200-step sampler takes 15.16 s per 512 px output on one V100, and a 4-step SD-Turbo variant takes 0.83 s 2305.07015 tab. 5. Card: StableSR.

Diffusion over a compact restoration prior

DiffIR does not diffuse pixels. A pretrained encoder turns the clean image into a 256-value restoration prior, and a diffusion model learns to predict that prior from the damaged input 2303.09472 §4.1 §4.2. Training has two stages: the encoder first, then the diffusion model and the restorer together 2303.09472 §4.1 §4.2. Sampling uses four steps [arxiv:2303.09472 §5.1], and a transformer restorer produces the image from the predicted prior 2303.09472 §4.1. Author-reported inpainting FID with wide masks is 5.14 on CelebA-HQ and 1.98 on Places, against 5.70 and 2.25 for LaMa, at about the same parameter count 2303.09472 tab. 1. On x4 super-resolution over DIV2K100 the PSNR is 29.13, against 28.62 for BebyGAN and 28.58 for SRdiff 2303.09472 tab. 2. Motion deblurring on GoPro reaches 33.20 PSNR, against 32.92 for Restormer 2303.09472 tab. 3. Training the two stages jointly gives CelebA-HQ FID 5.14, against 5.98 with the traditional scheme 2303.09472 tab. 4. Caveats: the efficiency claim against RePaint is shown in a figure and is not derivable from the tables [arxiv:2303.09472 fig. 1]. The released code uses a sqrt-linear noise schedule and predicts the clean prior, not the noise the paper's equations use 2303.09472 §5.1 eq. 11. Card: DiffIR.

Semantic tags and a low-resolution start for real-photo upscaling

A tagger is fine-tuned on degraded inputs, so its tags and embeddings on a damaged photo match what a frozen tagger gives on the clean image 2311.16518 §3.2 eq. 1. The tag text goes through the text encoder. The tagger's embeddings enter through representation cross-attention layers in a ControlNet-style branch on Stable Diffusion 2-base 2311.16518 §3.3 eq. 2. Sampling starts from the low-resolution latent noised to the training schedule, not from pure noise. The authors argue that pure-noise starts let the model treat degradation as content to enhance, and they say the low-resolution start removes false sky texture 2311.16518 §3.4 fig. 3. Author-reported: best FID, CLIPIQA and MUSIQ where the table gives those metrics, and PSNR and SSIM below the best baseline on the three paired sets 2311.16518 tab. 2. In a semantic check on degraded COCO images, a COCO-trained detector and segmenter score 30.0 panoptic PQ, against 27.2 for DiffBIR [arxiv:2311.16518 §4.3]. Card: SeeSR.

Camera-controlled video with ray encodings

UCPE gives each video token its own viewing ray in a local frame, so lens distortion and field of view enter the attention layers 2512.07237 §3.2. A latitude-and-up map adds absolute pitch and roll 2512.07237 §3.2. The encoding enters a Wan2.1 T2V 1.3B model through a parallel attention adapter whose output projection starts at zero 2512.07237 §3.3 app. D. The adapter has 35.6M trainable parameters 2512.07237 tab. 1. On a synthetic lens dataset the author-reported rotation error is 4.12° against 9.23° for ReCamMaster 2512.07237 tab. 1. Zero-shot on RealEstate10K it is 0.56° against 1.10° 2512.07237 tab. 2. Card: UCPE.

Diffusion policy with a visuo-tactile encoder

Touch in the Wild first pretrains a visuo-tactile encoder by reconstructing masked tactile images, with a CLIP ViT-B/16 on the camera image and cross-attention between the two embeddings 2507.15062 §4.2. The encoder then feeds a diffusion policy, which samples an action chunk in 16 denoising steps 2507.15062 §4.3 tab. 5. The authors report 0.85 whole-task success on the test-tube task with pretraining, 0.70 without it and 0.25 with vision only 2507.15062 tab. 1. Each cell is 20 trials, so one trial moves a rate by 0.05. Card: Touch in the Wild.

Diffusion action head on a pretrained robot transformer

Octo tokenizes the language or goal image, the camera views and one readout slot per time step, and runs one transformer over them 2405.12213 §3.1. The readout embedding goes to a small diffusion head, which denoises an action chunk over 20 steps on a cosine schedule 2405.12213 app. D. The transformer runs once per prediction, and only the head loops 2405.12213 §3.3. The paper reports that this head beats a continuous MSE head and a discretized action head, at 83% against 35% and 18% aggregate success on the WidowX ablation 2405.12213 tab. 2. The ablations use Octo-Small and 40 trials, so the gap is measured on one small setup.

Finetuning updates all parameters with about 100 target trajectories and 50k steps, and a new sensor or action space gets a new small input encoder or head 2405.12213 §3.3 §3.1. The authors report 72% average success over six real-robot setups, against 20% for training from scratch, with 20 trials per setup 2405.12213 tab. 1. The paper says the pretrained model was trained for about 14 hours on a TPU v4-128 for the Base size, and each finetune took about 5 hours on one A5000 2405.12213 §3.4. Card: Octo.

Diffusion expert on a vision-language backbone, trained in stages

DexVLA puts a vision-language model (Qwen2-VL-2B) in front of a separate diffusion transformer of about 1B parameters, the action expert 2502.05855 §3.1 app. C.3. The expert is conditioned on the VLM through FiLM layers, and the same VLM also writes sub-step text as intermediate output 2502.05855 §3.1 §3.2. The loss is the diffusion loss on actions plus the next-token loss on that text, with weight 1 in all runs 2502.05855 §3.1. The paper does not state the noise schedule, the prediction target or the loss form 2502.05855 §3.1.

Training has three stages: the expert alone on about 100 hours of cross-embodiment data, then the whole model on one embodiment, then task-specific finetuning, which the authors skip for simpler tasks 2502.05855 §3.2 app. C.4. In the paper's ablation, skipping stage 1 gives zero shirt-folding success, and stage 3 is needed for laundry folding, where the score goes from 0.0 to 0.4 2502.05855 §4.4. The expert size also matters on shirt folding: 0.17 for a 93M U-Net, 0.63 for a 410M diffusion transformer and 0.92 for the 1B one 2502.05855 §4.4. These are the authors' results on their own robots, with 10 trials per task 2502.05855 §4. Card: DexVLA. The official code is licensed non-commercially, so do not copy it, and see the card's Paper vs code section for the places where it departs from the paper.

Diffusion policy co-trained with a large diverse dataset

Co-training fills each training batch half from a task's own demonstrations and half from a large external dataset 2403.12945 §5.1 app. F.2. The policy is a diffusion policy. Two external views at 128x128 go through an ImageNet-pretrained ResNet-50, a frozen DistilBERT embedding of the instruction joins the robot state, and a U-Net head denoises 16 action steps, of which eight run open loop 2403.12945 §5.1 app. F.1. Author-reported over six tasks, the co-trained policy beats the next-best method by 22 points in-distribution and 17 points out-of-distribution, both averaged over tasks 2403.12945 §5.2. Each task has ten A/B rollouts per method, so the per-task differences are noisy §5.2. The paper's sampler and schedule do not all match the released configs, so check the card's Paper vs code section before reproducing the setup. Card: DROID.

Latent diffusion over grasp latents, with a per-task denoiser

Grasps are compressed by a conditional VAE, and a DDPM is trained on the VAE latents rather than on poses. The VAE is trained first with the ELBO, and its KL weight is annealed from 1e-7 to 0.1 over the first half of training 2312.11243 §3.1 2312.11243 eq. 2. The VAE is then frozen, and the DDPM learns the latent with the noise as the target 2312.11243 §3.2 2312.11243 eq. 6. The forward process has 1000 steps with a linear schedule 2312.11243 §3.3. DDIM with 100 steps can replace ancestral sampling without retraining 2312.11243 §4.3. On the simulated 63C objects with full point clouds, the median success is 78% [arxiv:2312.11243 §4.1]. The sampler table gives 0.792 for DDPM with 1000 steps, at 7.39 s per 100 grasps, and 0.756 for DDIM with 100 steps, at 0.75 s 2312.11243 tab. 1. A new task only retrains the small denoiser, according to the authors 2312.11243 §4.2. Card: GraspLDM.

Latent diffusion for hand grasps, with a frozen-prior adaptation step

A hand autoencoder gives a 768-d latent, which the denoiser treats as 256 tokens of 3 values. The denoiser is conditioned on a PointNet embedding of 3000 object surface points 2411.14786 §3.2 2411.14786 eq. 8. Sampling uses DDIM 2411.14786 §3.4. Training happens in two steps. After the autoencoder and the diffusion model are trained, both are frozen, and a small MLP predicts an offset to the latent 2411.14786 §3.3 2411.14786 eq. 9. The authors report that training the diffusion model directly on physical constraints oscillated and did not converge 2411.14786 §3.3. On out-of-domain HO-3D, the penetration volume is 5.23 against 7.37 for GraspTTA, at 0.14 s per grasp 2411.14786 tab. 2. The step count N is not stated in the paper 2411.14786 §3.4. Card: FastGrasp.

Object-then-scene diffusion for LiDAR

Two diffusion models run in sequence. An object model denoises object point clouds of 1024 points, voxelised at size 32, and a scene model denoises range images of height 64 on KITTI-360 2412.17226 app. A. The object condition is a CLIP text vector with a Fourier-embedded box [arxiv:2412.17226 eq. 1-3]. The scene model uses ControlNet-style zero-convolution features from the object range image 2412.17226 eq. 7. The authors' reason is that foreground points are usually under 10% of a scene's points, so scene-level models are biased to background 2412.17226 §3.2. Reported KITTI-360 results are FPD 7.60 against 25.1 for UltraLiDAR, and JSD 0.42 against 0.71 2412.17226 tab. 1. Sampling steps and guidance are not stated 2412.17226 §3.2. Card: OLiDM.

Diffusion over action chunks, scaled to a billion parameters

The policy is a DDPM over action chunks, with adaptive layer norm in place of cross-attention, and with the causal mask removed from self-attention 2409.14411 §3.2 2409.14411 fig. 2. Depth, width and heads are scaled together, from 10M to 1B parameters 2409.14411 tab. 1. On the Franka tasks, the average success is 87.50 for the largest model against 51.25 for DP-T 2409.14411 tab. 2. On bimanual UR5, it is 98.33 against 23.33 2409.14411 tab. 3. Removing the causal mask from ScaleDP-L drops the average from 90.00 to 66.66 2409.14411 tab. 4. The optimizer, learning rate, step count and chunk length are not stated [arxiv:2409.14411 §3]. Card: ScaleDP.

Policy-gradient fine-tuning of the denoising chain

A diffusion policy is first trained by behaviour cloning, then fine-tuned with PPO. Each denoising step is treated as a Gaussian policy with an analytic likelihood, and the inner chain is nested inside the environment MDP 2409.00588 §4.1 2409.00588 eq. 6. Reward arrives only at the last denoising step 2409.00588 eq. 4. The paper's best practices fine-tune only the last K' steps, or use a few DDIM steps with stochastic training and deterministic evaluation 2409.00588 §4.3 2409.00588 eq. 12. Reported results include more than 90% on Robomimic Transport from state 2409.00588 §5.1, and 80% (16 of 20) zero-shot on FurnitureBench One-Leg hardware 2409.00588 §5.4. Fine-tuning the Gaussian head without BC regularisation gave 88% in simulation and 0% on hardware, and adding the BC term gave 53% and 50% 2409.00588 §5.4. Training from scratch with 10 denoising steps was about six times slower than the Gaussian policy 2409.00588 app. D. Card: DPPO.

Diffusion over action chunks for dexterous manipulation with a VLM planner

A fine-tuned VisualGLM-6B plans the task, and a transformer action-diffusion policy (GCAD) predicts a chunk of actions at each denoising step 2409.05493 §3.1 2409.05493 §3.2. The policy is conditioned on the observation and on an embedded return-to-go 2409.05493 §3.2. It executes two actions before replanning 2409.05493 app. A.5. In simulation, the average success is 0.70 for this system and 0.23 for DP-T 2409.05493 tab. 1. On a real UR5e, it is 0.65 (26 of 40) 2409.05493 tab. 2. The paper says the transformer backbone beats the CNN diffusion backbone, and that BC and decision transformer fail to learn the task 2409.05493 §4.2. Card: DexDiff.

Diffusion as a baseline on a large motion dataset

In the piano benchmark, diffusion policies (a U-Net, DP-U, and a transformer, DP-T) are baselines trained with DDPM, and no diffusion design is proposed 2408.11048 app. C.1. With 12 songs, the in-distribution average F1 is 0.539 for DP-U, 0.357 for DP-T and 0.529 for BC-MLP 2408.11048 tab. 2. On 150 songs, evaluated out of distribution, it is 0.256 for DP-U and 0.316 for DP-T 2408.11048 tab. 2. The paper does not state the step count for the transformer baseline 2408.11048 app. C.1. Card: RP1M.

Diffusion policy for humanoid whole-body control

An action-sequence diffusion policy, following DiffuseLoco, controls a simulated Unitree H1 humanoid 2411.01349 §4. Its training data is generated by a source RL policy under eight domain-randomization setups, with 0.5M, 2M and 8M transitions 2411.01349 §3 2411.01349 §5. The best policy, trained on 8M transitions from all setups, reaches success 1.0 in both the non-randomized and the randomized evaluation 2411.01349 tab. 3 2411.01349 tab. 5. The card states that terrain and perturbation randomization give the most robust policies, and that walking needs far more data than manipulation 2411.01349 §1 2411.01349 §6. Hyperparameters are deferred to DiffuseLoco and the prediction horizon is not stated 2411.01349 §4. Card: The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control.

Hierarchical diffusion with a contact guider

Two conditional diffusion models run at each control step. The Guider denoises a contact target, which is each finger's position in the object frame plus a touch flag 2411.12982 §3.2. The Actor denoises an action sequence, predicting 16 steps and executing 8 in open loop 2411.12982 tab. 8 2411.12982 fig. 2. The Guider uses 100 denoising steps and the Actor 10 2411.12982 tab. 9. The Actor loss adds a Q-value term with weight 0.001, and the paper uses a one-step denoising form for it [arxiv:2411.12982 eq. 9] 2411.12982 tab. 9 2411.12982 §4.4. The authors report a 20.8% average success gain over Diffusion Policy across six tasks 2411.12982 §6. On Move-T stage 2, the success is 0.60 against 0.30 for Diffusion Policy 2411.12982 tab. 5. The authors state that Diffusion Policy performs worse on contact-rich, multi-contact tasks than on single-contact ones 2411.12982 §1. The 20.8% figure is not derived in the text, so it is quoted as reported. Card: HDP.

Trajectory diffusion for visual navigation

A 1D U-Net denoises a sequence of relative waypoints, conditioned on a transformer output that combines a depth-CNN, a target-position encoder and a goal encoder 2412.20226 §II-C 2412.20226 §II-D. Training uses 100 denoising iterations, and deployment uses 5 2412.20226 §III-A. The horizon is 32 with 16 waypoints executed, and the goal is masked with probability 0.7 during training 2412.20226 §III-A. On a Unitree Go2, with 25 trials per difficulty level, the hard-level success is 84% for this policy, against 20% for ViNT with a relative-distance goal and 12% for BC-ConvMLP 2412.20226 tab. I. Only one robot was tested, so transfer to other embodiments is not shown 2412.20226 tab. I. Card: ViDEN.

Goal-masked diffusion policy for exploration and goal reaching

One Transformer policy serves both jobs. During training the goal token is masked for about half of the samples. Masking gives undirected exploration at test time, and an unmasked goal photo gives goal-reaching 2310.07896 §IV-A. The action head is a diffusion model over short sequences of 2-D moves, denoised in 10 steps and conditioned on the observation context 2310.07896 §IV-B §IV-C. The training loss adds a temporal-distance term with weight 1e-4 2310.07896 §IV-C eq. 2. The sampling update in the paper leaves its coefficients unspecified 2310.07896 eq. 1. Real-robot tests on a LoCoBot in unseen indoor and outdoor places give 98% exploration success with 19M parameters and 0.2 collisions, against 77% and 1.7 collisions for Subgoal Diffusion with 335M parameters 2310.07896 tab. 1. Navigation success is 90%, equal to Random Subgoals and Subgoal Diffusion in the same table tab. 1. The unified model reaches 98% undirected and 92% goal-conditioned, against 92% for the goal-conditioned ViNT policy 2310.07896 tab. 2. Trial counts are not stated 2310.07896 §V-A. Card: NoMaD.

Diffusion policy with audio from a contact microphone

A diffusion policy head is conditioned on a fused vision and audio feature, and on the last two end-effector poses, at every denoising step [arxiv:2406.19464 §3.2]. The microphone is mounted on the gripper finger [arxiv:2406.19464 §3.1]. Training adds two background overlays to the audio, each with probability 0.5 2406.19464 §3.2. On wiping, the success is 85% with audio and 40% with vision alone 2406.19464 §4.2. On flipping, it is 90% against 25% 2406.19464 §4.1. The authors name objects that make little sound, quasi-static tasks and robot motor noise as limits [arxiv:2406.19464 §4.5]. The step count is not stated in the paper 2406.19464 §3.2. Card: ManiWAV.

Diffusion policy for physics-based character control

A diffusion policy over action sequences is trained by behaviour cloning on states paired with the clean expert action for the state reached 2406.00960 §3.2 2406.00960 §3.3. On the perturbation task, the out-of-distribution success is 96.3 for this policy, against 91.3 for the best C-VAE and 81.0 for an MLP 2406.00960 tab. 3. Tracking on the AMASS test set gives 97.1 in the table 2406.00960 tab. 2. The authors name inference cost as the main limit, since it is K times the cost of an MLP 2406.00960 §6. Card: PDP.

When to use which

Step count is model-specific. In MLD's table, DDIM-50 gives FID 0.473 and DDPM-1000 gives 0.568 2212.04048 tab. 10. For MDM, the same two samplers give 7.334 and 0.544 in that table. Measure the step count on your own model instead of copying a sampler setting. The theory in 2501.12982 §3.1 gives a scaling law in the target error, not a number of steps to use.

Guidance also trades metrics against each other. MLD's sweep has its best FID at scale 2.5 (0.350), while R-precision rises from 0.648 at 1.5 to 0.772 at 7.5 2212.04048 tab. 11. Choose the scale against the metric you need. The guidance mechanism is on Classifier-free guidance. Scales differ by model: the Imagen base uses 1.25 2304.08466 §4.2, Composer uses 3.0 for its prior and base models 2302.09778 §3.5. Neither transfers to another model without a sweep.

Budget note: the GLIGEN larger runs use 500k iterations at batch 32 for Stable Diffusion, and 400k iterations for the LDM run 2301.07093 app. A. Training compute varies widely across the cards: Grad-TTS trains on one RTX 2080 Ti 2105.06337 §4, the MLD VAE and denoiser take about 8 h and 4 h on eight V100s 2212.04048 §4.2, the eDiff-I base uses 256 A100s 2211.01324 §5, and the Imagen fine-tune uses 256 TPU-v4 chips for 210k and 490k steps 2304.08466 §4.1. Inference cost of the GLIGEN grounding tokens is not stated 2301.07093 app. A.

Implementation sketch

Common core: noise a clean sample at a random level and regress the noise. GLIGEN, MLD and MID use this objective, with their own noise schedules 2301.07093 §4.2 2212.04048 §3.2 2203.13777 §3.2. The comments cite the card section that each step comes from, with the arXiv id short form.

# One training step, noise-prediction objective  [2301.07093 §4.2] [2203.13777 eq. 10]
# Input:  clean samples x0 (B, ...), conditions c (may be empty)
# Output: scalar loss

def training_step(x0, c, model):
    # Draw one noise level per sample; the schedule is variant-specific
    t = sample_timestep(batch=B)
    eps = sample_normal_like(x0)
    # Corrupt x0 to level t with the variant's forward process  [2212.04048 §3.2]
    x_t = add_noise(x0, eps, t)
    # Drop the condition for some samples, so the model also learns the
    # unconditional branch used by guidance  [2212.04048 §3.3]
    c = drop_with_probability(c, p=0.1)
    # Predict the noise from the noisy sample, the level and the condition
    eps_hat = model(x_t, t, c)
    return mean((eps - eps_hat) ** 2)

# Sampling: start from noise and apply the learned step  [2212.04048 §4.2]
# Input:  shape of the working space, conditions c, number of steps
# Output: one sample in the working space

def sample(shape, c, steps, model):
    # Start from Gaussian noise in the working space (latents or data)
    x = sample_normal(shape)
    for t in timesteps(steps):
        # Noise estimate under the condition  [2212.04048 §3.3]
        eps_hat = model(x, t, c)
        # Variant-specific update: DDIM, DDPM posterior or an ODE step
        x = update(x, eps_hat, t)
    return x

Where variants differ: the prediction target (noise in GLIGEN, MLD and MID; score in Grad-TTS; clean sample in Diffusion-LM, in eDiff-I's preconditioned denoiser and in MDLM's per-token rule 2406.07524 §3.2.3; velocity in W.A.L.T 2312.06662 §3), the sampler (DDIM with eta 0 in MLD, Euler steps in Grad-TTS, DEIS in eDiff-I), and how conditions enter (see Variants).

For the frozen-backbone variant, the added layer attends over visual and grounding tokens, and the gate starts at zero. Its scheduled weight switches the grounding off after the first fraction of steps 2301.07093 §4.2 eq. 8 eq. 10.

# Grounding token for one entity  [2301.07093 §4.1] [app. A]
# Input:  text_feature (768,)  CLIP text feature of the phrase
#         box (4,)             (x_min, y_min, x_max, y_max)
# Output: grounding_token (768,)

def grounding_token(text_feature, box):
    # Fourier-embed each of the four coordinates  [app. A]
    box_feature = fourier_embedding(box)          # (64,)
    # Concatenate the two inputs and run the MLP  [eq. 5] [app. A]
    joined = concat(text_feature, box_feature)    # (832,)
    return mlp(joined)                            # (768,)

# Gated self-attention, inserted between the frozen self-attention and
# cross-attention of each transformer block  [2301.07093 §4.2] [eq. 6-8]
# Input:  visual (B, M, d), grounding (B, N, 768), gamma (learned, init 0)
# Output: visual (B, M, d)

def gated_self_attention(visual, grounding, gamma, beta):
    num_visual = visual.shape[1]                    # M
    # Project grounding tokens to the visual width d  [app. A]
    grounding = linear_projection(grounding)        # (B, N, d)
    # Joint self-attention over visual and grounding tokens  [eq. 8]
    joint = concat(visual, grounding, axis=1)       # (B, M + N, d)
    attended = self_attention(joint)[:, :num_visual]
    # Gated residual; tanh(0) = 0 at initialization  [eq. 8]
    return visual + beta * tanh(gamma) * attended

# Scheduled sampling at inference  [2301.07093 §4.2] [eq. 10] [§5.4]
# tau in [0, 1]: fraction of denoising steps that use grounding

def grounding_weight(step, total_steps, tau):
    # beta = 1 for the first tau * T steps, then 0  [eq. 10]
    if step <= tau * total_steps:
        return 1.0
    return 0.0

Contradictions

• Prediction target. Diffusion-LM predicts the clean embedding and reports that predicting the noise collapsed as the embedding dimension grew 2205.14217 §7.4 fig. 4. The reported evidence is one figure, with no table. GLIGEN, MLD and MID train the noise prediction and report results 2301.07093 eq. 1 2212.04048 eq. 2 2203.13777 eq. 10. Grad-TTS predicts a score under its own weighting 2105.06337 eq. 17. MDLM predicts clean tokens in a masked setting 2406.07524 §3.2.3, which is a different space from the continuous cases above. The papers work in different spaces, so none tests the other's claim. Both sides are stated here, not resolved.

• Clean target versus noise target. MDM predicts the clean motion 2209.14916 §3, and the unCLIP diffusion prior predicts the clean embedding 2204.06125 §2.2. DiT predicts noise with a covariance 2212.09748 §3.1, and VDM predicts noise under its learned schedule 2107.00630 §3.4. Point-voxel diffusion 2104.03670 eq. 6 and GeoDiff 2203.02923 eq. 9 also predict noise. The cards work on different data and report no head-to-head comparison, so neither choice is shown to be better.

• Equivalent targets versus practice. The tutorial states that predicting the clean sample, the noise or the score gives the same objective up to time-dependent weights, and that predicting the noise has been found to work better in practice, citing other work 2208.11970 §3. It does not test this. The cards cited here train noise prediction [arxiv:2212.04048 eq. 2] 2203.13777 eq. 10 and report results for their own tasks. This page found no head-to-head comparison of the two targets, so the practical preference is stated here, not measured.

• Diffusion policy against simple baselines. On the piano benchmark, the transformer diffusion baseline is below BC-MLP with 12 songs in distribution, 0.357 against 0.529 2408.11048 tab. 2. On Franka tasks, the transformer diffusion baseline is below the larger ScaleDP model, 51.25 against 87.50 2409.14411 tab. 2. The two papers use different tasks, data and baselines, and neither tests the other's setup, so the ranking is not resolved here.

• Human preference versus FID. SDXL has the worst FID of the three models it compares, but its human study favours it [arxiv:2307.01952 app. On FID Assessment]. The DiT and unCLIP cards report FID as their main metric 2212.09748 tab. 2 2204.06125 tab. 2. ImageReward makes the same argument: on its six-model ranking of 100 prompts, FID has a Spearman correlation of 0.09 with the human order, and CLIP 0.60 [arxiv:2304.05977 tab. 1]. That table uses the authors as raters and six models, so it is small. Both views are stated here, not resolved.

• Teacher gap for distillation. The one-step DMD student trails SD v1.5 by 2.71 FID at guidance 3, with the teacher's number taken from the paper 2311.18828 tab. 3. At guidance 8, with the teacher rerun by the authors, the gap is 1.48 2311.18828 tab. 4. The two rows use different guidance scales and different teacher numbers.

• Cross-attention versus concatenation. MLD reports that concatenating its condition beats cross-attention, with FID 0.473 against 1.922 2212.04048 tab. 5. eDiff-I conditions its text through cross-attention 2211.01324 §4.2, and the card here has no comparison against concatenation. GLIGEN finds gated cross-attention worse than gated self-attention for its boxes, with AP 16.6 against 21.7 [arxiv:2301.07093 app. B]. The results are for different conditions and models.

• Step count. MLD's DDIM-50 beats its DDPM-1000 on FID, while MDM's DDIM-50 is far worse than its DDPM-1000 2212.04048 tab. 10. Grad-TTS finds that 10 steps are near saturation 2105.06337 tab. 2. Diffusion-LM uses 200 steps for control and 2000 for ROCStories, because 200 hurt quality there 2205.14217 §6.1. These are model-specific results, not a conflict over one setting.

• Autoregressive gap for discrete diffusion. MDLM argues that much of the gap to autoregressive models came from training recipes, not from the framework 2406.07524 §1 §6. Its own table gives a bound of 23.00 against 20.86 for a retrained autoregressive model 2406.07524 tab. 1. Diffusion-LM reports its E2E bound at 2.28, against 1.77 for an autoregressive Transformer 2205.14217 §7. The two comparisons use different datasets and bound types, so the gaps are not comparable.

• Time-series baselines. TimeGrad reports the lowest CRPS_sum on five of six datasets. On Exchange its value is 0.006, against 0.005 for three baselines 2101.12072 tab. 2. The Diffusion Forcing card reports 0.003 for its own method on Exchange 2407.01392 §4.5. It also lists a diffusion baseline, ScoreGrad, with CRPS_sum no higher than TimeGrad's on every column it reports 2407.01392 §4.5. The TimeGrad card does not include ScoreGrad, so its "best on five of six" claim is relative to a smaller baseline set. The two papers use different baselines and seeds, so their rankings cannot be merged.

• Headline gain of DiffTPT. The abstract and conclusion state an average gain of 5.13% over TPT. Section 4.2 gives 5.1% for RN50 and 2.3% for ViT-B/16 on the cross-dataset scenario 2308.06038 §4.2. The printed averages in that scenario give 2.87 and 1.48 points [arxiv:2308.06038 tab. 2]. The paper does not say how the headline is computed, so both are stated here.

• Which algorithm wins in video RL. On the Gemini "All" column RWR on AI feedback (56.41%) is above DPO (56.17%), but on human "All" DPO (34.06%) is above RWR (31.09%) 2412.02617 tab. 2. The authors also say DPO generalises better to held-out prompts 2412.02617 §4.2 §4.3, while RWR keeps improving over two rounds and DPO saturates 2412.02617 §4.3. The ranking depends on the evaluator and the number of rounds.

• CogACT on SimplerEnv WidowX. Three cards report the same CogACT policy on this benchmark, with different averages: 47.9 in STORM 2512.18477 tab. 1, 51.3 in VideoVLA 2512.06963 tab. 1 and 51.8 in FPC-VLA 2509.04018 tab. 3. The per-task values differ as well. None of the cards says whether the baseline was re-run or copied, so the gain each paper reports over CogACT is not comparable across papers. Both sides are stated here, not resolved.

• Talking-face sync scores. EMO reports that its SyncNet score is below Wav2Lip and SadTalker 2402.17485 tab. 1. VASA-1 reports the best lip sync against SadTalker and two other baselines 2404.10667 §4. The two papers use different benchmarks, so their sync numbers cannot be merged.

• FID for StableSR on DIV2K. The StableSR entry above reports FID 24.44 on DIV2K-Valid, the best value in its table 2305.07015 tab. 1. SeeSR's table reports StableSR at 36.57 on its own DIV2K-Val set, where SeeSR scores 31.93 2311.16518 tab. 2. SeeSR builds that set with its own degradation pipeline 2311.16518 §4.1. The StableSR entry does not state its test degradation, so the two StableSR numbers cannot be checked against each other. Both are stated here, not resolved.

Within a card, the released code departs from the paper in places, for example the Grad-TTS sampling temperature, the MID diffused variable and the Denoising Student's teacher start index. Those are on the cards and are not repeated here.

Papers

• Variational Diffusion Models: a diffusion model with a learned noise schedule, trained on the variational bound.

• Understanding Diffusion Models: A Unified Perspective: a tutorial derivation that links the ELBO, the three training targets and guidance, with no experiments.

• Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models: a retrieval detector that counts generated images copying training images, for small DDPMs and Stable Diffusion.

• Hierarchical Text-Conditional Image Generation with CLIP Latents: a CLIP-embedding prior with a diffusion image decoder, for text-to-image and image variations.

• Scalable Diffusion Models with Transformers: a transformer backbone on latent patches, conditioned by adaLN-Zero, with FID scaling in Gflops.

• T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models: small adapters that add control-map features to a frozen Stable Diffusion encoder.

• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis: a larger latent UNet with size and crop conditioning, and an optional refiner.

• SparseGS: Sparse View Synthesis using 3D Gaussian Splatting: Gaussian splatting from few views, with score distillation on unseen camera poses.

• Hybrid Gaussian Splatting for Novel Urban View Synthesis: Gaussian splatting on COLMAP-densified points, with a one-step diffusion enhancer that cleans its renders.

• One-step Diffusion with Distribution Matching Distillation: a one-step generator trained to match a teacher's output distribution, with a regression term.

• Human Motion Diffusion Model: a transformer diffusion model for human motion that predicts the clean motion.

• MaPa: Text-driven Photorealistic Material Painting for 3D Shapes: painted part views fitted to editable procedural material graphs for 3D shapes.

• Understanding World or Predicting Future? A Comprehensive Survey of World Models: a survey of world models, including video generators as simulators. It reports no experiments.

• GLIGEN: Open-Set Grounded Text-to-Image Generation: adds box grounding to a frozen latent text-to-image diffusion model with gated attention layers.

• Executing your Commands via Motion Diffusion in Latent Space: latent diffusion for text- and action-conditioned human motion, with a single-token transformer VAE latent.

• eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers: a text-to-image cascade whose base is an ensemble of noise-level experts, with paint-with-words control.

• Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech: a score-based diffusion decoder for mel-spectrograms, where the number of Euler steps trades speed for quality.

• Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion: reverse diffusion over future pedestrian positions, sampled to give several plausible futures.

• Diffusion-LM Improves Controllable Text Generation: diffusion over word embeddings for text, with classifier-guided control on the latents.

• Diffusion models as plug-and-play priors: an independently trained diffusion model as a prior, with a differentiable constraint for search over solutions.

• MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model: mask diffusion conditioned on the scan, with image-gated features and a learned Fourier filter.

• Robust Compressed Sensing MRI with Deep Generative Priors: posterior sampling for MRI with a score-based prior trained on brain slices.

• Learning Universal Policies via Text-Guided Video Generation: a text-conditioned video diffusion planner with an inverse-dynamics policy, executed open-loop.

• Photorealistic Video Generation with Diffusion Models: one transformer diffusion model for images and video, in a shared causal-autoencoder latent.

• One-2-3-45++: a multi-view tile from a fine-tuned image diffusion model, lifted to 3D by two volume diffusion stages.

• SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation: latent diffusion over 3D shapes with per-input classifier-free weights.

• Composer: Creative and Controllable Image Synthesis with Composable Conditions: a cascade trained to rebuild images from random subsets of eight representations.

• MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation: overlapping denoising windows averaged at each step, for panoramas and region layouts.

• Semantic diffusion guidance (More Control for Free!): CLIP-based guidance of an unconditional DDPM at sampling time, with a finetuned CLIP encoder.

• Human Motion Diffusion as a Generative Prior: composes a pretrained motion model into long, two-person and controlled motion without retraining it.

• Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion: per-token noise levels with a causal recurrent denoiser, for rollouts, guidance and planning.

• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models: a causal few-step video student distilled from a bidirectional teacher, with a KV cache.

• Simple and Effective Masked Diffusion Language Models: masked diffusion over text tokens with a simplified continuous-time objective.

• DiGress: Discrete Denoising diffusion for graph generation: discrete denoising over labelled graphs with a graph transformer.

• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed: regresses a one-step student onto a deterministic multi-step DDIM teacher.

• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting: an LSTM-conditioned diffusion sampler for multivariate forecasts, one time step at a time.

• Low-dimensional adaptation of diffusion models: Convergence in total variation: sampling-step bounds that scale with intrinsic dimension, for original DDIM and DDPM coefficients (theory only).

• Synthetic Data from Diffusion Models Improves ImageNet Classification: fine-tunes an Imagen cascade on ImageNet and uses its samples as extra classifier training data.

• CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images: uses Stable Diffusion v1.4 outputs as a labelled fake class for a real-versus-generated classifier.

• LayoutDM: Discrete Diffusion Model for Controllable Layout Generation: discrete diffusion over quantized layout tokens, with conditions applied at inference.

• Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning: Stable Diffusion image variations as extra views for test-time prompt tuning of CLIP.

• Dense Text-to-Image Generation with Attention Modulation: attention modulation of a frozen Stable Diffusion model for dense captions and region layouts.

• State of the Art on Diffusion Models for Visual Computing: a 2023 survey of diffusion models for images, video, 3D and 4D, with no experiments of its own.

• To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images: a prompt-level attack that tests whether concept unlearning holds in text-to-image models.

• StyleAligned: Style Aligned Image Generation via Shared Attention: shared self-attention with a reference image, for one style across a batch with no training.

• DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion: a fine-tuned Stable Diffusion for pose-guided fashion video from one photo.

• EMO: Emote Portrait Alive: audio-to-video diffusion for portrait videos, with no skeleton or 3D face model.

• VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time: a diffusion transformer over a learned face latent, for talking faces that render in real time.

• JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization: one diffusion transformer for video and audio, linked by cross-attention and shared prior tokens.

• Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback: offline reward-weighted and DPO fine-tuning of a text-to-video diffusion model, with accept or reject labels from a VLM.

• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation: a video model's motion used to fit a material field for physics simulation of 3D Gaussian objects.

• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control: video diffusion transformer features feeding an action diffusion transformer for robot control.

• Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model: a distilled causal generator for interactive game video, with a sink-token KV cache.

• Label-Efficient Semantic Segmentation with Diffusion Models: per-pixel features from a frozen DDPM decoder, classified by an MLP ensemble trained on a few labels.

• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation: a 2D denoiser's score back-propagated through a voxel renderer for text-to-3D.

• One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization: Zero123 views fused in one feed-forward pass into a textured mesh.

• CLIP-Mesh: Generating textured meshes from text using pretrained image-text models: a subdivision mesh, texture and normal map optimised from a text prompt through CLIP loss, with an optional diffusion-prior term.

• SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model: inpaints a named object into a mask whose precision the user sets, with a predicted-mask output that keeps the background.

• LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation: a layout-conditioned diffusion model trained from scratch on box tokens, with object-aware cross-attention.

• Training-Free Layout Control with Cross-Attention Guidance: gradient guidance of cross-attention maps during the first denoising steps, steering a frozen Stable Diffusion to a layout.

• Common Diffusion Noise Schedules and Sample Steps are Flawed: zero terminal SNR, v prediction, trailing steps and rescaled guidance, for a wider brightness range.

• 3D Shape Generation and Completion through Point-Voxel Diffusion: DDPM-style denoising on point clouds, for unconditional generation and completion.

• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation: equivariant diffusion over atom coordinates, conditioned on the molecular graph.

• Exploiting Diffusion Prior for Real-World Image Super-Resolution: a frozen Stable Diffusion 2.1 with a trained time-aware encoder, for blind super-resolution.

• Unified Camera Positional Encoding for Controlled Video Generation: per-token ray encodings with a latitude-and-up map, added to a video diffusion transformer through a parallel adapter.

• Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper: a visuo-tactile encoder pretrained by masked reconstruction, feeding a diffusion policy.

• Octo: An Open-Source Generalist Robot Policy: a pretrained tokenized transformer policy with a small diffusion action head, finetuned to new sensors and action spaces.

• DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control: a VLM with a 1B diffusion action expert, trained in three stages, that writes sub-step text for long tasks.

• Magic3D: High-Resolution Text-to-3D Content Creation: a two-stage text-to-3D method, a hash-grid field with a 64 px prior, then a textured mesh with a 512 px latent prior.

• ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation: a BLIP-based preference reward model, and ReFL, which back-propagates it through one late denoising step of Stable Diffusion v1.4.

• Adversarial Diffusion Distillation: a one- to four-step student from a frozen diffusion teacher, trained with a hinge-loss discriminator and score distillation.

• DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset: a large multi-lab robot dataset, with a diffusion-policy co-training study on six tasks.

• DiffIR: Efficient Diffusion Model for Image Restoration: a diffusion model over a 256-value restoration prior, in four steps, with a two-stage training run.

• Emergent Correspondence from Image Diffusion: frozen diffusion features with nearest-neighbour matching for dense correspondence, with no training.

• FPC-VLA: A Vision-Language-Action Framework with a Supervisor for Failure Prediction and Correction: a diffusion action head in a VLA, with a VLM supervisor that corrects actions at gripper changes.

• VideoVLA: Video Generators Can Be Generalizable Robot Manipulators: a video diffusion transformer that denoises future frames and a 6-step action chunk together.

• STORM: Search-Guided Generative World Models for Robotic Manipulation: a diffusion policy's candidate actions searched with MCTS over a video world model.

• DeepCache: Accelerating Diffusion Models for Free: a training-free U-Net sampler that reuses cached high-level features between full passes.

• GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Models: a DDPM in the latent of a conditional VAE over grasps, with DDIM sampling.

• PDP: Physics-Based Character Animation via Diffusion Policy: a diffusion policy over action sequences for physics-based character control.

• ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data: a diffusion policy head on fused vision and contact-microphone audio.

• RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands: a piano motion benchmark that uses DDPM diffusion policies as baselines.

• Diffusion Policy Policy Optimization: PPO fine-tuning of a pretrained diffusion policy, with each denoising step treated as a Gaussian policy.

• DexDiff: Towards Extrinsic Dexterity Manipulation of Ungraspable Objects in Unrestricted Environments: a VLM planner with a transformer action-diffusion policy for dexterous manipulation.

• Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation: a transformer diffusion policy with adaptive layer norm, scaled from 10M to 1B parameters.

• The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control: an action-sequence diffusion policy for a simulated humanoid, trained on randomized RL data.

• Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance: a contact-target Guider and an action Actor, both diffusion models, with a Q-value term.

• FastGrasp: Efficient Grasp Synthesis with Diffusion: latent DDIM over a hand latent, with a frozen-model adaptation MLP.

• OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving: object-level diffusion, then range-image scene diffusion with a controller.

• Embodiment-Agnostic Navigation Policy Trained with Visual Demonstrations: a 1D U-Net diffusion policy over relative waypoints for visual navigation.

• SegDiff: Image Segmentation with Diffusion Probabilistic Models: a DDPM over masks that adds image features to the noisy-mask features, trained from scratch and averaged over several chains.

• Perception Prioritized Training of Diffusion Models: reweights the simple diffusion loss by 1/(k+SNR)^gamma, down-weighting the lowest-noise steps.

• Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design: a guide to metrics for design generators, with a TopoDiff topology case study.

• FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention: personalizes Stable Diffusion to several people from one photo each, with a cross-attention localization loss in training.

• NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration: one diffusion policy that explores when the goal is masked and heads to the goal when it is shown.

• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution: Stable Diffusion super-resolution guided by tags from a degradation-aware tagger, with a low-resolution start for sampling.

• A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction: a diffusion prior trained only on fine flow fields, sampled from a noised coarse or sparse guide, with an optional PDE-residual term.

• MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation: two coupled denoisers for video and audio, linked by random-shift cross-modal attention.

• Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models: correlated frame noise that lets a text-to-image diffusion model be finetuned to video.