Paper · 2026-10-29 · cs.CV, cs.LG
sDMD: Sinkhorn Distribution Matching Distillation
sDMD adds an optimal-transport (Sinkhorn) term to distribution matching distillation, applied only to samples a DMD2 discriminator flags as weakly supported by the teacher. Author-reported: 2- or 4-step students on Qwen-Image, FLUX.1-dev and Wan2.1 match or beat DMD2 and other baselines on GenEval, DPG, WISE and VBench, with more diversity.
Diffusion modelsKnowledge distillationOptimal transportDistribution matching distillationdiffusion-distillationfew-step-generationoptimal-transporttext-to-imagetext-to-video
AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Image and video generators based on diffusion are slow, because they refine a sample over many steps. Distillation trains a faster student that copies a slow teacher in a few steps. One popular recipe, DMD, makes the student's outputs look like the teacher's by following the teacher's sense of which regions are likely. That sense can be unreliable where the teacher has seen little data, and the student can drift toward a few favourite looks and ignore the rest.
This paper adds a second signal: a transport step, which asks how far each student sample would have to move to match the set of teacher samples. Such a step pulls stray samples toward the teacher's distribution and encourages the student to cover more of it. The transport step is noisy when computed from a small batch, so it would hurt samples the teacher already handles well. The paper therefore uses a discriminator, already present in the DMD2 recipe, to decide which samples get the transport correction. The rest keep the original teacher signal.
The authors report that this gives better scores and more varied outputs than DMD2 on image and video benchmarks, at a small extra training cost.
TL;DR
sDMD is a distillation method for diffusion models that keeps DMD's teacher score signal and adds an entropic optimal-transport (Sinkhorn) correction. The correction is applied only to samples that a discriminator marks as having weak teacher support §5.2. Author-reported results, with students that take two or four steps, on three text-to-image and video backbones match or beat the compared few-step baselines on most metrics, and improve diversity over DMD2 §6. No official code was found [Paper vs code].
Problem
Diffusion models need many sampling steps, and distillation into few-step students is the usual fix §1. DMD is widely used and has high fidelity, but the paper argues that its update uses only local density: teacher scores can be unreliable where the teacher has low density, and the reverse-KL objective can make the student cover only some teacher modes §1§4.2. Adding diffusion noise helps overlap but blurs local structure §1§4.2. The paper's aim is an additional term that handles outliers and mode coverage, without making the student depend on learned teacher scores alone §1§5.
Method
The method has three parts.
- Connection to Wasserstein gradient flow. The DMD gradient is recovered from a fixed-point regression whose energy is the integral KL (IKL): reverse KL averaged over diffusion noise levels §4.1 eq. 7 eq. 8. The equivalence holds "up to a constant" §4.1; the derivation is in app. B.1.
- Sinkhorn velocity. The transport velocity is the difference between two barycentric projections: one from student samples to teacher samples, one from student samples to themselves (self-transport) §3 eq. 6. The combined velocity adds the weighted transport term to the IKL velocity §5.1 eq. 9 eq. 10. Training uses the same stop-gradient regression target as the fixed-point form §5.1 eq. 11.
- Gated scheduler. The discriminator D from DMD2 is trained to tell perturbed teacher samples from perturbed student samples §5.2 eq. 12. Its output defines a gate lambda = 1 - D, which is near q_t / (p_t + q_t) for an optimal discriminator §5.2 eq. 13. The transport term is switched on for a sample only when its gate is above a threshold lambda; otherwise the update is the IKL velocity alone §5.2 eq. 14. No separate discriminator is trained; the DMD2 adversarial objective is reused §5.2.
Practical choices: the transport is computed on clean latents for the large text-to-image and video models and on a feature space (DINOv2 or MAE, by application) elsewhere §5.3 tab. 6 app. D. For text conditioning, student and teacher samples are drawn for the same prompt, and teacher samples are generated in advance and stored in a buffer §5.3. The paper also sets the number of Sinkhorn iterations as a fixed number per application, with no convergence test alg. 2 tab. 6.
# sDMD training step [alg. 1]
# Input: noise batch (M_q, d), teacher samples (M_p, d), generator f
# Output: generator and discriminator update
def sdmd_step(noise, teacher_samples):
# 1. Few-step sampling of the student [alg. 1]
students = generator(noise) # (M_q, d)
# 2. Random noise level and noise for each student [alg. 1]
t = sample_noise_levels(M_q) # (M_q,)
eps = normal_like(students) # (M_q, d)
noisy = alpha(t) * students + sigma(t) * eps # x_t
# 3. IKL velocity from teacher and student score networks [alg. 1] [eq. 8]
v_ikl = alpha(t) * (teacher_score(noisy, t) - fake_score(noisy, t))
# 4. Gate from the discriminator (lambda = 1 - D) [alg. 1] [eq. 13]
gate = 1 - discriminator(noisy, t) # (M_q,)
active = gate > threshold # (M_q,) boolean
# 5. Sinkhorn barycentric projections for the transport term [alg. 2]
t_qp = sinkhorn_projection(students, teacher_samples, beta, L)
t_qq = sinkhorn_projection(students, students, beta, L)
# 6. Add the weighted transport term only where the gate is active [eq. 14]
v_sdmd = where(active, v_ikl + w * (t_qp - t_qq), v_ikl)
# 7. Regression to the stop-gradient target [alg. 1] [eq. 11]
target = stop_gradient(students + v_sdmd)
loss_gen = mean(squared_norm(students - target)) # reduction not stated
# 8. Add adversarial term; update generator and discriminator [alg. 1]
loss = loss_gen + adversarial_loss(students, D)
update_generator_and_discriminator(loss)
# 9. Update the student score by denoising score matching [alg. 1]
update_fake_score(students)# Sinkhorn barycentric projection [alg. 2]
# Input: samples (N, d), targets (M, d), beta, iteration count L
# Output: projected point for each sample (N, d)
def sinkhorn_projection(samples, targets, beta, iters):
# 1. Uniform marginals on both sides [alg. 2]
a = ones(N) / N # (N,)
b = ones(M) / M # (M,)
# 2. Gibbs kernel from squared distance [alg. 2] [§3]
kernel = exp(-squared_distance(samples, targets) / (2 * beta))
v = ones(M) # (M,)
# 3. Fixed number of alternating updates, no convergence test [alg. 2]
for step in range(iters):
u = a / (kernel @ v) # (N,)
v = b / (kernel.T @ u) # (M,)
# 4. Transport plan from the scaling vectors [alg. 2]
plan = diag(u) @ kernel @ diag(v) # (N, M)
# 5. Barycentric map: plan-weighted average of targets [alg. 2] [§3]
row_mass = sum(plan, axis=1) # (N,)
return (plan @ targets) / row_mass[:, None] # (N, d)Hyperparameters
All values from tab. 6 unless marked. Columns are per application; Wan2.1 has two sizes, given as 1.3B / 14B.
| Name | ImageNet-256 | FLUX.1-dev | Qwen-Image | Wan2.1 (1.3B / 14B) |
|---|---|---|---|---|
| Backbone | SiT-XL/2 (675M) | FLUX.1-dev (12B) | Qwen-Image (20B) | 1.3B / 14B |
| Resolution | 256 x 256 | 512 x 512 | 1024 x 1024 | 832 x 480 |
| Latent shape | 32 x 32 x 4 | 64 x 64 x 16 | 128 x 128 x 16 | 104 x 60 (spatial) |
| Which weights train | full | full | LoRA | LoRA |
| Adapter rank (student / score) | none | none | 256 / 64 | 256 / 64 |
| Batch size | 256 | 128 | 128 | 48 / 32 |
| Training iterations | 100K | 5K | 3K | 1K |
| Student / score learning rate | 3e-5 | 1e-5 | 5e-5 | 1e-5 |
| Discriminator learning rate | 2e-6 | 1e-5 | 1e-5 | 1e-5 |
| Student / score weight decay | 0 | 0.01 | 0.01 | 0.01 |
| Discriminator weight decay | 0.01 | 0.01 | 0.01 | 0.01 |
| Student NFE | 1 | 4 | 2 and 4 | 4 |
| Teacher guidance scale | 1.5 | 3.5 | 4 | 5 |
| Adversarial loss weight | 0.003 | 0.03 | 0.03 | 0.03 |
| Samples per condition M_q = M_p | 8 | 8 | 8 | 6 / 8 |
| Self-transport T_qq | separate batch | self-pairs masked | self-pairs masked | self-pairs masked |
| Regularisation beta | 0.05 | 0.05 | 1 | 1 / 0.5 |
| Sinkhorn iterations L | 40 | 10 | 100 | 50 |
| Sinkhorn weight w | 0.5 | 0.5 | 2 | 0.5 |
| Transport representation | MAE features | DINOv2 features | latent | latent |
| Gate threshold lambda | 0.50 | 0.55 | 0.55 | 0.55 |
| Gate warmup iterations | 1,000 | 1,000 | 1,000 | 50 |
Other settings, all from app. D:
- Teacher samples use each model's default guidance scale and sampling steps app. D; the table's guidance row is listed separately tab. 6.
- The generator is stepped once per four updates of the discriminator and the fake score, in DMD2 and sDMD alike app. D.
- Compute: H200 nodes are allotted as one node each for ImageNet, FLUX and the 1.3B video model, two for Qwen-Image, and four for the 14B video model app. D.
- ImageNet: ImageNet-1K images encoded with a pretrained VAE to 32 x 32 x 4 latents; transport groups samples by class; MAE features from a frozen encoder; two-batch self-transport in the main run app. D.
- FLUX: 8,000 teacher images from 1,000 GenEval prompts, eight per prompt; transport computed separately per prompt; frozen DINOv2 ViT-B/14 at 224 x 224 with features read at four evenly spaced depths and stacked into one multiscale representation app. D.
- Qwen-Image: LoRA adapters; teacher images from the HPSv2 prompt set; diagonal masking in self-transport app. D.
- Wan2.1: 81-frame videos at 16 fps and 832 x 480; teacher latents from MovieGenVideoBench prompts; four-step schedule (0.999, 0.937, 0.833, 0.624, 0) from FastGen; for 14B, samples are pooled across nodes per prompt app. D.
- Optimiser type, betas, the distribution of the noise level t and the form of the adversarial loss are not stated in the main text or appendix.
Results
Author-reported. Teacher and checkpoint notes are as in the paper's tables; rows marked as official checkpoints were not retrained by the authors, and the DMD2 rows were trained with FastGen §6.1 §6.2.
Qwen-Image text-to-image, GenEval / DPG-Bench / WISE tab. 1
| Steps | Method | GenEval | DPG | WISE |
|---|---|---|---|---|
| teacher (50 x 2 NFE) | Qwen-Image 20B | 0.860 | 88.70 | 0.54 |
| 2 | TwinFlow (checkpoint) | 0.827 | 86.65 | 0.50 |
| 2 | DMD2 | 0.856 | 87.08 | 0.53 |
| 2 | sDMD | 0.862 | 87.03 | 0.54 |
| 4 | Lightning (checkpoint) | 0.864 | 88.41 | 0.51 |
| 4 | TwinFlow (checkpoint) | 0.817 | 86.24 | 0.49 |
| 4 | pi-Flow (checkpoint) | 0.863 | 87.90 | 0.53 |
| 4 | sDMD | 0.867 | 88.45 | 0.53 |
Wan2.1 text-to-video, VBench (4 steps) tab. 2
| Model | Method | Overall | Quality | Semantic |
|---|---|---|---|---|
| 1.3B | teacher | 83.03 | 84.29 | 78.00 |
| 1.3B | DMD2 | 83.65 | 85.73 | 75.32 |
| 1.3B | rCM (checkpoint) | 82.78 | 84.13 | 77.31 |
| 1.3B | AnyFlow (checkpoint) | 83.86 | 85.15 | 78.71 |
| 1.3B | sDMD | 84.12 | 85.94 | 76.81 |
| 14B | teacher | 83.71 | 85.13 | 78.02 |
| 14B | DMD2 | 84.69 | 86.67 | 76.75 |
| 14B | rCM (checkpoint) | 83.97 | 85.40 | 78.28 |
| 14B | AnyFlow (checkpoint) | 84.38 | 85.70 | 79.11 |
| 14B | sDMD | 84.87 | 85.96 | 80.51 |
ImageNet-256, one-step class-conditional FID tab. 7
| Method | NFE | FID |
|---|---|---|
| SiT (reported) | 250 x 2 | 2.06 |
| SiT + REPA (reported) | 250 x 2 | 1.42 |
| DMD2 | 1 | 3.14 |
| sCM | 1 | 5.11 |
| rCM | 1 | 3.51 |
| MeanFlow (reported) | 1 | 3.43 |
| sDMD | 1 | 2.83 |
| REPA + pi-Flow (reported) | 1 | 2.85 |
| REPA + sDMD | 1 | 2.62 |
FLUX.1 12B, four steps at 512 x 512 tab. 8
| Method | NFE | FID | CMMD | CLIP |
|---|---|---|---|---|
| teacher | 50 x 2 | 11.10 | 0.19 | 27.31 |
| DMD2 | 4 | 16.67 | 0.22 | 27.14 |
| sCM | 4 | 14.20 | 0.25 | 26.17 |
| sDMD | 4 | 13.78 | 0.19 | 27.46 |
The authors could not reproduce rCM on FLUX because training was unstable app. E.2. The FID, CMMD and CLIP are computed on MJHQ reference images app. E.2.
Diversity tab. 5
| Model | FLUX TCE | FLUX LPIPS MPD | Qwen TCE | Qwen LPIPS MPD | Wan 14B V-JEPA 2 | Wan 14B VideoMAE V2 |
|---|---|---|---|---|---|---|
| teacher | 7.25 | 0.469 | 5.86 | 0.397 | 28.91 | 18.36 |
| DMD2 | 5.93 | 0.206 | 4.72 | 0.161 | 16.78 | 13.74 |
| sDMD | 6.75 | 0.348 | 5.43 | 0.301 | 20.68 | 13.85 |
Energy functional, ImageNet-256 FID tab. 3
| Energy | FID |
|---|---|
| Forward KL | 3.23 |
| IKL | 3.14 |
| MMD | 6.21 |
| Sinkhorn | 5.09 |
| IKL + MMD | 3.01 |
| sDMD (IKL + Sinkhorn, gated) | 2.83 |
Transport space and scheduler tab. 4
| Space | Scheduler | ImageNet FID | FLUX FID | FLUX CMMD | FLUX CLIP |
|---|---|---|---|---|---|
| psi(x) | gated | 2.83 | 13.78 | 0.187 | 27.46 |
| psi(x) | ungated | 2.88 | 13.89 | 0.201 | 27.46 |
| x (latent) | gated | 2.84 | 14.48 | 0.211 | 27.45 |
Sinkhorn sample counts, ImageNet FID tab. 9
| M_q (M_p = 8) | FID | M_p (M_q = 8) | FID | |
|---|---|---|---|---|
| 1 | 3.23 | 1 | 3.97 | |
| 4 | 2.98 | 4 | 3.02 | |
| 8 | 2.83 | 8 | 2.83 | |
| 16 | 2.84 | 16 | 2.85 |
Training cost, ImageNet-256 tab. 10 (GPU hours on H200; the distillation rows leave out teacher pretraining, whose cost the first two rows give separately tab. 10)
| Method | Params | Iterations | NFE | GPU hours | FID |
|---|---|---|---|---|---|
| SiT (teacher) | 675M | 7M | 250 x 2 | 2,290 | 2.06 |
| SiT + REPA (teacher) | 675M / 87M | 4M | 250 x 2 | 1,220 | 1.42 |
| W-Flow, from scratch (reported) | 675M / 49M | 200K | 1 | 4,639 | 1.29 |
| DMD2 | 675M | 100K | 1 | 114 | 3.14 |
| sDMD, diagonal mask | 675M / 49M | 100K | 1 | 118 | 2.91 |
| sDMD, two batch | 675M / 49M | 100K | 1 | 145 | 2.83 |
| REPA + sDMD, two batch | 675M / 49M | 100K | 1 | 145 | 2.62 |
Headline claims as stated by the authors: sDMD matches or beats the compared Qwen-Image baselines except a marginally lower DPG score than DMD2 at 2 steps §6.1; it is highest overall and semantic on Wan2.1-14B and highest overall and quality on Wan2.1-1.3B §6.2; it improves every reported diversity metric over DMD2 §6.4; and it gives the best one-step ImageNet FID among the rows listed, 2.83 without REPA and 2.62 with REPA app. E.1.
Figures, described in words and not reproduced: fig. 1 shows four-step samples from sDMD with FLUX.1 and Qwen-Image. fig. 2 is a 2D Gaussian-mixture toy with four panels: IKL on well-aligned samples, IKL on outliers under poor initialisation, IKL mode collapse, and the Sinkhorn velocity on outliers, with exact and learned teacher scores and large and small batches as variants §4.2 fig. 2. fig. 3 and fig. 4 are qualitative comparisons with DMD2 and the released baselines for images and videos §6.1 §6.2. Further qualitative examples for FLUX, Qwen-Image and Wan2.1-14B are in app. G.
Paper vs code
No official code.
The paper does not link a repository. The project page (sinkhorn-dmd.github.io) shows a "Code (Coming Soon)" button that points back to the project page. The arXiv comments field names only the project page. The searches were: the project page and arXiv listing; GitHub repository search for "sDMD", "sinkhorn distillation", "sinkhorn-dmd", "sinkhorn DMD distillation", "distribution matching distillation" restricted to the Netflix organisation, and "sDMD" restricted to the Netflix organisation. One GitHub organisation named Sinkhorn-DMD holds a repository with the project website's source, but its owner is not linked from the paper or project page and could not be tied to the authors, so it was not used. Third-party Sinkhorn distillation repositories (SinKD and others) are different methods and were not used.
Pitfalls
- No code, so the details below are from the paper only. Every value in Hyperparameters comes from tab. 6 and app. D; nothing has been checked against an implementation.
- Gating as written is ambiguous. Algorithm 1 gives one "if" on the gate without an index alg. 1, while Eq. 14 applies the indicator per sample eq. 14. The per-sample reading is used in the pseudocode above. Batch-level gating would change the cost and the update.
- Gate warmup is not described. Table 6 gives a warmup length per application, but the paper does not say what the gate or the transport term do during warmup tab. 6.
- The gate is an approximation. lambda = 1 - D equals q_t / (p_t + q_t) only for an optimal discriminator §5.2 eq. 13. The paper does not measure how close the trained DMD2 discriminator is to optimal, and the gate threshold was tuned per application (0.50 or 0.55) tab. 6.
- Self-transport conflicts across sources. Algorithm 1 computes the self term from the same student batch with no mask described alg. 1. Table 6 uses a diagonal mask for FLUX, Qwen-Image and Wan2.1, and independent batches for ImageNet tab. 6. The main ImageNet result (2.83) matches the two-batch setting tab. 10. Table 6 is the only source that names the choice per application, so an implementation should follow it and note the conflict.
- Sinkhorn is run for a fixed number of iterations. L is 10 to 100 with no convergence check alg. 2 tab. 6. The paper's own convergence results need a bound on the transport error app. B.4; the iteration count is not tied to any such bound.
- Numerical stability of the kernel. Algorithm 2 forms exp(-||x - y||^2 / (2 beta)) directly, with beta as small as 0.05 for ImageNet and FLUX alg. 2 tab. 6. With many latent dimensions the exponent can become very large in magnitude, so the kernel can underflow to zero and the divisions in the update fail. The paper does not state a log-domain implementation or any clamping. This is an inference from the formula, not a reported failure.
- Scale of beta depends on the representation. beta is given in the units of the squared distance in each space (MAE features, DINOv2 features or latents) tab. 6. The paper does not state the scale of the latents or features, so beta cannot be transferred across representations without a sweep.
- Which space is used is inconsistent in the text. Section 5.3 recommends the clean latent space for "large-scale" models and names Wan2.1 and Qwen-Image in Section 6.3 §5.3 §6.3. Table 6 uses DINOv2 features for FLUX, which is also a large model tab. 6. The FLUX ablation in Table 4 is the same configuration as the main FLUX result tab. 4 tab. 8.
- Gradient through the feature encoder. Section 6.3 says feature-space transport needs backpropagation through the encoder §6.3. Eq. 11 applies a stop-gradient to the whole velocity, so the transport term is not differentiated eq. 11. Whether gradients reach the encoder through any other path is not stated.
- Teacher guidance scale is listed but its use is unclear. Table 6 lists a teacher guidance scale for each application, and Appendix D says teacher samples use the model's default guidance tab. 6 app. D. The paper does not say whether the score networks use the same guidance.
- Update ratio and optimiser. Appendix D gives one generator update per four discriminator and fake-score updates app. D, while Algorithm 1 updates all parts in each loop iteration alg. 1. The optimiser type, its betas and the reduction in the squared-norm loss (sum or mean over dimensions and batch) are not stated.
- Adversarial weight. The table gives an adversarial loss weight, but the paper does not say whether it multiplies the generator's adversarial term only tab. 6.
- Cost claim depends on the setting. The text says diagonal masking adds about 3.5 percent training cost app. E.4. Table 10 agrees (118 against 114 GPU hours), but two-batch self-transport costs 145 GPU hours, about 27 percent more than DMD2 tab. 10. The "fewer GPU hours than training from scratch" comparison excludes teacher pretraining in the distillation rows, which for the SiT teacher alone is 2,290 GPU hours tab. 10 app. E.4.
- Small gaps, single numbers. Several ablation gaps are small (for example 2.83 against 2.88 for gated against ungated on ImageNet, and 13.78 against 13.89 on FLUX) tab. 4. The paper reports no variance over seeds or runs.
- Diversity metrics are not perfectly aligned with quality. On Wan2.1-14B, the VideoMAE V2 diversity of sDMD (13.85) is barely above DMD2 (13.74) tab. 5. The claim of higher diversity on all metrics rests on small margins there.
- Teacher samples for text conditioning. Teacher samples are generated in advance per prompt and stored in a buffer §5.3. The buffer size, how many samples per prompt were stored, and how they were refreshed are not stated.
- Noise schedule and timestep distribution. The main text does not give (alpha_t, sigma_t) or the distribution of t for the experiments. The appendix theory assumes monotone endpoints app. B.1, and the Wan sampling schedule is from FastGen app. D. The training distribution of t is not stated.
- Numbering. Equation, table and figure numbers here were counted in the order of the LaTeX source, not from a compiled PDF. Check them against the compiled paper before citing a number in a new page.
Lineage
- Builds on: Distribution Matching Distillation (DMD) §3 §4 and its adversarial successor DMD2, which supplies the discriminator and the baseline trained with FastGen §5.2 §6.
- Builds on: entropic optimal transport and the debiased Sinkhorn divergence §3 eq. 4 eq. 5.
- Related: IKL is the energy used by Diff-Instruct; the paper's Wasserstein gradient flow view of DMD is new in this paper §4.1.
- Compared against: TwinFlow, pi-Flow, Qwen-Lightning, rCM, AnyFlow, sCM, MeanFlow and W-Flow §6 tab. 1 tab. 2 tab. 7 tab. 8 tab. 10.
- Concurrent work on WGF fixed-point regression for drifting models is discussed in §4.1 and app. C.
Source and license
Adapted from “sDMD: Sinkhorn Distribution Matching Distillation” by Jaihoon Kim, Lingxiao Li, Soshi Shimada, Roshni Cooper, Minhyuk Sung, Yiwei Zhao (arXiv:2610.39889), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.