---
kind: concept
slug: score-distillation-sampling
title: Score distillation sampling
summary: >-
  Score distillation sampling (SDS) turns a frozen diffusion model into a training signal for a differentiable generator, such as a 3D scene or a one-step student. The core idea is the noise-prediction residual, sent back through the generator. The trade-offs are a noisy gradient that needs weighting and regularization, and a prior that must be queried at every step.
tags: [diffusion, distillation, text-to-3d, sampling, video-diffusion]
papers: ["2211.10440", "2302.10663", "2303.14184", "2309.16585", "2311.17042", "2403.12008"]
related: [diffusion-models, knowledge-distillation, generative-adversarial-network, neural-radiance-field, differentiable-rendering, image-to-3d-generation, neural-implicit-surface, explicit-feature-grid, gaussian-splatting, latent-diffusion, classifier-free-guidance, text-to-image-generation, noise-schedule, novel-view-synthesis, video-generation]
status: draft
updated_at: 2026-11-01
---

## Overview

SDS uses a pretrained diffusion model as a judge rather than as a sampler. A
differentiable generator produces an image, the image is noised, and the frozen
prior predicts the noise given the prompt. The difference between the predicted
and the injected noise is the gradient, and it is back-propagated through the
renderer into the generator's parameters [arxiv:2211.10440 §3] [arxiv:2211.10440 eq. 1].

The invariant all variants share: the prior is frozen, and the signal comes
from its noise prediction. Some variants use the residual directly; others use
the clean estimate derived from it [arxiv:2311.17042 §3.3] [arxiv:2303.14184 §3.2].
Variants differ in what the parameters describe (one 3D scene per prompt, one
3D object from a photo, an explicit cloud of Gaussians, or the weights of a
few-step generator), how the signal
is weighted or switched over the timestep, and which prior sets the target: an
image model at low resolution, a latent model, or a video model
[arxiv:2211.10440 §4] [arxiv:2311.17042 §3.3] [arxiv:2403.12008 §4].

## Variants

### Per-scene SDS on a neural field

The scene is a multi-resolution hash grid with two small output networks for
colour, density and normals. Stage 1 of Magic3D uses this form with a 64x64
base diffusion prior and uniform timesteps with weight w(t) = 1
[arxiv:2211.10440 §4.1] [arxiv:2211.10440 §4.2] [arxiv:2211.10440 app. B].
Card: [Magic3D: High-Resolution Text-to-3D Content Creation](/papers/2211.10440).
The paper reports about 15 minutes for this stage on 8 A100 GPUs
[arxiv:2211.10440 §5].

### Coarse-to-fine SDS on a textured mesh

Magic3D's second stage converts the field into a deformable tetrahedral grid,
extracts a mesh, and optimizes it with a 512x512 latent diffusion prior. The
timestep range is narrowed to (0.02, 0.5) with weight σ_t², and the authors
say a maximum in [0.5, 0.7] works well [arxiv:2211.10440 §4.1] [arxiv:2211.10440 §4.3]
[arxiv:2211.10440 app. B]. Card: [Magic3D](/papers/2211.10440). The high-resolution
gradient through the encoder is the main cost of this stage [arxiv:2211.10440 §4.1].

### SDS on explicit Gaussians with a point-cloud term

GSGEN stores the asset as anisotropic 3D Gaussians, each with a centre,
covariance, colour and opacity, and renders them by alpha compositing
[arxiv:2309.16585 §3.2] [arxiv:2309.16585 eq. 2]. It starts from a Point-E
point cloud for the prompt, with random colours rather than the cloud's
[arxiv:2309.16585 §4.3]. The geometry stage adds a 3D SDS term on the Gaussian
centres, with Point-E as the noise predictor. The 2D SDS term has weight 0.1 and
the 3D term has weight 0.01 [arxiv:2309.16585 §4.1] [arxiv:2309.16585 §5.1]
[arxiv:2309.16585 eq. 3]. The appearance stage keeps only 2D SDS, acting on
colours and opacities, with penalties on position and opacity and a
compactness-based densification that fills gaps between Gaussians
[arxiv:2309.16585 §4.2] [arxiv:2309.16585 eq. 4]. Card:
[GSGEN](/papers/2309.16585). The paper reports about 40 minutes per asset on one
NVIDIA 3090 and gives no quantitative metrics; its comparisons are visual
[arxiv:2309.16585 §5.2]. It also reports that a small densification threshold
over-densifies under noisy SDS gradients, and a large one over-smooths
[arxiv:2309.16585 §4.2] [arxiv:2309.16585 §5.3].

### Single-image SDS with a switch to a CLIP loss

Make-It-3D fits a NeRF to one reference photo, with no multi-view training data
[arxiv:2303.14184 §1]. Its novel views get SDS with classifier-free guidance at
weight 10, with timesteps drawn from 200 to 600 [arxiv:2303.14184 §4.1]. For
timesteps up to 400, the step uses a CLIP-based loss on a one-step denoised
estimate instead, and SDS and that loss are never applied to the same step
[arxiv:2303.14184 §3.2] [arxiv:2303.14184 app. C.1]. The paper states the SDS
weight only as uniform over the timestep [arxiv:2303.14184 §4.1]. A masked
reference-view loss and a depth correlation term sit alongside the SDS term
[arxiv:2303.14184 §3.2]. Card: [Make-It-3D](/papers/2303.14184).

The switch is a trade-off. In the coarse-stage ablation, SDS alone gives LPIPS
0.3045, and the CLIP loss alone gives the lowest LPIPS, 0.1260. The threshold of
400 gives LPIPS 0.1427, but better contextual (1.74 against 2.43) and CLIP
(87.50% against 80.27%) scores [arxiv:2303.14184 tab. 3].

### SDS with a prompt token learned from the photo

RealFusion fits an Instant-NGP radiance field to one masked RGBA photo
[arxiv:2302.10663 §3.1]. Its random prior views get SDS through Stable Diffusion
at guidance scale 100 [arxiv:2302.10663 §3.1] [arxiv:2302.10663 app. A]. The
prompt is "an image of a <e>", where <e> is a new token learned from random
augmentations of the photo with the diffusion model frozen
[arxiv:2302.10663 §3.2] [arxiv:2302.10663 app. A]. The paper's SDS omits the
diffusion model's Jacobian, which it says improves quality and reduces compute
and memory [arxiv:2302.10663 §3.1]. The photo itself supervises a fixed
reconstruction camera, and normal smoothness applies on the prior views
[arxiv:2302.10663 §3.2] [arxiv:2302.10663 eq. 6]. Card:
[RealFusion](/papers/2302.10663).

On 7 CO3D categories the paper reports a mean F-score of 9.58 against 8.24 for
Shelf-Supervised Mesh Prediction, but it is lower on two categories
[arxiv:2302.10663 tab. 1]. On a synthetic novel-view test that favours
depth-supervised methods, it reports PSNR 16.5, below the depth-supervised
methods in its table [arxiv:2302.10663 app. D] [arxiv:2302.10663 tab. 2].

### Masked SDS with a video prior

SV3D's 3D stage fits an Instant-NGP NeRF to its own generated orbit, converts it
to a DMTet mesh, and uses photometric losses [arxiv:2403.12008 §4]. In the last
200 iterations of the fine stage, SDS with the video model is applied only where
no reference view of the orbit sees the surface well. Each surface point takes
its best-aligned reference camera, and a smoothstep over the dot-product range
[0, 0.5] sets the mask [arxiv:2403.12008 §4] [arxiv:2403.12008 app. F]. The
paper says plain SDS with this guidance oversaturates and blurs the result,
which is why the mask is applied [arxiv:2403.12008 §4]. Card:
[SV3D](/papers/2403.12008).

### Distillation into a few-step student

Adversarial Diffusion Distillation (ADD) replaces the scene with a student
network initialised from the teacher's weights. The student sample is re-noised,
the frozen teacher denoises it, and the distillation term is a squared L2 loss
against that stop-gradient target, weighted by c(t) and combined with an
adversarial term at λ = 2.5 [arxiv:2311.17042 §3.3] [arxiv:2311.17042 §4] [arxiv:2311.17042 eq. 4].
The paper shows that squared L2 with c(t) = α_t / (2σ_t) · w(t) gives the
SDS gradient [arxiv:2311.17042 app. A]. The final model uses a different
weighting, NFSD, whose formula the paper does not give [arxiv:2311.17042 §4.1].
Card: [Adversarial Diffusion Distillation](/papers/2311.17042).

## When to use which

| Situation | Use | Why |
|---|---|---|
| One asset per prompt, no training data, long per-asset compute is acceptable | Per-scene SDS on a neural field | Only the prompt is needed; the paper reports about 40 minutes in total on 8 A100 for its two-stage pipeline [arxiv:2211.10440 §5] |
| Sharper geometry and texture are needed, and a mesh is the target | Coarse-to-fine: field, then mesh with a high-resolution prior | Magic3D reports 61.7% preference over DreamFusion in a user study [arxiv:2211.10440 tab. 1]; it also says a low-resolution upsampled coarse model gives worse shapes [arxiv:2211.10440 §5] |
| One reference photo, and the output must match that photo from its view | Single-image SDS with a CLIP switch | The reference-view loss and depth term keep the photo [arxiv:2303.14184 §3.2]. Needs a caption, a foreground mask and a depth estimate [arxiv:2303.14184 app. B.1]. On its benchmark, the enhanced model beats DreamFusion+ and Point-E on LPIPS, contextual and CLIP scores [arxiv:2303.14184 tab. 2] |
| One object from a photo, with no caption, and the prompt should come from the photo itself | SDS with a prompt token learned from the photo | The prompt is learned from the photo rather than written by the user [arxiv:2302.10663 §3.2]. Scored by mean F-score on 7 CO3D categories, but author-reported and below Shelf-Supervised on two of them [arxiv:2302.10663 tab. 1] |
| Explicit, editable geometry is wanted, or a 3D point-cloud prior should steer the SDS loop | SDS on explicit Gaussians with a point-cloud term | The 3D term is applied to Gaussian centres, which the paper says makes 3D priors easier to add than in implicit fields [arxiv:2309.16585 §1]. No quantitative comparison is reported; the visual results show fewer collapsed or multi-face assets [arxiv:2309.16585 §5.2] |
| Unseen sides of a mesh need a prior, and a video model can supply views | Masked SDS with a video prior, in the last fine-stage iterations | The mask keeps SDS to unseen surface [arxiv:2403.12008 §4]. The authors' ablation shows gains in 2D metrics but no change in Chamfer distance over photometric loss alone [arxiv:2403.12008 app. G.3]. Released weights are non-commercial [arxiv:2403.12008 Paper vs code] |
| Fast generation at inference, with one to four steps | Distillation into a student | Inference is one U-Net call and needs no guidance [arxiv:2311.17042 §1] [arxiv:2311.17042 §4.2] |
| Output diversity matters more than the best average quality | Exponential weight c(t) = α_t | Exponential weighting gives more diverse samples; SDS-style and NFSD weighting improve quality and text alignment [arxiv:2311.17042 §4.1] [arxiv:2311.17042 tab. 1] |
| Reference-image guidance at high resolution | Not supported on the latent path | The latent prior does not accept a reference image, so this guidance is limited to the eDiff-I prior [arxiv:2211.10440 app. D] [arxiv:2211.10440 §6] |

Treat the reported numbers as author-reported. The Magic3D speed comparison
uses a DreamFusion runtime quoted from its paper, not a re-run
[arxiv:2211.10440 §1] [arxiv:2211.10440 §5].

## Implementation sketch

Common core of the per-scene variant. The prior is used only to produce the
residual, and the gradient reaches the scene through the renderer.

```pseudocode
# One SDS step for a differentiable scene  [arxiv:2211.10440 §3] [eq. 1]
# Inputs: scene params theta, prompt y, frozen prior
# Output: updates theta in place

def sds_step(theta, prompt, prior):
    # Render from a random camera and light  [arxiv:2211.10440 app. B]
    x = render(theta, random_camera(), random_light())
    # Sample timestep and noise; stage 1 uses t ~ U(0, 1)  [app. B]
    t = sample_timestep()
    eps = normal_like(x)
    x_t = add_noise(x, eps, t)
    # Prior predicts the noise; the prior itself is not updated  [§3]
    eps_hat = prior.predict_noise(x_t, prompt, t)
    # Residual on the render, weighted by w(t)  [eq. 1] [app. B]
    grad_x = w(t) * (eps_hat - eps)
    # Back-propagate through the renderer into theta  [§3]
    x.backward(grad_x)
```

Distillation variant. The same residual becomes a loss on the student's sample,
with the teacher's clean estimate as a fixed target.

```pseudocode
# Distillation loss for one student sample  [arxiv:2311.17042 §3.3] [eq. 4]
# Inputs: student, real batch x0, prompt, frozen teacher
# Output: scalar loss for the student parameters

def distill_loss(student, x0, prompt, teacher):
    # One student evaluation at a sampled student timestep  [§3.1]
    x_hat = student(noise_real(x0), s)
    # Re-noise with the teacher's forward process  [§3.3]
    x_t = renoise(x_hat, t)
    # Teacher's clean estimate is a stop-gradient target  [§3.3]
    target = stop_gradient(teacher.denoise(x_t, prompt, t))
    # Weighted squared L2; c(t) = alpha_t / (2 sigma_t) * w(t) gives SDS  [app. A]
    return c(t) * sum_squares(x_hat - target)
```

Where the variants differ: the timestep range and weight w(t) are chosen by
the paper for each stage [arxiv:2211.10440 app. B]; Make-It-3D switches to a
CLIP-based loss at small timesteps [arxiv:2303.14184 §3.2]; RealFusion's prompt
is a learned token [arxiv:2302.10663 §3.2]; GSGEN adds a point-cloud SDS term
on Gaussian centres [arxiv:2309.16585 §4.1]; SV3D masks the SDS term per
surface point [arxiv:2403.12008 §4]; the distillation weight c(t) is a free
choice in the student case [arxiv:2311.17042 §4.1]; and the renderer is either
a volume, a Gaussian cloud or a mesh rasteriser [arxiv:2211.10440 §4.2]
[arxiv:2309.16585 §3.2].

## Contradictions

Most of the differences below are about scope, not disagreement. Each side is
cited so the reader can check them.

- Magic3D optimizes each scene with SDS as its training signal
  [arxiv:2211.10440 §3]. ADD reports that distillation alone is ineffective at
  one step, with a COCO FID-5k of 315.6, and that the adversarial term is
  needed [arxiv:2311.17042 tab. 1]. That result concerns training a student,
  not optimizing one scene, so it does not contradict Magic3D.
- Weighting differs across the papers. Magic3D uses w(t) = 1 in stage 1
  and σ_t² in stage 2 [arxiv:2211.10440 app. B]. Make-It-3D leaves its weight
  as uniform over the timestep [arxiv:2303.14184 §4.1], so its weight cannot be
  matched to ADD's equivalence result. RealFusion also states its weight only
  as w(t) [arxiv:2302.10663 §3.1].
- Guidance scale differs. RealFusion says DreamFusion-style SDS needs a
  classifier-free guidance weight of 100 [arxiv:2302.10663 §3.1], and it
  reports that values above 30 gave good results [arxiv:2302.10663 app. A].
  GSGEN also uses 100 [arxiv:2309.16585 §5.1]. Make-It-3D uses weight 10 on its
  novel views [arxiv:2303.14184 §4.1]. The papers do not reconcile these
  values, so a reader should not assume one setting carries over.
- ADD's text and its table disagree on quality. The text says SDS-style and
  NFSD weighting give better quality and text alignment than the exponential
  default [arxiv:2311.17042 §4.1]. Table 1 agrees on the CLIP score (0.325 for
  SDS and 0.319 for the exponential default), but gives the SDS row a worse FID
  (22.3 against 20.6) [arxiv:2311.17042 tab. 1]. NFSD also has a worse FID than
  the exponential default (21.8) in that table.
- SV3D's text says masked SDS improves the result, and its 2D metrics agree.
  Its 3D table does not: the Chamfer distance is 0.024 with or without the soft
  mask, and the 3D IoU moves from 0.611 to 0.614 [arxiv:2403.12008 app. G.3]
  [arxiv:2403.12008 §4].

## Papers

- [Magic3D: High-Resolution Text-to-3D Content Creation](/papers/2211.10440): coarse-to-fine SDS, a hash-grid neural field and a textured mesh stage with a 512x512 latent prior.
- [Make-It-3D](/papers/2303.14184): SDS with classifier-free guidance on novel views of a single-image NeRF, switched to a CLIP-based loss at timesteps up to 400.
- [RealFusion: 360° Reconstruction of Any Object from a Single Image](/papers/2302.10663): SDS on random views of a radiance field fit to one photo, with a prompt from a token learned from that photo.
- [Text-to-3D using Gaussian Splatting](/papers/2309.16585): SDS on explicit Gaussians, with a Point-E point-cloud SDS term in the geometry stage and 2D SDS alone in the appearance stage.
- [SV3D](/papers/2403.12008): masked SDS with a video diffusion prior, applied to unseen surface in the last fine-stage iterations.
- [Adversarial Diffusion Distillation](/papers/2311.17042): a distillation loss equivalent to SDS under one weighting, used to train one-to-four-step text-to-image students.