wikillm

Paper · 2026-10-23 · cs.CV, cs.LG, cs.SD

Support-guided OOD Removal with Diffusion

SORD removes a localized, structured corruption given only its support mask and one diffusion prior for clean signals. It inpaints the support with K samples, then alternates residual estimation, diffusion refinement and measurement consistency, with the sample spread setting the refinement scale and the stop. Author-reported: 27.16 dB inside the support for MNIST digits on CelebA faces.

Diffusion modelsGenerative priors for inverse problemsDiffusion inpaintingdiffusioninverse-problemsinpaintingwatermark-removalshadow-removalaudio-source-separation

arXivPDFCodeMarkdown15 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Sometimes you know where a signal is damaged but not how. A noise patch is pasted over part of a face, a logo sits on a photo, a shadow falls across a scene, or another instrument plays over a short stretch of a song. You may have a good model of clean faces or clean music, but no examples of the damage itself.

The method fills the damaged region with guesses drawn from the clean model, several times over. The original values inside the damaged region are not thrown away: they still mix the true signal with the damage. The method uses them to estimate the damage, cleans that estimate with the same clean model, and repeats. The spread between the guesses acts as a confidence signal. It sets how strongly each round refines the result and when to stop.

It works on images and on audio, and it needs only the damage's location. It does not beat methods trained on the exact damage in every case. If the location is drawn too small, damage can be left behind.

TL;DR

SORD recovers a clean signal from an observation that is the clean signal plus a corruption confined to a known support Ω, or, for shadows, a multiplicative attenuation §1 eq. 1 eq. 9. It needs only Ω and one diffusion prior trained on clean signals, with no paired data and no model of the corruption §1 §3. It starts from K inpainting samples and alternates three steps, with the spread across samples setting the refinement scale and the stopping test §3 alg. 1. Author-reported, the method is competitive with supervised baselines and often outperforms them, but it is not best on every benchmark §1 tab. 1 tab. 2. The official code has no LICENSE file and differs from the paper's Algorithm 1 in several places (see Paper vs code).

Problem

Localized corruptions (intermittent background noise in audio, watermarks, shadows, localized image patches) are common, and their structure is hard to model §1. Three existing families each need something this setting lacks:

  • Supervised restoration needs paired clean and corrupted data for each corruption class §1 §2.
  • Posterior-sampling methods (for example DPS) need a known likelihood; the Gaussian variants the paper tests fail on highly structured corruptions §1 §4.1.
  • Source separation needs a generative model for each corruption class §1 §2.

SORD assumes only that the corruption is additive and confined to Ω, that y equals the clean x outside Ω, and that a clean-signal prior is available §3 eq. 1. No statistical model of the corruption is used §3. Watermarks follow an alpha-compositing model with a pixel-varying opacity that the method is not given eq. 8 §4.3. Shadows are treated as a multiplicative attenuation, handled in the log domain for the measurement update eq. 9 §4.4.

Method

SORD is Algorithm 1 alg. 1. Its components are:

  1. Initialisation. Draw K samples of diffusion inpainting, RePaint-style, conditioned on y outside Ω. The paper says the method is fairly robust to the inpainting choice §3.
  2. Uncertainty. σ*_m is the root mean square, over pixels in Ω, of the spread across the K samples eq. 4 alg. 1. The paper does not state whether the variance is the sample (unbiased) variance.
  3. Residual. For each sample k, the residual on Ω is the observation minus the sample, masked to Ω eq. 2. If the sample were exact, the residual would equal the true corruption §3.
  4. Residual refinement. The residual is re-noised at scale σ*_{m−1} and denoised by the prior eq. 3. The paper instantiates the denoiser D with DDIM §3.
  5. Measurement consistency. Inside Ω the new estimate is y minus the refined residual; outside Ω it is y eq. 5.
  6. Prior refinement. The result is re-noised at the fixed schedule σ_m and denoised, then y is restored outside Ω eq. 6.
  7. Stop test. Recompute σ*m. If σ*{m−1} − σ*m ≤ τ_stop σ*{m−1}, return the mean of the previous iterate; otherwise repeat until M iterations and return the mean of the last iterate alg. 1.

The output is always the mean over the K samples alg. 1. The paper's own description and the code differ in several places (see Paper vs code).

Pseudocode
# SORD as printed in the paper  [alg. 1]
# y: observation (H, W, C); omega: support (H, W), bool; omega broadcasts
# prior.denoise(z, sigma): clean-signal denoiser D  [§3]
# sigmas: schedule sigma_m, linear from 0.4 to 0.1 over M steps  [§7.1]

def sord(y, omega, prior, K, M, sigmas, tau_stop):
    # Initialise: K RePaint inpaintings, y kept outside omega  [alg. 1]
    samples = [inpaint(prior, y, omega) for _ in range(K)]
    # Uncertainty on the support: RMS of the spread across samples  [eq. 4]
    sigma_prev = spread(samples, omega)
    for m in range(1, M + 1):
        new_samples = []
        for x_prev in samples:
            # Raw residual on the support  [eq. 2]
            residual = omega * (y - x_prev)
            # Refine the residual with the prior at scale sigma_prev  [eq. 3]
            noisy = residual + sigma_prev * randn_like(y)
            correction = prior.denoise(noisy, sigma_prev)
            # Measurement consistency: y minus correction on omega  [eq. 5]
            evidence = where(omega, y - correction, y)
            # Prior refinement at scale sigma_m; y kept outside omega  [eq. 6]
            sigma_m = sigmas[m - 1]
            refined = prior.denoise(evidence + sigma_m * randn_like(y), sigma_m)
            new_samples.append(omega * refined + (1 - omega) * y)
        # New uncertainty and adaptive stop  [alg. 1; eq. 4]
        sigma_next = spread(new_samples, omega)
        if sigma_prev - sigma_next <= tau_stop * sigma_prev:
            # Stop: return the previous iterate, averaged over samples
            return mean(samples)
        samples, sigma_prev = new_samples, sigma_next
    # No stop within M iterations: return the last iterate, averaged
    return mean(samples)

For shadows, the measurement update is carried out on log intensities, log y = log x + log s, while the prior is applied to images rather than to logs §4.4. The code converts the log-gain into an equivalent additive residual steps.py.

Hyperparameters

NameValueWhereNotes
Diffusion samples K32 (images, default); 64 (audio, main); 4 (audio ablation)§7.1 tab. 3 tab. 6Code: K=32 for image configs celeba_cifar.yaml, K=64 for audio audio_piano.yaml
Maximum iterations M6 (CelebA localized benchmarks)§7.1Paper gives no M for watermark, shadow or audio. Code: 6 for CelebA+MNIST and CLWD and FFHQ watermark; 8 for CelebA+CIFAR, CelebA+natural and ISTD; 3 for FFHQ and Imagenette shadow and Imagenette watermark; 6 for audio. See Paper vs code. celeba_cifar.yaml ffhq_shadow.yaml audio_piano.yaml
Image schedule σ_mlinear 0.4 to 0.1 over M steps§7.1 eq. 6Same in all image configs celeba_cifar.yaml; constant 0.8 and two-to-four-step schedules are compared in tab. 7
Audio schedule σ_mnot stated—Code: 0.32 to 0.08, i.e. 1.6 and 0.4 times σ_data = 0.2 audio_piano.yaml steps_audio.py
Stop threshold τ_stopnot stated in paperalg. 1Code: 0.01 for CelebA and CLWD, 0.2 for FFHQ, Imagenette and ISTD, 0.8 for audio. The file says these values were derived from saved iterates of the reported runs tau_stop.yaml tau_stop.yaml
InpaintingRePaint, DDIM steps§3Not stated in paper: code resamples every 10th ladder level twice steps.py steps.py
Residual refinementnot statedeq. 3Code: 64×64 crop, floor σ 0.02, 8 DDIM levels steps.py steps.py
Corruption class boxnot statedeq. 2Code: 0 ≤ n ≤ y for patches ("pos"), y−1 ≤ n ≤ y for watermarks ("free") steps.py celeba_cifar.yaml clwd.yaml
Shadow constantsnot statedeq. 9Code: offset 2/255, minimum gain 0.05 steps.py
Audio solvernot stated§4.2Code: init σ 0.32, 16 Karras steps, 8 residual steps, floor 0.02, pad to 8192 samples steps_audio.py
Images (benchmark)100 CelebA test images at 64×64; natural patches 32×32; CIFAR-10 patches with height and width 16 to 48 px§7.1Corruptions are additive: y = x + M⊙n
Watermark dataCLWD test: 40 logos, 99 images at 64×64, support ≈14% of image; FFHQ and Imagenette: 12 logos on 25 images at 256×256, support ≈3%§7.1Composite uses the dataset's per-pixel opacity §7.1
Shadow dataISTD: 100 test shadows at 64×64, support ≈17%; FFHQ and Imagenette: 25 images at 256×256 with ISTD shadow fields§7.1ISTD input is recomposed to match the unshadowed reference outside Ω §7.1
Audio data10 s clips at 22.05 kHz; interferer in one window covering half the clip; Slakh piano target; K=64§7.1 §4.2Default Slakh mixing gain g=1 without SIR normalisation §4.2
Clean-signal priorsCelebA score model trained on CelebA; EDM2 fine-tuned on CLWD clean train images (64²); off-the-shelf ADM for FFHQ and ImageNet 256²; EDM2 fine-tuned on shadow-free ISTD images; MSDM piano prior§4.1 §4.3 §4.4 §7.1 §4.2Code: sord/priors.py#L1-L13
DPS baseline steps250 reverse steps at 64²; 1000 at 256²; 250 Karras-Euler steps for audio§7.1Baselines only

Results

Author-reported. Each table shows mean ± standard deviation, with the best mean per row marked in the paper.

Localized image corruptions (PSNR in dB over Ω; 100 CelebA images at 64×64) tab. 1 §4.1:

CorruptionInputDPS InpaintDPS μ=μ_nDPS μ=0SGM, two priorsSGM, two priors, maskedSORD
MNIST digit12.09 ± 2.4719.85 ± 2.7711.06 ± 1.1811.91 ± 2.1725.78 ± 3.4222.51 ± 3.8927.16 ± 2.04
Natural patch9.37 ± 1.6216.84 ± 2.9415.74 ± 2.759.32 ± 1.5818.92 ± 7.1519.38 ± 6.3821.96 ± 2.32
CIFAR-10 patch7.92 ± 1.9017.28 ± 3.9214.87 ± 2.187.83 ± 1.8914.36 ± 3.9315.34 ± 4.1622.05 ± 3.44

Localized audio interference (SI-SDRi in dB over the window; piano target) tab. 1 §4.2. Speech (IV) uses MSDM with a speech prior; Speech (OOV) uses MSDM without one. See Pitfalls for an inconsistency between tab. 1 and tab. 9.

InterfererMixNDPS InpaintDPS μ=μ_nDPS μ=0MSDMSORD
IV instrumentSIR 060−0.34 ± 6.17−0.01 ± 0.05−0.06 ± 0.435.32 ± 7.641.12 ± 5.11
IV instrumentSIR 560−0.12 ± 5.670.00 ± 0.070.01 ± 0.091.40 ± 6.852.74 ± 6.23
IV instrumentg=1600.34 ± 0.92−0.02 ± 0.13−0.02 ± 0.126.48 ± 9.061.29 ± 2.58
Speech (IV)SIR 01601.02 ± 1.18−0.01 ± 0.100.00 ± 0.058.25 ± 8.645.51 ± 3.37
Speech (IV)SIR 51601.85 ± 2.02−0.03 ± 0.280.00 ± 0.052.00 ± 10.036.40 ± 3.39
Speech (IV)g=11600.94 ± 1.420.00 ± 0.020.00 ± 0.029.74 ± 9.733.74 ± 3.33
OOV instrumentSIR 0160−0.43 ± 17.01−1.14 ± 14.35−1.22 ± 15.50−0.24 ± 16.941.15 ± 17.54
OOV instrumentSIR 51600.03 ± 17.15−1.21 ± 15.50−1.13 ± 14.35−2.67 ± 18.391.97 ± 17.54
OOV instrumentg=1160−0.34 ± 17.08−1.23 ± 15.50−1.23 ± 15.50−1.54 ± 17.400.79 ± 17.41
Speech (OOV)SIR 01051.14 ± 1.35−0.01 ± 0.120.00 ± 0.063.57 ± 6.015.71 ± 3.60
Speech (OOV)SIR 51052.06 ± 2.29−0.04 ± 0.330.01 ± 0.04−0.05 ± 6.806.44 ± 3.40
Speech (OOV)g=11600.94 ± 1.420.00 ± 0.020.00 ± 0.024.15 ± 6.593.74 ± 3.33

Watermark removal (PSNR in dB over the support) tab. 2 §4.3. Baselines SLBR and PatchWiper are trained on paired data; SORD uses only the clean prior and the support.

BenchmarkInputDPS InpaintDPS μ=μ_nDPS μ=0SLBRPatchWiperSORD
FFHQ13.49 ± 3.2027.13 ± 3.8118.80 ± 5.1114.35 ± 3.8918.90 ± 5.1023.44 ± 5.5927.80 ± 3.50
CLWD22.55 ± 5.3317.14 ± 4.7424.26 ± 5.1522.73 ± 5.3031.55 ± 4.4731.37 ± 6.8425.09 ± 4.87
Imagenette13.64 ± 4.3024.81 ± 7.5619.63 ± 7.1814.56 ± 4.6121.66 ± 5.0821.79 ± 6.8926.43 ± 6.56

Shadow removal (PSNR in dB over the shadow support) tab. 2 §4.4. ShadowDiffusion and ShadowFormer are trained on paired shadow data.

BenchmarkInputDPS InpaintDPS μ=μ_nDPS μ=0ShadowDiffusionShadowFormerSORD
ISTD14.48 ± 3.7027.44 ± 7.0827.60 ± 4.2114.57 ± 3.7730.60 ± 4.9330.49 ± 5.1829.10 ± 5.55
FFHQ14.89 ± 4.1317.39 ± 3.8519.85 ± 3.8014.83 ± 4.0919.15 ± 3.2323.29 ± 2.8920.34 ± 4.19
Imagenette14.30 ± 4.7115.97 ± 6.5519.51 ± 4.9614.12 ± 4.5918.57 ± 2.8321.79 ± 4.3219.05 ± 6.50

Ablations and sensitivity (all author-reported):

  • Samples K tab. 3 §7.2: averaged over five settings, corrupted-region PSNR rises from 21.37 dB at K=2 to 24.84 dB at K=32, then saturates. K=32 is the default.
  • Refinement schedule tab. 7 §4.5: the 0.4 to 0.1 schedule with M=6 gives 21.96 / 27.16 / 22.05 dB (natural / MNIST / CIFAR-10). A constant 0.8 gives 20.50 / 24.41 / 20.24 dB. The three- and four-level schedules fall between these.
  • Diffusion residual step tab. 6 §3: removing the diffusion step from the residual lowers PSNR on the three CelebA benchmarks (21.96 to 20.16 natural; 27.16 to 24.28 MNIST; 22.05 to 20.40 CIFAR-10), but raises it on FFHQ watermark (26.68 to 29.22).
  • Diffusion steps in the x-step tab. 4 §7.3: capping at 8 or 4 DDIM levels changes most image results by 0.1 dB or less, the largest change being 0.57 dB (Imagenette shadow at 4 levels). The number of network calls falls (natural patch: 1640 to 840 per image).
  • Mask errors tab. 5 §7.4 fig. 4: dilating the support by 3 px costs 2 dB on natural patches (21.96 to 19.94); eroding it by 1 px costs 8 dB (to 13.84). On CLWD, eroding by 1 px matches or slightly exceeds the exact mask (25.47 vs 25.09).
  • Support and contaminant model tab. 8 §4.5: on MNIST, SORD with the exact support scores 27.16 dB, against 20.29 dB without a mask; SGM with two priors scores 25.64 dB without a mask and 22.38 dB with the exact one.

Paper figures: Figure 1 (teaser, §1), Figure 2 (visual results, §4.1), Figure 3 (audio spectrograms, §7), Figure 4 (mask error examples, §7.4).

Paper vs code

Repository: https://github.com/voilalab/SORD, cloned at commit 7b9575e6a4c81aeb44f7a62526c55329cba8f48e. The link is given in the paper's abstract. The repository has no LICENSE file. Its README says the manuscript is under double-blind review.

  • Residual per sample vs one residual per image. Algorithm 1 forms one residual for each sample k alg. 1 eq. 2. The code forms one residual from the mean reconstruction and shares its correction across all K samples algorithm.py steps.py steps.py.
  • Residual refinement. Eq. 3 denoises the raw residual at scale σ*. The code crops the support's bounding box, resizes it to 64×64, normalises it to mean 0.5 and standard deviation 0.25, denoises it at the ladder level nearest max(σ*, 0.02) with 8 DDIM levels, maps it back and clips it to the class box steps.py steps.py. None of these steps is described in the paper.
  • Class box. The code clips each corruption estimate to its class bounds. These are 0 ≤ n ≤ y for patches and y−1 ≤ n ≤ y for watermarks, set per config steps.py celeba_cifar.yaml clwd.yaml. Eq. 2 has no clip.
  • x-step. Eq. 6 shows one denoising call at σ_m. The code runs DDIM over every ladder level from σ_m downward, and clips the samples to [0, 1] steps.py. The paper's Table 4 caption says this multi-level DDIM run is the "shipped" variant behind the main results tab. 4.
  • Inpainting. The paper names RePaint. The code adds resampling every 10th ladder level, twice steps.py. The paper gives no such values.
  • Variance. The code uses unbiased variance (ddof=1) and averages it over colour channels steps.py. The paper does not specify either.
  • Iterations. Paper: M=6 for the CelebA benchmarks §7.1. Code: M=8 for CelebA+CIFAR and CelebA+natural celeba_cifar.yaml celeba_natural.yaml. The config comments say "as in the reported run", so the paper's stated M may not match its reported runs.
  • Stop threshold. Not stated in the paper. The code uses 0.01, 0.2 or 0.8 by task tau_stop.yaml. The file says these were derived from saved iterates of the reported runs. The method therefore has a threshold that was chosen on the evaluation runs.
  • Stop position. The repository's implementation notes say the code's test differs from the manuscript's. In the code I read, the stop returns the previous iterate, as Algorithm 1 prints it algorithm.py. IMPLEMENTATION_NOTES.md lists several differences, including pooling uncertainty over chunks of 25 images in the paper's experiments, where the code computes it per image IMPLEMENTATION_NOTES.md.
  • Audio. The audio code uses its own constants, a 16-step Karras solver, normalisation by the window's standard deviation and no class box steps_audio.py steps_audio.py. The paper gives none of these.
  • Not verified. I did not check the baselines (DPS, SGM, MSDM, PatchWiper, SLBR, ShadowFormer, ShadowDiffusion) against their own repositories. I did not check how the benchmark files were built, and the CLWD support mask construction did not show the opacity logic I looked for.

Pitfalls

  • Support must cover the corruption. Under-covering is much worse than over-covering. Eroding the mask by 1 px drops natural-patch PSNR from 21.96 to 13.84 dB tab. 5 §5. The paper's advice is to over-cover when the boundary is uncertain §5 §7.4.
  • Iterations and thresholds. The code's per-task M differs from the paper's single M=6 (see Paper vs code). The paper gives no stop threshold. The code's values (0.01, 0.2 or 0.8) were chosen on the reported runs tau_stop.yaml, so choose a threshold on held-out data rather than copying them.
  • Clipping and normalisation are undisclosed. The residual is cropped, resized and normalised before the prior sees it, and its estimate is clipped to a class box. These steps are needed to reproduce the reported numbers and are absent from the paper steps.py steps.py.
  • Multi-level x-step vs the single denoise in Eq. 6. Table 4 says the main results use the multi-level DDIM descent, so a single denoising call as in Eq. 6 is not the variant the main results report tab. 4 steps.py.
  • ISTD evaluation input. The ISTD input is recomposed so that it equals the unshadowed reference outside Ω §7.1. The method is therefore scored where the outside-support pixels are already correct, and the paper notes that the two ISTD images come from separate captures with exposure differences §4.4. Real-world shadow removal is harder than these numbers suggest.
  • Watermark opacity is not given to the method. The paper deliberately withholds the pixel-wise α at inference, so model mismatch is part of the result §4.3. A few ablation rows use the true α ("declared α") and show higher scores tab. 6.
  • Audio results are mixed and high-variance. MSDM, which uses per-source priors, beats SORD on several instrument rows and on speech-IV SIR 0 (8.25 vs 5.51 dB) tab. 1 §4.2. Standard deviations are often larger than the means, so one sample is not a stable comparison.
  • Table 9 and Table 1 disagree on Speech (OOV). Table 9's Speech (OOV) rows repeat the Speech (IV) values for DPS-Inpaint, both DPS likelihood variants and SORD at SIR 0 and SIR 5, and list N=110 where Table 1 gives N=105. At SIR 0, Table 1 gives SORD 5.71 dB and Table 9 gives 5.51 dB tab. 1 tab. 9. I could not determine which is correct.
  • Oracle baselines. The DPS μ=μ_n variants use the true per-image mean of the corruption, which is unavailable in practice tab. 1 §7.1. Compare SORD against DPS μ=0, the realistic variant.
  • Metric. PSNR is computed over Ω only, because full-image metrics are dominated by unchanged pixels §4. Report corrupted-region metrics the same way.
  • Cost. The inner loop runs K·M·(levels) denoiser calls per image. The paper reports 129 to 1,640 network evaluations per image for image benchmarks, excluding initialisation tab. 4. Audio cost is not stated.
  • Scale parameters. The shadow model clamps the gain to at least 0.05 steps.py, and the residual is normalised to a fixed contrast steps.py. Neither is in the paper.
  • Version. The paper is a 2026 preprint under review, and the code notes say numbers should be rechecked against the exact source revision before publication.

Lineage

  • Builds on: RePaint for inpainting initialisation §3; DDIM as the denoiser §3; DPS, whose Gaussian-likelihood setting the paper tests as baselines §4.1; the two-prior SGM of Stevens et al. §4.1; and MSDM for audio §4.2. SORD changes the setting: it drops the corruption prior and keeps only a known support.
  • Compared against: DPS (three variants); SGM with one or two priors (with and without the mask); MSDM (audio); SLBR and PatchWiper (watermarks); ShadowDiffusion and ShadowFormer (shadows) §4 tab. 1 tab. 2.
  • Superseded by: not known from the paper.

Source and license

Adapted from “Support-guided OOD Removal with Diffusion” by Alireza Kheirandish, Sara Fridovich-Keil (arXiv:2610.33194), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/voilalab/SORD at commit 7b9575e6a4c8 (none (no LICENSE file in repository)). Any quoted code is shown under that license.