Image denoising

Overview

Image denoising takes a noisy image and returns a clean estimate of the same scene. Three settings appear in the cards. In the synthetic Gaussian setting, Gaussian noise of a known standard deviation σ (on a 0-255 scale) is added to clean images, and the model is trained and tested on that noise 2402.15648 §4.1 train_MambaIR_ColorDN_level15.yml. RIDNet also reports Gaussian results, but its card does not state the noise levels used in training 1904.07396 §4.1. In the real setting, the noisy input is a camera image and the clean target comes from a paired capture 2402.15648 §4.4 2204.13348 §4.4. In the Poisson setting, photon noise is added to DIV2K images and a network is fitted to each noisy image on its own; the card describes no separate training set 2301.05187 §4.2.

Five paper cards cover this concept so far. MambaIR and RIDNet each cover the Gaussian and real settings. GRL covers only the Gaussian setting, with no camera results. DGUNet covers only the real setting, and its card has no Gaussian results. WIRE covers only the Poisson setting, and its card has no Gaussian or camera results. Other denoising methods are not compared here, and the scores from the four cards come from different papers' tables and protocols. WIRE's denoising numbers are plotted, not stated in its text.

Variants

Gaussian color denoising at known σ (MambaIR)

The network is a restoration backbone that takes the noisy image as input and adds a convolution output back onto it, so the model learns the residual mambair_arch.py. It is trained for σ = 15, 25 and 50 2402.15648 tab. 5. The σ15 model is trained from scratch; the σ25 and σ50 models start from σ15 weights 2402.15648 §4.1 train_MambaIR_ColorDN_level50.yml. Card: MambaIR: A Simple Baseline for Image Restoration with State-Space Model.

On the paper's own table, MambaIR scores highest on BSD68 and Urban100 at all three σ values, and on McMaster at all three 2402.15648 tab. 5. Its Urban100 σ50 score is 30.30 dB against 30.02 dB for Restormer and 29.82 dB for SwinIR. On Kodak24, Restormer is higher at all three σ values, for example 35.47 dB against 35.42 dB at σ15 2402.15648 tab. 5. All numbers are author-reported.

Gaussian colour denoising with windowed and stripe attention (GRL)

GRL splits each transformer layer into three parallel branches: window attention on half of the channels, anchored stripe attention on the other half, and a channel-attention convolution for local detail 2303.00748 §4 fig. 6. The stripe branch attends to a small set of pooled anchors rather than to every token, which keeps its cost down 2303.00748 §3.2. The paper gives three sizes. On its colour denoising table, GRL-T has 0.88 M parameters, GRL-S 3.12 M and GRL-B 19.81 M 2303.00748 tab. 4. On CBSD68 at σ 15, 25 and 50, GRL-B scores 34.45, 31.82 and 28.62 dB, and GRL-T scores 34.30, 31.66 and 28.45 dB 2303.00748 tab. 4. On Urban100, GRL-S scores 35.24, 33.07 and 30.09 dB, above the 35.13, 32.96 and 30.02 dB that Restormer reports, with 3.12 M against Restormer's 26.13 M 2303.00748 tab. 4. All numbers are author-reported. The card has no camera-noise results for GRL. The released denoising config trains GRL-T, not GRL-B, with L1 loss, AdamW at 1e-4 for 225k steps and 256-pixel patches grl_p256.yaml grl_p256.yaml. Card: GRL: Efficient and Explicit Modelling of Image Hierarchies for Image Restoration.

Real-image denoising with MambaIR (SIDD and DND)

The paper describes the same architecture for real denoising, trained with progressive patch sizes that follow Restormer, with the details not stated 2402.15648 §4.4. The reported scores are close to the strongest baselines. On SIDD, MambaIR reaches 39.89 dB against 40.02 dB for Restormer; on DND it reaches 40.04 dB against 40.03 dB for Restormer [arxiv:2402.15648 tab. 6]. The released real-denoising config uses a different U-Net model and optimiser (see Contradictions) train_MambaIR_RealDN.yml.

Real-image denoising with an unfolded network (DGUNet)

DGUNet unfolds seven stages of proximal gradient descent. Each stage has a gradient module and an informative proximal mapping 2204.13348 §3.2. For denoising, the paper trains on 320 SIDD pairs and tests on SIDD and DND 2204.13348 §4.4. DGUNet scores 39.88 dB on SIDD and 40.04 dB on DND. DGUNet+, which makes every stage independent, scores 39.91 dB and 40.12 dB 2204.13348 tab. 3. The text puts DGUNet+ 0.20 dB above MPRNet on SIDD and 0.32 dB above it on DND 2204.13348 §4.4. The paper does not say which of its two gradient branches the denoising models use [arxiv:2204.13348 §3.2]. Card: Deep Generalized Unfolding Networks for Image Restoration.

Blind real-image denoising in one pass (RIDNet)

RIDNet is a single-stage denoiser. It predicts the residual, which is added back to the noisy input, and its body has four enhancement attention modules, each ending in a channel gate 1904.07396 §3.2 1904.07396 §3.2.1. It is blind, so it needs no noise level set by hand 1904.07396 §1. On DnD sRGB, blind, it reports 39.23 dB and 0.9526 SSIM, against 38.06 dB for CBDNet 1904.07396 tab. 5. On the Nam and SSID real sets it reports 39.09 dB and 38.71 dB; the SSID figure uses the validation split 1904.07396 tab. 6 1904.07396 §4.3.1. On Gaussian data it reports 34.01, 31.37 and 28.14 dB for colour BSD68 at σ15, σ25 and σ50 1904.07396 tab. 4. The card gives its inference time as about 0.2 s for a 512 × 512 image, on hardware not stated 1904.07396 §3.3. The official repository has only a test model and no licence file, so training cannot be reproduced from it README.md. Card: Real Image Denoising with Feature Attention.

Poisson denoising by fitting one image (WIRE)

WIRE is an implicit neural representation: a small MLP that maps coordinates to values, with a complex Gabor wavelet as its activation 2301.05187 §3.2 eq. 2. For denoising, the noisy image is the only data. The input has Poisson noise with a maximum mean of 30 photons per pixel and a readout count of 2, which gives an input PSNR of 17.6 dB. The denoising MLP has two hidden layers of 256 features 2301.05187 §4.2. The network is fitted to the noisy measurements with Adam and an l2 loss, with no other regularisation 2301.05187 §4. The paper's text calls the denoised result the highest of the compared implicit networks, and qualitatively close to deep image prior (DIP). The bar values are plotted, not stated 2301.05187 §4.2 fig. 6. It reports denoising reaching a target PSNR about an order of magnitude sooner in epochs than other implicit networks, on a Kodak experiment 2301.05187 §3.3. The paper does not state the denoising w0 and s0; its photon sweep is in 2301.05187 §3.4 fig. 3. Card: WIRE: Wavelet Implicit Neural Representations.

When to use which

Implementation sketch

Pseudocode for the Gaussian colour model, with the residual output that the code uses for denoising. Layers are shown at a high level.

# Denoise a noisy RGB image with a known Gaussian sigma
# (sigma is on the 0-255 scale; training noise is sigma/255 on [0, 1] images)
# [arxiv:2402.15648 §4.1] [code: basicsr/data/paired_image_dataset.py#L105-L112]

def denoise(noisy):                       # noisy: (3, H, W) in [0, 1]
    # shallow 3x3 conv to C channels      [arxiv:2402.15648 §3.2]
    feat = conv3x3(noisy, out=C)
    # stacked residual state-space groups, each with its own residual
    # [arxiv:2402.15648 §3.2] [code: basicsr/archs/mambair_arch.py#L602-L606]
    deep = residual_state_space_groups(feat)
    # final conv, then the long skip from shallow features
    # [arxiv:2402.15648 §3.2]
    feat = conv3x3(deep) + feat
    # the output is the noisy input plus a learned residual
    # [code: basicsr/archs/mambair_arch.py#L679-L683]
    return noisy + conv3x3(feat, out=3)

# Training loss: Charbonnier against the clean target, not against the input
# [arxiv:2402.15648 §3.6] [code: basicsr/losses/losses.py#L109-L116]
def loss(pred, clean):
    diff = pred - clean
    # paper: sqrt(diff^2 + eps^2), eps = 1e-3 [arxiv:2402.15648 eq. 9]
    # code:  sqrt(diff^2 + eps) per element [code: basicsr/losses/losses.py#L24-L26]
    return mean(sqrt(diff * diff + EPS))

DGUNet has a different structure. It runs seven stages, each a gradient step followed by a proximal step, passes features between stages, and sums the error over all stage outputs 2204.13348 §3.2 2204.13348 eq. 10. Its denoising code, however, trains with Charbonnier loss only (see Contradictions).

Contradictions

• Charbonnier term. The paper writes sqrt(‖d‖² + ε²) with ε = 1e-3 2402.15648 eq. 9. The code computes sqrt(d² + eps) per element with eps = 1e-3, which has an effective ε of about 0.032 losses.py. The paper's loss equations also compare the output with the input; the card notes the target should be the ground truth 2402.15648 §3.6.
• Real-denoising architecture. The paper describes one architecture for both Gaussian and real denoising 2402.15648 §3.6. The released real-denoising config uses a different U-Net (MambaIRUNet, dim 48), L1 loss, AdamW, gradient clipping and mixup, with progressive patches from 128 to 384 over 300k iterations train_MambaIR_RealDN.yml train_MambaIR_RealDN.yml. The paper does not say which of the two produced its table scores.
• Colour metric. MambaIR reports Y-channel PSNR for every table 2402.15648 §4.1. The colour-denoising test configs compute RGB PSNR with test_y_channel: false test_MambaIR_ColorDN_level15.yml. DGUNet lists PSNR and SSIM without naming a colour channel 2204.13348 §4.1. The two papers' SIDD and DND scores may therefore use different protocols, and neither paper's table includes the other's model 2402.15648 tab. 6 2204.13348 tab. 3.
• Gaussian colour tables, MambaIR against GRL. Both papers list SwinIR and Restormer with identical numbers, which suggests they share a protocol for those rows 2402.15648 tab. 5 2303.00748 tab. 4. Neither card states the protocol, so this is an inference from the numbers. On BSD68, MambaIR is higher than GRL-B at all three σ values: 34.48 against 34.45 dB at σ15, 32.24 against 31.82 dB at σ25, and 28.66 against 28.62 dB at σ50 2402.15648 tab. 5 2303.00748 tab. 4. On Kodak24, McMaster and Urban100, GRL-B is higher at all three σ values, for example 35.54 against 35.37 dB on Urban100 at σ15 2303.00748 tab. 4 2402.15648 tab. 5. GRL-B does not top every column: Restormer is higher on Kodak24 at σ15 (35.47 against 35.43 dB) 2303.00748 tab. 4.
• RIDNet on DnD and SSID. The RIDNet card gives 39.23 dB on DnD 1904.07396 tab. 5. DGUNet's table lists RIDNet at 39.26 dB on DND 2204.13348 tab. 3. The RIDNet card's SSID figures, 38.71 dB for RIDNet and 30.78 dB for CBDNet, are the same numbers that DGUNet's table gives under SIDD 1904.07396 tab. 6 2204.13348 tab. 3. The RIDNet card uses SSID's validation split, and the DGUNet card does not say which split its SIDD column uses. The cards do not settle which dataset or split each column means.
• BSD68 colour, RIDNet against MambaIR. RIDNet's colour BSD68 row gives 34.01 dB at σ15 1904.07396 tab. 4. MambaIR's table gives 34.48 dB 2402.15648 tab. 5. The RIDNet card does not name a colour channel, while MambaIR reports Y-channel PSNR 2402.15648 §4.1. The two rows are not a like-for-like comparison. The DnCNN rows also differ: 33.89 dB at σ15 in RIDNet's table 1904.07396 tab. 4 against 33.90 dB in MambaIR's 2402.15648 tab. 5. The IRCNN and FFDNet rows match in both tables.
• Denoising loss in DGUNet. The paper's stated loss is an l2 error summed over stages, and only the compressive-sensing code matches it 2204.13348 §3.3 2204.13348 eq. 10. The denoising code uses Charbonnier loss only, with the output clamped to [0, 1] train.py.
• WIRE photon count. The paper gives a maximum of 30 photons per pixel for the denoising input 2301.05187 §4.2, and 50 in its sweep 2301.05187 §3.4. The denoising script uses 30 wire_image_denoise.py.

Papers

• MambaIR: A Simple Baseline for Image Restoration with State-Space Model: a restoration backbone with a four-direction Mamba scan; reports Gaussian colour denoising at σ15/25/50 and real denoising on SIDD and DND.
• Deep Generalized Unfolding Networks for Image Restoration: a seven-stage unfolded network with learned gradient and proximal modules; reports real denoising on SIDD and DND, alongside deraining, deblurring and compressive sensing.
• Real Image Denoising with Feature Attention: a single-stage blind denoiser with four enhancement attention modules and feature attention; reports DnD, Nam and SSID for real noise, and BSD68 and classical images for Gaussian noise.
• WIRE: Wavelet Implicit Neural Representations: an implicit network with a complex Gabor wavelet activation; reports per-image denoising of Poisson-noisy images, alongside fitting, super-resolution, CT and NeRF.
• GRL: Efficient and Explicit Modelling of Image Hierarchies for Image Restoration: a transformer whose layers run window attention, anchored stripe attention and a channel-attention convolution in parallel; reports Gaussian colour denoising at three sizes, alongside six other restoration tasks.