Learning a Dilated Residual Network for SAR Image Despeckling

In plain words

Radar pictures taken from satellites or planes look grainy, because the radar signal bounces back in a random, speckled way. That grain hides the real shapes on the ground. Older clean-up methods either blur fine detail or leave patches of grain behind.

This paper trains a small neural network to do the clean-up. The network does not try to draw the clean picture from scratch. It estimates only the grain, and the grain is then subtracted from the noisy picture. The authors argue that learning the grain is an easier job for the network than learning the whole image.

The network looks at a wide area at once without getting large. It does this by spacing out the pixels its filters read, a trick called dilation. Shortcut connections let the early layers' information reach later layers unchanged, which keeps small details and makes training easier.

The authors trained it on ordinary optical photos to which they added simulated radar grain, because clean radar pictures are hard to get. They then tested it on simulated radar images and on three real radar scenes.

TL;DR

• Predicts the multiplicative speckle component of a SAR image with a 7-layer dilated residual CNN and subtracts it from the input §3, eq. 16, fig. 1.
• Uses dilations 1, 2, 3, 4, 3, 2, 1, two skip connections, ReLU, and no batch normalisation tab. 1, fig. 1, §4.4.
• Trained with Adam on 40x40 patches from simulated 1-, 2-, 4- and 8-look images, one model per look count §4.1, §4.4.
• Author-reported: best PSNR in all 12 simulated settings and 9 of 12 SSIM values against four baselines tab. 2-tab. 4, §4.2.
• No official training code is in the repository; the architecture here is taken from the paper alone utils.py.

Problem

Speckle in SAR amplitude images is multiplicative and signal-dependent, so linear filters trade homogeneous-area smoothing against edge preservation and often blur or erase point targets §1. Non-local and variational methods improve detail but are slow or depend on hand-tuned parameters and noise models §1. The authors aim for a non-linear, learned mapping that handles strong speckle, especially at few looks §1, §4.2.

Assumptions: the speckle is modelled as multiplicative with a Gamma-distributed factor whose shape is the number of looks L eq. 1, eq. 2. Clean reference SAR images are assumed unavailable, so training uses simulated speckle on optical images §4.1.

Method

Noise model. The speckled image y is the clean image x times a speckle factor n eq. 1. For amplitude SAR, n follows the Gamma density in eq. 2, with L the equivalent number of looks. The ENL index of a homogeneous region is its mean divided by its variance eq. 3 and is used for evaluation §4.3.

Dilated convolutions §3.1. The network replaces the usual 3x3 kernels with dilated ones: the kernel taps are spread out by a dilation factor d eq. 13, so the receptive field grows without more weights fig. 2. The paper's receptive-field growth rule for doubling dilations is in running text after eq. 14. The network used here does not double dilations, so that rule does not apply to it; see Pitfalls.

Layer stack tab. 1, fig. 1, fig. 3:

1. Layer 1: 64 filters of 3x3, dilation 1, ReLU.
2. Layers 2 and 3: 64 filters of 3x3, dilations 2 and 3, ReLU.
3. Layers 4, 5 and 6: 64 filters of 3x3, dilations 4, 3 and 2, ReLU.
4. Layer 7: 1 filter of 3x3, dilation 1, no ReLU listed.
5. Padding equals the dilation, so spatial size is preserved tab. 1.

Skip connections §3.2, eq. 15, fig. 1, fig. 4. Two additive shortcuts are used, one around layers 2 and 3 and one around layers 5 and 6. Each adds an earlier 64-channel feature map to a later one. Their exact tap points are ambiguous; see Pitfalls.

Residual learning §3.3. The target residual is the speckle component: φ = y − x eq. 16. The network outputs its estimate φ(y; θ), and the denoised image is y minus that estimate fig. 1, fig. 5.

Loss §3.3, eq. 17: mean squared error between predicted and target residuals, averaged over N training pairs and scaled by 1/(2N).

Optimiser §4.1, eq. 18-eq. 20: Adam with the update below.

# Adam update as written in the paper  [eq. 18] [eq. 19] [eq. 20]
# Input:  params, grad, first_moment, second_moment, step, learning_rate
# Constants: beta_1 = 0.9, beta_2 = 0.999, eps = 1e-8  [§4.1]

def adam_step(params, grad, first_moment, second_moment, learning_rate):
    # Running mean of gradients  [eq. 18]
    first_moment = beta_1 * first_moment + (1 - beta_1) * grad
    # Running mean of squared gradients  [eq. 19]
    second_moment = beta_2 * second_moment + (1 - beta_2) * grad ** 2
    # Parameter update; eps sits inside the square root  [eq. 20]
    params = params - learning_rate * first_moment / sqrt(second_moment + eps)
    return params, first_moment, second_moment

Forward pass (own rendering of fig. 1 and tab. 1, channel count of the input assumed to be one, see Pitfalls):

# SAR-DRN forward pass  [§3] [fig. 1] [tab. 1]
# Input:  noisy, speckled patch (B, 1, H, W)
# Output: denoised estimate (B, 1, H, W)

def sar_drn(noisy):
    # Layer 1: dilation 1, ReLU  [tab. 1]
    feat_1 = relu(conv3x3(noisy, dilation=1, channels=64))
    # Layers 2 and 3: dilations 2 and 3, ReLU  [tab. 1]
    feat_2 = relu(conv3x3(feat_1, dilation=2, channels=64))
    feat_3 = relu(conv3x3(feat_2, dilation=3, channels=64))
    # First skip: layer 1 output added to layer 3 output  [§3.2] [fig. 4a]
    block_1 = feat_3 + feat_1
    # Layers 4 to 6: dilations 4, 3, 2, ReLU  [tab. 1]
    feat_4 = relu(conv3x3(block_1, dilation=4, channels=64))
    feat_5 = relu(conv3x3(feat_4, dilation=3, channels=64))
    feat_6 = relu(conv3x3(feat_5, dilation=2, channels=64))
    # Second skip: layer 4 output added to layer 6 output  [§3.2] [fig. 1]
    block_2 = feat_6 + feat_4
    # Layer 7: one output channel, dilation 1, no ReLU  [tab. 1]
    speckle_estimate = conv3x3(block_2, dilation=1, channels=1)
    # Residual learning: subtract the predicted residual  [§3.3] [fig. 1]
    return noisy - speckle_estimate

Training loop §4.1, eq. 16, eq. 17:

# Training with simulated speckle  [§4.1]
# Input:  clean_images (UC Merced, 400 images of 256x256), looks L
# Constants: patch 40x40, stride 10, batch 128, 50 epochs

def train(clean_images, looks):
    # Cut overlapping training patches  [§4.1]
    patches = crop_patches(clean_images, size=40, stride=10)
    for epoch in range(50):
        # Halve the learning rate every 10 epochs (gamma = 0.5)  [§4.1]
        for batch in shuffled_batches(patches, size=128):
            # Simulate speckle with L looks  [§4.1] [eq. 1] [eq. 2]
            noisy = batch * sample_gamma_speckle(looks, batch.shape)
            # Target residual: noisy minus clean  [eq. 16]
            target = noisy - batch
            # Forward pass and loss  [eq. 17]
            loss = mse_half(sar_drn(noisy) - target)
            # Adam update  [eq. 18] [eq. 19] [eq. 20]
            update_parameters(loss)

Hyperparameters

Results

Author-reported. Setup: UC Merced patches for training, three simulated test images at L = 1, 2, 4 and 8, and three real AIRSAR scenes (4-look) §4.1, §4.2, §4.3. Baselines: PPB [13], SAR-BM3D [14], SAR-POTDF [16] and SAR-CNN [28]. Values are mean ± standard deviation over 10 simulations tab. 2. Bold marks the best value as in the source tables.

Simulated: Airplane tab. 2

Simulated: Building tab. 3

Simulated: Highway tab. 4

The text states the gains at L = 1 over SAR-BM3D are about 1.1, 0.6 and 0.6 dB for Airplane, Building and Highway §4.2; the tables agree tab. 2-tab. 4.

Real images, ENL tab. 5 (regions chosen by the authors, §4.3)

Real images, EPD-ROA tab. 6

Runtime for a 256x256 image, MATLAB R2014b tab. 7 (hardware not stated for each method)

Ablations fig. 12, fig. 13, evaluated on Set14 with 1-look images §4.4. Values are read from the plots, so they are approximate:

• With and without dilated convolutions and skip connections: the full model has the lowest training loss and highest PSNR; the text states it is about 1.1 dB above the model with neither §4.4.
• Without batch normalisation the PSNR curve ends higher than with it, and training is about 3 hours shorter §4.4, fig. 13.

Qualitative results: described in the text, not tabulated. SAR-DRN is stated to keep local detail and avoid over-smoothing and texture artefacts compared with the baselines §4.2, §4.3, fig. 6-fig. 11.

Paper vs code

The official repository is qzhang95/SAR-DRN at the commit above. Its owner is the first author (Qiang Zhang, from GitHub profile), and its README cites this arXiv id. The paper itself does not link the code, so the identification is based on owner and README. Searches: the method name (SAR-DRN) and despeckling dilated residual on the GitHub API; the first hit matches.

The repository contains only README.md and utils.py README.md, utils.py. It has no LICENSE file, so its license is not stated, and the card does not quote any code.

• No model or training code in the repository. The architecture, dilations, skip connections, optimiser and loss cannot be checked against code. All Method and Hyperparameters values are from the paper only.
• Augmentation. utils.py defines eight dihedral augmentations (flips and 90-degree rotations) utils.py. The paper does not mention augmentation §4.1. Whether training used it is not confirmed.
• Initialisation. utils.py applies Kaiming normal initialisation (fan-in) to layers whose class name contains "Conv" or "Linear", and a small-variance initialisation to BatchNorm layers utils.py. The paper states no initialisation §4.1. The BatchNorm branch does not indicate that the model uses BatchNorm, which the paper removes §4.4.
• PSNR. The helper computes PSNR with skimage compare_psnr and takes a data_range argument utils.py. The data range used in the paper is not stated.

No other discrepancies can be checked, since the model and training code are absent from the repository.

Pitfalls

• Output definition. Section 3.3 says the network predicts "the speckled image" §3.3, but eq. 16 defines the target residual as y minus x, and fig. 1 subtracts the network output from the input. Implement the network as predicting the speckle, train against y minus x, and output y minus the prediction.
• Skip tap points. The text says the second skip runs from layer 4 to layer 7 §3.2. fig. 1 and fig. 4b place its addition before layer 7, after layer 6. Layer 7 outputs one channel, so the skip cannot end at layer 7's output. The card's reading (layer 4 output added to layer 6 output) follows the figures; this is an inference. The first skip is drawn from the input of layer 1 in fig. 1 but from the output of layer 1 in fig. 4a; the 64-channel input in fig. 4a suggests the output of layer 1. Both are unconfirmed.
• Receptive field. The formula in running text after eq. 14 gives (2^(l+1) - 1)^2 for a stack of doubling dilations §3.1. The network's dilations are 1, 2, 3, 4, 3, 2, 1 tab. 1, so the formula does not describe it. Derived here, not stated: each 3x3 layer with dilation d adds 2d to the receptive field, so the total is 1 + 2 * 16 = 33, i.e. 33x33.
• Iteration count. The text says 50 epochs and "about 1,500 iterations" §4.1. At batch size 128 and 193,664 patches one epoch is about 1,513 iterations, so 50 epochs would be about 75,000. Figures 12 and 13 span about 5 x 10^4 iterations fig. 12, fig. 13. Which figure is correct is not stated.
• Patch count. A 40x40 window with stride 10 on a 256x256 image gives 22 positions per axis, or 484 patches per image §4.1. For 400 images that is 193,600, against the 193,664 stated §4.1. Not reproducible exactly from the stated parameters.
• Adam. eq. 18-eq. 20 show no bias-correction terms and place eps inside the square root. Whether the Caffe implementation used bias correction is not stated.
• Loss scale. eq. 17 uses 1/(2N) times a squared L2 norm over each patch. Whether this is summed over pixels or averaged, and how the batch is reduced, is not stated. This changes the effective learning rate.
• Speckle simulation. eq. 2 with mean 1 implies the speckle factor is Gamma-distributed with shape L and rate L. The sampling method is not stated. Section 4.1 says optical amplitudes are "processed" before training to match SAR statistics but gives no procedure §4.1.
• Input channels and range. A single channel is assumed for the SAR amplitude input (inferred). The value range, normalisation, and clipping are not stated.
• Full-image testing. Training uses 40x40 patches. Whether test images are processed whole is not stated; the fully convolutional structure suggests so (inferred).
• One model per look count. The paper trains a model for each number of looks §4.4. A single blind model is not evaluated.
• Naming mismatch. Section 4.1 names the simulated classes "Airplanes, Buildings and Rivers", while the tables and figures use Airplane, Building and Highway §4.1, tab. 2-tab. 4.
• Ablation data. Figures 12 and 13 use Set14, a natural-image dataset, not SAR images §4.4.
• Real-data metrics. ENL is computed on four hand-picked homogeneous regions §4.3. Higher ENL means smoother homogeneous areas, but it can rise through over-smoothing, so it should be read together with EPD-ROA §4.3, tab. 6.
• Runtime comparison. The hardware for each method in tab. 7 and whether SAR-DRN ran on GPU are not stated. The 0.38 s figure is not comparable without that.
• Parameter count. Not stated. The paper calls the model lightweight §1, §3.

Lineage

• Builds on: dilated convolutions §3.1 (source cited as [33]), skip connections §3.2 (source cited as [34]), and residual learning §3.3 (source cited as [35]). These are referenced by number in the paper only.
• Uses the DnCNN idea of learning a residual for denoising §2.2 (source cited as [26]).
• Compared against: PPB [13], SAR-BM3D [14], SAR-POTDF [16], SAR-CNN [28] §4.2.
• Superseded by: not stated in this card.