Data-consistency layer

Overview

A learned reconstruction gets a set of measurements y and must output an image that agrees with them. A network trained alone can drift away from the measurements, because nothing in its forward pass forces agreement with the data. The data-consistency (DC) layer is a fixed, differentiable step placed after each network block. It transforms the output into k-space and combines it with the entries that were actually measured. The invariant is that the measured entries of the output equal the measurements, or sit close to them when the measurements are noisy 1704.02422 §2 1704.02422 §3.

In the paper's setting, k-space is the Fourier transform of the image, and the measurements are a masked subset of it. The paper argues that a plain CNN from the aliased image ignores the measured data unless the data term is part of the learning 1704.02422 §2. The DC layer supplies that term without adding learned parameters in the noise-free case 1704.02422 §3.

A second family enforces the same idea through the loss instead of the architecture. In a GAN-based reconstruction, the adversarial loss alone does not force the generator to respect the measurements, so a cyclic term is added that compares the generator's output with them 1709.00753 §III-D.

Variants

Noise-free hard replacement

On the sampled set Ω, the layer sets the k-space value to the measured value. Off Ω it keeps the network output. This is the limit of the weighted form as the weight λ goes to infinity 1704.02422 §3. The paper uses this limit for its noise-free experiments 1704.02422 §3. Card: Schlemper et al. 2017, cascaded CNN for dynamic cardiac MRI.

Noisy soft data consistency

When the measurements carry noise, the paper uses a finite weight λ. On Ω the output is a convex combination of the network value and the measurement, with weights 1/(1+λ) and λ/(1+λ) 1704.02422 §3, eq. 6. The paper sets λ = q/σ, where σ is the noise level, starts λ at 0.025, and makes it trainable 1704.02422 §3. In its noise experiments λ ends near 0.5 after fine-tuning 1704.02422 §7.3.3. The paper gives the training range of σ² but no numbers for q or σ 1704.02422 §7.3.3.

Loss-based data consistency (cyclic loss in a GAN)

A GAN generator is trained with an adversarial loss plus a cyclic term. The cyclic term has two parts. The first takes the generator's output for the measured samples, under-samples it again with the same mask, and compares the result with the measurements. This is the data-consistency part. The second applies the same round trip to fully sampled images [arxiv:1709.00753 §III-D, eq. 6]. The distance d in the first part may be MSE, MAE or another pixel-wise distance. The paper does not say which one its experiments use 1709.00753 §III-D. The weights are fixed at α = 1.0 for the first part and γ = 10.0 for the second, for all experiments [arxiv:1709.00753 §III-F, eq. 7]. The authors report that the cyclic loss raised PSNR by about 1 dB over the single-generator baseline. This is a text-only figure in the paper 1709.00753 §IV-A. Card: Compressed Sensing MRI Reconstruction using a GAN with a Cyclic Loss.

Cascade with data-sharing (dynamic sequences)

For a time series, the paper pairs the DC layer with a data-sharing layer. For each frame it averages the sampled k-space of neighbouring frames in a window, and fills only the unsampled locations with that average. The averages enter the next block as extra input channels 1704.02422 §5 1704.02422 §6. The layer applies to sequences only, not to 2D frames 1704.02422 §6. The paper reports lower error with data sharing than without it at 9-fold undersampling in its 3D setting 1704.02422 fig. 10 1704.02422 §7.3.1. The layer is a companion to the DC step, not a replacement for it.

When to use which

Implementation sketch

The steps below are the paper's eq. 6 for a single 2D frame, the cascade that uses it, and the cyclic loss term. Shapes and the FFT convention are in the paper card's pseudocode.

# Data-consistency step, paper eq. 6  [arxiv:1704.02422 §3]
# s_cnn: network output in k-space (complex)
# s_0:   measured k-space, zero off the sampled set
# omega: boolean mask, true where a sample was acquired
# lam:   weight; None means the noise-free limit (lambda -> infinity)

def data_consistency(s_cnn, s_0, omega, lam):
    # Start from the network output everywhere  [arxiv:1704.02422 §3]
    out = s_cnn.copy()
    if lam is None:
        # Noise-free: replace sampled entries with the measurements
        # [arxiv:1704.02422 §3]
        out[omega] = s_0[omega]
    else:
        # Noisy: weighted mix of output and measurement on Ω, divided
        # by (1 + lam)  [arxiv:1704.02422 §3, eq. 6]
        out[omega] = (s_cnn[omega] + lam * s_0[omega]) / (1 + lam)
    # Entries off Ω keep the network output  [arxiv:1704.02422 §3, eq. 6]
    return out

# Cascade: block, then DC, repeated  [arxiv:1704.02422 §4] [§6]
# n_c stages; each block output is added to its input (residual)

def cascade(x_u, s_0, omega, n_c, lam):
    x = x_u
    for stage in range(n_c):
        # Residual CNN block  [arxiv:1704.02422 §6]
        x = x + cnn_block(x)
        # Back to k-space, apply DC, and back to the image domain
        # [arxiv:1704.02422 §3]
        s = fft2(x)
        s = data_consistency(s, s_0, omega, lam)
        x = ifft2(s)
    return x

# Cyclic data-consistency term, first part of paper eq. 6  [arxiv:1709.00753 §III-D]
# m:    measured samples, under-sampled by the same mask
# dist: pixel-wise distance; the paper does not say which one it uses

def cyclic_dc_loss(gen, zero_filled, measured, mask, dist):
    # Generator output for the measured sample  [arxiv:1709.00753 §III-D]
    recon = gen(zero_filled)
    # Under-sample the output again with the same mask  [arxiv:1709.00753 §III-D]
    resampled = mask * fft2(recon)
    # Compare with the measurements that were actually taken  [arxiv:1709.00753 §III-D]
    # The caller adds this term with weight alpha = 1.0  [arxiv:1709.00753 §III-F, eq. 7]
    return dist(measured, resampled)

Notes for implementers:

• The layer is a fixed linear map on the output, so in the noise-free case it adds no trainable parameters 1704.02422 §3. In the noisy case the paper makes λ trainable 1704.02422 §3. Gradients flow through the layer, and the paper trains the whole cascade jointly 1704.02422 §4.
• The paper's noisy-branch formula, eq. 6, leaves unacquired entries unchanged. The official code also scales those entries by 1/(1 + λ). The paper card's Paper vs code section records this. Implement the equation as written, or document the difference.
• The paper computes the forward model with a unitary 2D DFT per frame 1704.02422 §2. Centred and non-centred FFT conventions move the mask; keep the mask and the FFT in one convention.
• The cyclic term is a soft constraint, and its distance is not stated. Each reimplementation has to pick d and document the choice 1709.00753 §III-D.

Contradictions

Within 1704.02422, the text and the released code disagree on the noisy DC branch and on whether λ is trainable in the released code. The paper card records both points 1704.02422 §3.

Between the two papers there is no direct contradiction, but their numbers do not compare. 1704.02422 reports MSE on cardiac cine data [arxiv:1704.02422 §7.1]. 1709.00753 reports PSNR, SSIM and NRMSE on IXI brain and Data Science Bowl chest images 1709.00753 §IV-A. Different data, metrics and baselines make the two sets of results unrelated quantitatively.

Papers

• Schlemper et al. 2017, cascaded CNN for dynamic cardiac MRI: introduces the DC layer as a fixed step after each CNN block, its noise-free and noisy forms, and the data-sharing layer for dynamic cardiac MRI.
• Compressed Sensing MRI Reconstruction using a GAN with a Cyclic Loss: uses a cyclic loss that ties a GAN generator's output back to the measurements, and chains a second generator for refinement.