Image super-resolution

Overview

Super-resolution (SR) takes a low-resolution (LR) image and predicts a high-resolution (HR) one, so the network has to produce detail that the input does not contain. The cards in this wiki cover several questions: how a reconstruction network gathers context from many pixels without prohibitive cost, how a frozen diffusion model can restore real photos with unknown degradations 2305.07015 §3.1, how semantic prompts from a degradation-aware tagger can steer such a model 2311.16518 §3.2, how SR appears as one stage of a diffusion cascade, how information is shared across frames in video 2012.02181 §3.1, and how a second-order version of that video design aligns frames with flow-guided deformable offsets 2104.13371 §3.1 2104.13371 §3.2, how a training-only shape term can keep medical SR outputs anatomically plausible 1705.08302 §2.4, how a network fitted directly to the measurements of one image or a few frames can recover detail 2301.05187 §4.2 2301.05187 app. A.4, how to estimate the blur kernel of one unknown-blur image from that image alone 1909.06581 §2, how to see which input pixels an SR network relies on, and how to score an SR output when no sharp reference image exists. Two later cards add a recurrent video design that back-projects per-neighbour estimates 1903.10128 §3.1, and a diffusion model that runs on a short restoration prior instead of on pixels 2303.09472 §4.2. The two classic reconstruction cards train with an L1 loss and Adam at an initial learning rate of 2e-4 at the paper level 2108.11084 §4.2 2402.15648 §3.6 2402.15648 §4.1. The other trained variants use different losses, listed under Implementation sketch. Two further cards widen the set. GRL is a restoration transformer whose layers run attention along thin stripes through a small set of anchor points, which keeps long-range attention affordable 2303.00748 §3.2 2303.00748 §4. The physics-informed card applies the same denoising idea to two-dimensional turbulent flow fields rather than photographs, with the fluid equation as an optional guide 2211.14680 §2.3.1.

The scores on this page are not on one scale. The classic reconstruction cards report PSNR in dB on classic benchmarks. StableSR reports FID, CLIP-IQA and MUSIQ on real-world sets, alongside PSNR and SSIM, and it tests on Real-ESRGAN degradations 2305.07015 §4.2. SeeSR reports on DIV2K-Val, RealSR, DRealSR and RealLR200, and follows the StableSR protocol for the two real-world sets 2311.16518 §4.1. The video card reports PSNR and SSIM, with REDS4 scored in RGB and the other sets on the Y channel 2012.02181 §4, and the BasicVSR++ card uses the same convention 2104.13371 tab. 1. The ACNN card reports SSIM and a 1 to 5 opinion score on cardiac MR 1705.08302 tab. 3. The attribution card reports correlations between its index and image metrics, and the quality metric reports a rank correlation with human ratings. The WIRE card gives its super-resolution results as an ordering and a gain over other activations, not as a benchmark table 2301.05187 §4.2 2301.05187 app. A.4. RBPN reports PSNR and SSIM on Vimeo-90k, Vid4 and SPMCS-11 1903.10128 tab. 9 1903.10128 tab. 7 1903.10128 tab. 8. DiffIR reports PSNR and LPIPS on five SR sets, but its main table compares only with GAN and diffusion restorers 2303.09472 tab. 2. The metric card was trained on outputs of nine older SR methods, and it reports no evaluation on the networks described here 1612.05890 §3 1612.05890 tab. 7. The KernelGAN card reports PSNR and SSIM on its own synthetic DIV2KRK benchmark, with blur kernels that differ per image 1909.06581 §5.2 1909.06581 tab. 1. GRL reports PSNR on classic ×4 sets 2303.00748 tab. 5, and the flow-field card reports an L2 norm and an equation residual on its own simulated fields 2211.14680 tab. 1. Neither number is an image-quality score on the scale above.

Variants

Lightweight CNN-Transformer hybrid (ESRT)

A small convolutional body with three High Preserving Blocks works at half resolution and carries a high-frequency residual. One Efficient Transformer then attends over unfolded 3×3 neighbourhoods, and its attention is split into four segments so each segment attends only within itself 2108.11084 §3.1 2108.11084 §3.3.2. Author-reported ×4 results: Set5 32.19 dB and Urban100 26.39 dB with 751K parameters 2108.11084 tab. 1. Card: Transformer for Single Image Super-Resolution.

Training memory is the main claim. The authors measure 4,191M for one Efficient Transformer against 16,057M for an original Transformer, both on 48×48 patches at batch size 16 2108.11084 tab. 4. The paper does not state how that baseline was configured.

State-space restorer (MambaIR)

The network follows a SwinIR-style three-stage layout. Each block replaces window attention with a state-space scan over four directions of the feature map, followed by a convolution bottleneck with channel attention 2402.15648 §3.2 2402.15648 §3.3 2402.15648 §3.5. Author-reported ×4 results on classic SR: Set5 33.03 dB and Urban100 27.68 dB for MambaIR, against 32.92 dB and 27.45 dB for SwinIR 2402.15648 tab. 3. The lightweight variant reaches Urban100 26.52 dB at 924K parameters and 42.3G MACs, where SwinIR-light is at 26.47 dB with 930K parameters and 31.8G MACs 2402.15648 tab. 4. Card: MambaIR: A Simple Baseline for Image Restoration with State-Space Model.

The paper does not state the width, depth or state size of its models; those values exist only in the authors' code, which the card records.

Anchored stripe attention for restoration (GRL)

GRL is a restoration transformer that also covers denoising, deblurring, JPEG artifact removal and demosaicking 2303.00748 §5. Each layer runs three branches in parallel: window attention for the regional range, anchored stripe attention for long-range context, and a channel-attention convolution for local structure 2303.00748 §4 2303.00748 fig. 6. The stripe branch does not build a full attention map. Its M anchors summarise the keys and values first, then each query reads the anchors, with a stated cost of O(NMd) time and O(NM) space 2303.00748 eq. 2 2303.00748 §3.2. Stripes alternate between horizontal and vertical across layers 2303.00748 §3.3.

Author-reported ×4 classical SR, PSNR: GRL-B reaches 33.10 dB on Set5 and 28.53 dB on Urban100 with 20.20M parameters, against 32.92 dB and 27.45 dB for SwinIR at 11.90M 2303.00748 tab. 5. The smaller models are GRL-S at 3.49M, with 27.90 dB on Urban100, and GRL-T at 0.91M, with 27.15 dB 2303.00748 tab. 5. GRL-S is below that SwinIR on Set5 (32.76 dB against 32.92 dB), so the gain is not uniform across sets 2303.00748 tab. 5. Card: Efficient and Explicit Modelling of Image Hierarchies for Image Restoration.

The paper gives no training protocol in the main text; it states Adam, an L1 loss and an initial rate of 2e-4 2303.00748 §5. The released configs for each task differ from that, and the card's Paper vs code section lists them.

Diffusion prior for real photos (StableSR)

StableSR keeps a pre-trained Stable Diffusion 2.1-base U-Net and its autoencoder frozen. It trains only a time-aware encoder and the scale-and-shift layers that modulate the U-Net's residual blocks 2305.07015 §3.1 [arxiv:2305.07015 eq. 1]. The encoder reads the LR image and the diffusion step, so its guidance changes over the sampling trajectory, and the authors place the strongest guidance where the signal-to-noise ratio is near 5e-2 2305.07015 §3.1 2305.07015 fig. 3.

Two optional parts sit on top. A controllable feature wrapping module, weighted by w between 0 and 1, favours realism near 0 and fidelity near 1; it is trained after the first stage 2305.07015 §3.2 2305.07015 tab. 4. An aggregation sampler cuts the latent into overlapping 64×64 patches, blends their noise predictions with Gaussian weights at each step, and so handles arbitrary sizes without visible seams 2305.07015 §3.3 2305.07015 alg. 1.

Author-reported on DIV2K-Valid, StableSR has the best FID, 24.44 against 26.47 for LDM, but lower PSNR and SSIM than several baselines 2305.07015 tab. 1. On the RealSR set its PSNR is 24.65 dB, below 27.30 dB for the RealSR baseline and 26.38 dB for BSRGAN 2305.07015 tab. 1. Table 3 turns off one part at a time: without pixel colour correction, RealSR PSNR falls from 24.65 to 22.24 dB [arxiv:2305.07015 tab. 3]. Cost is high: 200 DDPM steps take 15.16 s for a 512×512 output on one V100, and the 4-step SD-Turbo variant takes 0.83 s 2305.07015 tab. 5. Card: Exploiting Diffusion Prior for Real-World Image Super-Resolution.

Semantic prompts from a degradation-aware tagger (SeeSR)

SeeSR also runs Stable Diffusion for ×4 real-world SR, but it conditions the model on what the image shows. A RAM tagger is first fine-tuned on degraded inputs (DAPE), so its outputs on a low-resolution image match its outputs on the clean original 2311.16518 §3.2 2311.16518 eq. 1. The predicted tags enter as text through the frozen text encoder. The tagger's representation embeddings enter through extra cross-attention modules, and a ControlNet branch reads the LR image 2311.16518 §3.2 2311.16518 §3.3. At inference the sampler starts from the LR latent noised by the training schedule, not from pure noise 2311.16518 §3.4. The paper does not give the start step.

Author-reported ×4: SeeSR has the best FID, CLIPIQA and MUSIQ on all four test sets, and the best LPIPS and DISTS on DIV2K-Val 2311.16518 §4.2 2311.16518 tab. 2. On DIV2K-Val its FID is 31.93, against 36.57 for StableSR, the best other method there. Its PSNR and SSIM are below the best baseline on the three paired sets 2311.16518 tab. 2. The reported cost is 2283.7M parameters and about 7.24 s per 512×512 output with 50 steps on one V100; these figures come from the supplement, not the main text 2311.16518 §4.1. Card: SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution.

Diffusion SR stages (eDiff-I)

In a text-to-image cascade, eDiff-I runs a 64 px base model, then a 256 px and a 1024 px super-resolution stage 2211.01324 §4.2. Both SR stages are diffusion denoisers. The 256 px stage has 128 channels, self- and cross-attention at one resolution, and two residual blocks per level 2211.01324 tab. 3. The 1024 px stage removes self-attention and keeps only cross-attention, and it is trained on random 256×256 crops and applied at 1024×1024 2211.01324 tab. 4 2211.01324 app. A. Sampling for the SR stage uses a DEIS solver of order 3 over 10 steps 2211.01324 tab. 7. Card: eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers.

The paper reports only end-to-end zero-shot FID for the whole cascade, with no SR-only quality metric 2211.01324 tab. 1. Its SR stages are one part of a much larger system trained on about one billion pairs 2211.01324 §5, so the design does not transfer to a standalone SR model without retraining and measurement.

Diffusion on a restoration prior (DiffIR)

DiffIR runs the diffusion process on a compact restoration prior, a vector of 256 values, instead of on pixels or feature maps 2303.09472 §4.1 2303.09472 §4.2. In the first stage an encoder that sees the ground-truth image gives the prior, and a transformer restorer uses it. In the second stage a conditional denoiser predicts the prior from the low-quality image in four steps 2303.09472 §5.1. Both stages are trained with L1 terms: one on the restored image and one on the prior 2303.09472 eq. 13.

Author-reported ×4 SR, PSNR on Set14, Urban100, Manga109, General100 and DIV2K100: 27.73, 26.05, 30.32, 30.58 and 29.13 dB for DiffIR, against 27.14, 25.12, 28.67, 29.83 and 28.58 dB for SRdiff 2303.09472 tab. 2. The table also lists GAN restorers. It does not include ESRT or MambaIR. On real-world ×4 SR, the card records 0.74 T Mult-Adds for DiffIR against 37.25 T for LDM at 256 px input 2303.09472 tab. 6. The headline efficiency claim against RePaint is shown in a figure and cannot be checked from the tables 2303.09472 §1. Card: DiffIR: Efficient Diffusion Model for Image Restoration.

The released code departs from the paper's equations in several places, such as the noise schedule, what the denoiser predicts and the channel width. An implementation that follows only the paper will not match the released numbers. The DiffIR card lists these differences.

Diffusion on a fluid field (physics-informed DDPM)

This variant is not a photo model. Its DDPM is trained only on high-fidelity two-dimensional Kolmogorov-flow vorticity, with no paired low-fidelity data 2211.14680 §2.2 2211.14680 §3.1. At use time the coarse field is noised to an intermediate step t, and the reverse chain runs from t to zero with a deterministic DDIM update 2211.14680 §2.3 2211.14680 alg. 2. A sparse field is first padded by nearest-neighbour fill, and the output of each round becomes the guide for the next. The paper uses three rounds for ×4 and 5% sparsity 2211.14680 §2.3.1 2211.14680 §3.1.

The PDE residual can steer sampling in two ways. It can be fed to the noise network as a condition, with the condition dropped at some rate during training 2211.14680 eq. 6 2211.14680 alg. 1, or it can be subtracted from the update as a gradient step 2211.14680 eq. 9. The paper gives no value for the drop rate or for the step size.

Author-reported at ×4 (64 to 256 px), the L2 norm is 0.5622 for the diffusion model, against 0.6048 for a direct-mapping UNet and 1.3355 for bicubic. The equation residual is 0.2178, against 0.5438 and 10.6639 2211.14680 tab. 1. The L2 gap to the direct-mapping model is small, so the reported advantage rests mainly on the equation residual and the energy spectrum 2211.14680 §4. At 5% sparsity the diffusion model's L2 is 0.9318 against 1.2426 for the direct mapping 2211.14680 tab. 1. The authors state that one trained model serves all these tasks without retraining 2211.14680 §4. Card: A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction.

The equation residual in the paper is normalised by the vorticity norm, and the released code does not normalise it in the same way 2211.14680 eq. 12. The reported equation-loss values may therefore not match a re-run, and the card's Paper vs code section records the differences.

Bidirectional video propagation (BasicVSR, IconVSR)

BasicVSR runs two recurrent branches over the whole clip, one forward in time and one backward, and combines their final states for each frame [arxiv:2012.02181 §3.1] 2012.02181 eq. 1 2012.02181 eq. 3. Before a carried state is used at the next frame, it is warped to that frame with optical flow from SPyNet. The warp acts on features, not pixels 2012.02181 eq. 2 2012.02181 app. A. Each branch has 30 residual blocks and 64 channels, and the upsampler uses pixel shuffle 2012.02181 app. A. The authors report that warping images instead of features costs 0.17 dB, and that removing alignment costs 1.19 dB 2012.02181 §3.1.

IconVSR adds two parts. At keyframes every five frames, a lightweight EDVR extractor refills the state from three consecutive LR frames. Coupled propagation passes the backward state into the forward branch 2012.02181 §3.2 2012.02181 eq. 4-6 2012.02181 app. A. Author-reported ×4 on REDS4 with BI degradation: BasicVSR 31.42 dB at 6.3M parameters and 63 ms, IconVSR 31.67 dB at 8.7M and 70 ms, and EDVR 31.09 dB at 20.6M and 378 ms 2012.02181 tab. 2. The two IconVSR parts add 0.28 dB over BasicVSR 2012.02181 tab. 3. Card: BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond.

Second-order propagation with deformable alignment (BasicVSR++)

BasicVSR++ keeps the BasicVSR layout and changes two stages [arxiv:2104.13371 §3]. In propagation, the clip is swept four times, backward, forward, backward, forward, so information crosses the clip twice 2104.13371 §3.1. Each step uses the two most recent states of its sweep, not one, and the two are aligned together 2104.13371 §3.2 2104.13371 eq. 1. In alignment, the pre-trained flow sets the base of each deformable offset, and a learned residue corrects it, so the network learns only a deviation from the flow 2104.13371 §3.2 2104.13371 eq. 6. The authors report that the flow base keeps training stable, where offsets learned from scratch can overflow 2104.13371 §1 2104.13371 §3.2.

The ablation adds the two parts in turn on a dataset its caption does not name. Plain BasicVSR gives 31.48 dB, flow-guided alignment 31.94 dB, and adding second-order propagation 32.08 dB, with the full model at 32.39 dB 2104.13371 tab. 3. Without flow, alignment training collapses at 27.44 dB 2104.13371 tab. 4. Author-reported ×4 on REDS4 with BI degradation: 32.39 dB at 7.3M parameters and 77 ms, against 31.42 dB at 6.3M and 63 ms for BasicVSR 2104.13371 tab. 1. The smaller BasicVSR++ (S) reaches 32.24 dB at 6.4M parameters and 69 ms 2104.13371 tab. 2. The runtime is measured at 180 × 320 LR input, and the GPU is not stated in the table caption 2104.13371 tab. 1. Card: BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment.

The paper's abstract and summary headline a 0.82 dB gain over BasicVSR. That is the (S) variant's gain, 32.24 dB against 31.42 dB, not the full model's 2104.13371 tab. 2. The released code is in the MMEditing repository, now MMagic, a later snapshot than the paper. Several details differ from the paper, and the card's Paper vs code section lists them.

Recurrent back-projection with optical flow (RBPN)

RBPN predicts one target frame of a video from the target and several neighbouring frames 1903.10128 §3. Each neighbour is stacked with the target and a dense optical flow between them, then passed through a projection module that combines a single-image upscaler with a multi-frame path 1903.10128 §3.1 1903.10128 §3.2. The module refines the high-resolution estimate one neighbour at a time, and one convolution then reconstructs the frame from all of the estimates 1903.10128 §3.1. The authors read the module as falling back to recurrent single-image upscaling when the neighbours add no information 1903.10128 §3.3.

Author-reported ×4 on Vimeo-90k, PSNR: RBPN/6-PF scores 34.18, 37.28 and 40.03 dB on slow, medium and fast clips, against 32.96, 35.84 and 37.49 dB for VSR-DUF/6 1903.10128 tab. 9. On SPMCS-11 it scores 30.10 dB against 29.42 dB for VSR-DUF 1903.10128 tab. 8. The authors say it beats prior methods on Vid4 except VSR-DUF 1903.10128 §4.3. Its Vid4 row uses a frame-dropping protocol that differs from the usual one 1903.10128 tab. 7. Context matters: on SPMCS-32, adding future frames to past ones gives 31.74 against 31.64 dB 1903.10128 tab. 4, and removing optical flow lowers a 5-frame model from 31.54 to 31.36 dB 1903.10128 tab. 6. Compute grows with the number of neighbours; the card records 0.141 s per 4× frame for RBPN/6-PF on a 120×160 input, with the hardware not stated 1903.10128 app. A.4. Card: Recurrent Back-Projection Network for Video Super-Resolution.

Anatomy-constrained SR (ACNN-SR)

The target is cardiac cine MR. Its LR stacks are 2D short-axis slices 10 mm apart, and the HR volumes have 2 mm slices 1705.08302 §3.1.1. The SR network is trained with a Huber reconstruction term, plus a shape term. A predictor maps the output volume and the true HR volume to 64-d codes, and the squared distance between the two codes is penalised 1705.08302 §2.4 [arxiv:1705.08302 eq. 2]. The predictor is trained in two stages, with an autoencoder fitted to label maps 1705.08302 §2.4. The predictor is not used at inference 1705.08302 §2.4, so the network runs as the baseline does.

Author-reported on 200 test volumes: SSIM 0.796 for ACNN-SR against 0.783 for SR-CNN and 0.784 for 3D-UNet, and mean opinion scores of 4.36 and 4.25 from two cardiologists against 3.59 and 3.85 for SR-CNN 1705.08302 tab. 3. Inference on a 120×120×12 LR input takes 0.06 s, against 0.29 s for SR-CNN on a GTX-1080 1705.08302 tab. 3. The same shape term also appears in the paper's segmentation model, with endocardial Dice of 0.939 against 0.923 for 3D-UNet 1705.08302 tab. 1. Card: Anatomically Constrained Neural Networks (ACNN).

Wavelet implicit network (WIRE)

WIRE is a coordinate network: an MLP maps a pixel position to an intensity. Its activation is a complex Gabor wavelet, a complex exponential under a Gaussian window 2301.05187 §3.2 2301.05187 eq. 2. For super-resolution, the network is fitted by gradient descent to the low-resolution measurements, with 4× downsampling as the forward model 2301.05187 §4.2. For one single image, the paper calls its result "quantitatively higher" than the baselines and the sharpest shown, but the text gives no PSNR number 2301.05187 §4.2 2301.05187 fig. 7. For 4× multi-frame SR from four Kodak frames with known sub-pixel shifts and rotations, it reports at least 1 dB PSNR and 0.04 SSIM above the other nonlinearities 2301.05187 app. A.4. The paper does not state w0, s0 or the learning rate for the single-image experiment 2301.05187 §4.2. Card: WIRE: Wavelet Implicit Neural Representations.

Explaining an SR network (LAM)

LAM is an attribution method for SR networks, not a new architecture. For one output patch it gives a heat map over the LR input pixels that drive that patch 2011.11036 §3.3. It is integrated gradients with two changes: the baseline is the LR image blurred by a Gaussian, and the path progressively sharpens the blur, instead of using a black baseline and a straight line 2011.11036 §3.2 2011.11036 §3.3. Each map is summarised by a diffusion index, a rescaled Gini coefficient, where a larger value means more input pixels are involved 2011.11036 eq. 6.

On 150 ×4 test sub-images, the authors report Pearson and Spearman correlations above 0.8 between this index and PSNR, and between it and LPIPS, across 48 networks 2011.11036 §4.1 2011.11036 §4.4. Receptive field does not track the index: RRDBNet's receptive field already exceeds the test-image size, and several higher-PSNR networks do not use significantly more pixels 2011.11036 tab. 1 2011.11036 §4.4. Card: Interpreting Super-Resolution Networks with Local Attribution Maps.

Estimating an unknown blur kernel (KernelGAN)

KernelGAN is a front end for blind SR. It estimates the downscaling kernel of one LR image from that image alone, with no training data, and the kernel is then given to a non-blind SR method such as ZSSR or SRMD 1909.06581 §2 1909.06581 §5.1. Two networks train on the input: a generator that downscales it and a patch discriminator that scores crops of the input and of the output 1909.06581 §2 1909.06581 §3. The generator has no activations. It has five 64-channel hidden layers with kernel sizes 7, 5, 3, 1 and 1, and a stride-2 output layer 1909.06581 §4.1 1909.06581 fig. 4. The product of its filters is the kernel, 13×13 at ×2 1909.06581 §4.2. A regulariser keeps the kernel summing to one, with its mass near the centre and sparse 1909.06581 §4.2 1909.06581 eq. 3. The ×4 kernel is derived from the ×2 kernel, not estimated again 1909.06581 §4.2 1909.06581 app..

Training runs 3,000 alternating generator and discriminator steps with Adam (β1 0.5, β2 0.999), and a learning rate of 2e-4 divided by ten every 750 steps 1909.06581 §2. The method assumes one kernel for the whole image 1909.06581 §5.3. Author-reported on 100 DIV2KRK images, with ZSSR as the SR step: 30.363 dB / 0.8669 at ×2 and 26.810 dB / 0.7316 at ×4 1909.06581 tab. 1. The same ZSSR with the Michaeli and Irani kernel scores 29.368 dB and 26.080 dB, about 1.0 dB and 0.73 dB below 1909.06581 tab. 1. The ground-truth kernel with ZSSR is the upper bound at 32.436 dB for ×2 1909.06581 tab. 1.

Each image is a full training run: 61 s on a V100 and 102 s on a K80, against 45 minutes per image reported for the earlier estimator 1909.06581 §5.3. Real photos are judged only by eye, with no numbers 1909.06581 §5.3 1909.06581 fig. 1. The depth ablation gives 3.8 dB at ×2 and 1.6 dB at ×4 over a single-layer generator, but the single-layer model is not described, so that gain is hard to reproduce 1909.06581 fig. 6. The paper's printed eq. 2 has a sign convention that the text does not match, and the card records the convention the authors' code uses 1909.06581 eq. 2. The code is licensed for academic and non-commercial use only, so a commercial product needs its own implementation. Card: Blind Super-Resolution Kernel Estimation using an Internal-GAN.

Learned no-reference quality metric

Full-reference metrics need a ground-truth HR image. The authors show a half-pixel-shifted SR image that looks fine to people but scores poorly on PSNR, SSIM and IFC 1612.05890 fig. 1 1612.05890 tab. 1. This metric is trained on human ratings instead. Each of 1,620 SR images has 50 absolute ratings from 0 to 10, and its score is the mean of the median 40 1612.05890 §3. The image is described by three feature groups: local DCT statistics, global steerable-pyramid statistics and singular values of image patches 1612.05890 §4. One random-forest regressor is trained per group, and a linear layer combines the three outputs 1612.05890 §4.4 1612.05890 eq. 7.

Author-reported Spearman correlation with human scores is 0.931 overall under 5-fold cross-validation. The best baseline overall is CORNIA at 0.919, and PSNR is at 0.604 1612.05890 tab. 7. Card: Learning a No-Reference Quality Metric for Single-Image Super-Resolution.

When to use which

The classic reconstruction cards do not compare with each other, so their Urban100 numbers are not a head-to-head ranking. ESRT, MambaIR-light and GRL-B report ×4 Urban100 scores of 26.39, 26.52 and 28.53 dB, each in its own table against its own baselines 2108.11084 tab. 1 2402.15648 tab. 4 2303.00748 tab. 5. The StableSR, BasicVSR, ACNN, DiffIR and RBPN cards each use their own baselines and metrics, so their numbers are not placed on one scale either.

Implementation sketch

The classic reconstruction variants share this training core. Only the body changes. ACNN-SR keeps the Huber reconstruction term and adds a training-only shape term. The diffusion stages, BasicVSR, WIRE, LAM and the no-reference metric do not use this loop.

# Common SR training step  [arxiv:2108.11084 §4.2] [arxiv:2402.15648 §4.1]
# Input:  LR/HR patch pair cut from DIV2K; LR patch is 48x48 in ESRT
# Output: updated model parameters
def train_step(model, lr_patch, hr_patch, optimizer):
    # feature extraction, body, then upsampling by the scale factor
    # (the body is the part that differs between variants)
    sr = model(lr_patch)                    # (B, 3, s*h, s*w)
    # L1 loss against the ground-truth HR patch  [arxiv:2108.11084 §4.2]
    loss = mean(abs(sr - hr_patch))
    optimizer.zero_grad()
    loss.backward()
    # Adam, initial learning rate 2e-4  [arxiv:2108.11084 §4.2] [arxiv:2402.15648 §4.1]
    optimizer.step()
    return loss

Where the variants differ in the body:

• ESRT: convolutional High Preserving Blocks, then one Efficient Transformer over unfolded 3×3 tokens, with attention split into four segments 2108.11084 §3.3.2.
• MambaIR: residual state-space blocks. Each runs a selective scan over four directions of the feature map, then a convolution bottleneck with channel attention 2402.15648 §3.3 2402.15648 §3.5.
• GRL (this loop, with the paper's stated values): the paper gives L1 loss, Adam and an initial learning rate of 2e-4 2303.00748 §5. The schedule and the batch and patch sizes are not stated in the paper. Its body is the three-branch layer described above, with window, stripe and convolution parts run in parallel 2303.00748 §4.
• Physics-informed DDPM (not this loop): the noise network is trained with a squared-error loss on the noise, as in the diffusion cards, and with a drop of the PDE condition 2211.14680 alg. 1. Sampling starts from the noised guide at step t, not from pure noise, and runs DDIM with sigma zero 2211.14680 alg. 2. The paper gives the step counts for each task, but not the drop rate or the guidance strength 2211.14680 §3.1.
• BasicVSR++ (video, not this loop): a Charbonnier loss, with Adam on 30-frame REDS samples and batch size 8 for 600K iterations 2104.13371 app. B 2104.13371 §4. The main network uses a learning rate of 1e-4, the flow network 2.5e-5, and the flow network is fixed for the first 5,000 iterations on REDS 2104.13371 §4 2104.13371 app. B. The paper does not state the Charbonnier epsilon or the Adam betas 2104.13371 app. B.
• ACNN-SR (this loop plus a shape term): the Huber reconstruction term, plus λ1 times the squared distance between predictor codes of the output and the true HR volume, plus weight decay. The paper gives λ1 = 0.01 and weight decay 5e-6, and a fixed learning rate of 0.001 with the optimiser variant not stated 1705.08302 eq. 2 1705.08302 supp. Implementation Details.
• BasicVSR (video, not this loop): a Charbonnier loss with ε = 1e-8, trained with Adam on 15-frame REDS clips and 14-frame Vimeo-90K clips, batch size 8, 300K iterations 2012.02181 app. A [arxiv:2012.02181 eq. 7] 2012.02181 §4. Initial learning rates are 2e-4 for most modules, 1e-4 for the feature extractor and 2.5e-5 for the flow estimator 2012.02181 §4.
• StableSR (diffusion, not this loop): the loss is the mean squared error between the predicted and the true noise on the latent, with the U-Net frozen. Only the time-aware encoder and the modulation layers are updated, with Adam at learning rate 5e-5 2305.07015 §3.1 2305.07015 §4.1 2305.07015 §4.3. At inference, the noise predictions for overlapping 64×64 latent patches are blended with Gaussian weights at every step 2305.07015 alg. 1 2305.07015 eq. 7.
• SeeSR (diffusion with tag and embedding prompts, not this loop): the tagger is first trained with an MSE term on representation embeddings and a cross-entropy term on the tag logits, both against the frozen tagger's output on the clean image 2311.16518 eq. 1 2311.16518 §3.2. The diffusion stage then minimises the noise-prediction MSE 2311.16518 eq. 2, with the UNet, text encoder and VAE frozen. Only the image encoder, ControlNet and the cross-attention modules are updated 2311.16518 §3.3. The paper reports Adam at 5e-5 for the diffusion stage, and 50 spaced DDPM steps at inference 2311.16518 §4.1.
• WIRE (not this loop): the fit is an l2 loss between the forward-modelled output and the measurements, with Adam and no other regularisation, and the learning rate decays to 0.1× its start 2301.05187 §4. There is no ground-truth HR patch in the loss.
• KernelGAN (kernel estimation, not this loop): the generator and discriminator are updated in turn for 3,000 steps, with Adam and the learning rate schedule given under Variants 1909.06581 §2. The generator's loss adds a regulariser on the kernel 1909.06581 eq. 3. The output is one kernel, not an SR image, so the SR step runs afterwards with that kernel.
• DiffIR (diffusion on a prior, not this loop): stage 1 trains the prior encoder and the transformer with an L1 loss on the restored image. Stage 2 adds an L1 loss on the prior and trains the denoiser on noised prior vectors 2303.09472 §4.1 2303.09472 §4.2 2303.09472 eq. 13. The paper gives Adam with betas 0.9 and 0.99, learning rate 2e-4 and batch 64 for SR, for 1M iterations 2303.09472 §5.1 2303.09472 app. E. It states no learning-rate schedule for SR 2303.09472 §5. At inference, the prior is sampled from a standard normal and denoised over the four steps, then passed to the transformer 2303.09472 alg. 2.
• RBPN (video, not this loop): a per-pixel L1 loss with Adam (momentum 0.9) and an initial learning rate of 1e-4, batch size 8, 150 epochs on Vimeo-90k 7-frame sequences 1903.10128 §4.1. The paper's wording on when the rate drops is ambiguous, so the RBPN card records how the released code does it.
• Test-time memory: ESRT's memory figures come from 48×48 training patches. The paper does not give memory for full-size inputs 2108.11084 tab. 4, so plan tiling or chopping at inference time and measure it yourself.
• Evaluation, not training: the no-reference metric is a separate pass over a finished SR output. Its feature extraction and forest prediction are sketched on its card 1612.05890 alg. 1, and the card's known gaps are listed under its Paper vs code section.

Contradictions

• SwinIR scale. ESRT lists SwinIR under ×4 with BSD100 29.20 dB and Manga109 33.98 dB 2108.11084 tab. 7. MambaIR lists the same two values for SwinIR-light at ×3, and gives different ×4 values for SwinIR-light 2402.15648 tab. 4. The two papers use different SwinIR variants, and the scale label in ESRT's table cannot be settled from these two cards.
• ESRT's own numbers. ESRT reports 770K parameters at ×3 and 751K at ×4 2108.11084 tab. 1, and 0.68M in its timing table 2108.11084 tab. 2. Its ×3 row (770K, BSD100 29.15 dB) reappears as the ESRT row of the table labelled ×4 2108.11084 tab. 7. The paper does not resolve these.
• Agreement on shared baselines. The baselines the two classic reconstruction cards share match. IMDN at ×4 on Urban100 is 26.04 dB in both, and LatticeNet is 26.25 dB in both 2108.11084 tab. 1 2402.15648 tab. 4. GRL's table has the same SwinIR rows: 27.45 dB at Urban100 ×4 for the larger model and 26.47 dB for the 0.9M model 2303.00748 tab. 5, matching MambaIR's SwinIR figures 2402.15648 tab. 3 2402.15648 tab. 4.
• Diffusion stages and reconstruction models. eDiff-I's SR stages are not compared with ESRT or MambaIR, and its paper gives no SR-only metric 2211.01324 tab. 1. StableSR's baselines are RealSR, BSRGAN, DASR, Real-ESRGAN+, FeMaSR, LDM, SwinIR-GAN and IF_III, not ESRT or MambaIR 2305.07015 tab. 1. SeeSR's baselines are BSRGAN, Real-ESRGAN, LDL, DASR, FeMaSR, LDM, StableSR, ResShift, PASD and DiffBIR, and they also exclude ESRT and MambaIR 2311.16518 §4.1. No head-to-head number exists across these cards.
• StableSR's FID on DIV2K-Val. The StableSR paper reports FID 24.44 for StableSR against 26.47 for LDM on DIV2K-Valid 2305.07015 tab. 1. The SeeSR paper reports 36.57 for StableSR on DIV2K-Val, the best other method there, against 31.93 for SeeSR 2311.16518 tab. 2. The two cards do not show that the test crops and degradations match, so the two StableSR values are not on one scale.
• Protocols, not one scale. StableSR's PSNR is measured on DIV2K-Valid with Real-ESRGAN degradations and on real-world sets 2305.07015 §4.2. The classic cards use classic benchmarks 2108.11084 tab. 1. KernelGAN's DIV2KRK uses synthetic anisotropic Gaussian kernels 1909.06581 §5.2. Their PSNR values are not comparable, and neither are the ACNN opinion scores. The flow-field card's L2 and equation residual are measured on vorticity fields, so they are not comparable with any PSNR on this page 2211.14680 tab. 1.
• Kernel estimation and SR scores. KernelGAN's ×2 and ×4 numbers depend on which non-blind SR method runs after it: with SRMD, KernelGAN scores 29.565 dB at ×2, and with ZSSR 30.363 dB 1909.06581 tab. 1. Compare kernels only with the same SR step. Its ×4 SRMD result is below PDN in PSNR, but above it in SSIM 1909.06581 tab. 1.
• BasicVSR: text against table. The text says BasicVSR beats prior methods on REDS4, UDM10 and Vid4 2012.02181 §4. In the table, EDVR scores higher on Vid4 BI (27.35 dB against 27.24 dB) and on Vimeo-90K-T BI (37.61 dB against 37.18 dB) 2012.02181 tab. 2. Quote the table column, not the summary.
• ACNN speed claim. The text says ACNN-SR is faster "by a factor of 5" 1705.08302 §3.5. Its table gives 0.29 s against 0.06 s, which is about 4.8× 1705.08302 tab. 3.
• RBPN flow timing. The paper says the optical flow is precomputed 1903.10128 §3.1. The RBPN card records that the released code computes it for each training sample during loading. The two differ in training cost and in the augmentation the flow sees.
• DiffIR text and table. The SR section lists Set5 as a test set, but the SR table has no Set5 column 2303.09472 §5.3 2303.09472 tab. 2. The table caption names LPIPS and DISTS, while its columns are PSNR and LPIPS 2303.09472 tab. 2. Quote the columns.
• DiffIR and the classic cards. DiffIR's Urban100 ×4 score of 26.05 dB is below ESRT's 26.39 dB and MambaIR-light's 26.52 dB 2108.11084 tab. 1 2402.15648 tab. 4 2303.09472 tab. 2. The three papers use different baselines and protocols, so the gap does not rank them.
• WIRE and the reconstruction models. WIRE's SR is compared only with other INR activations (SIREN, Gaussian, ReLU with positional encoding, MFN), not with ESRT or MambaIR 2301.05187 §4 2301.05187 app. A.4. Its single-image SR gives no number in the text, so it cannot be placed on the benchmark scale used by the two classic reconstruction cards 2301.05187 §4.2.

Papers

• Efficient and Explicit Modelling of Image Hierarchies for Image Restoration: a restoration transformer whose layers run window, anchored stripe and convolution branches in parallel.
• A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction: a DDPM trained on high-fidelity vorticity that reconstructs coarse or sparse flow fields from a noised guide, optionally steered by the PDE residual.
• Recurrent Back-Projection Network for Video Super-Resolution: a recurrent video SR network that refines high-resolution features one optical-flow-aligned neighbour at a time.
• DiffIR: Efficient Diffusion Model for Image Restoration: a diffusion model that denoises a compact 256-value restoration prior in four steps, then a transformer restores the image.
• Blind Super-Resolution Kernel Estimation using an Internal-GAN: a per-image linear-generator GAN that estimates the downscaling kernel, which a non-blind SR method then uses.
• Learning a No-Reference Quality Metric for Single-Image Super-Resolution: a learned, reference-free score trained on human ratings, from three random-forest feature groups.
• Interpreting Super-Resolution Networks with Local Attribution Maps: an integrated-gradients attribution for SR networks, with a blurred baseline and progressive path.
• Transformer for Single Image Super-Resolution: ESRT, a lightweight CNN-Transformer hybrid with a memory-reduced attention.
• eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers: a text-to-image cascade whose 256 px and 1024 px stages are diffusion SR models.
• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution: Stable Diffusion SR steered by tag text and tagger embeddings from a degradation-aware tagger, with an LR-noised start.
• MambaIR: A Simple Baseline for Image Restoration with State-Space Model: restoration backbone that replaces window attention with a four-direction state-space scan.
• WIRE: Wavelet Implicit Neural Representations: an MLP whose Gabor-wavelet activation fits images, including SR, from their measurements.
• Exploiting Diffusion Prior for Real-World Image Super-Resolution: StableSR, a time-aware encoder that steers a frozen Stable Diffusion for real photos, with patch-blended sampling.
• BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond: bidirectional recurrent video SR with flow-warped features, plus IconVSR's keyframe refill.
• Anatomically Constrained Neural Networks (ACNN): a training-only code-distance shape term for cardiac MR super-resolution and segmentation.
• BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment: second-order grid propagation with flow-guided deformable alignment, extending BasicVSR.