--- kind: concept slug: residual-learning title: Residual learning summary: >- A residual unit computes a learned function F(x) and combines it with a shortcut of its input, so the block learns a correction to x. Thirteen papers use it across vision, spiking networks, forecasting, offline RL and encrypted inference, with different blocks and combine rules. Results are author-reported. tags: [architecture, training] papers: ["1709.00753", "1709.02054", "1711.10684", "1803.07835", "1904.07396", "2006.03761", "2012.02181", "2102.04159", "2106.07229", "2201.03545", "2304.08424", "2601.10471", "2609.37870"] related: [convolutional-neural-network, homomorphic-encryption, semantic-segmentation, multi-scale-feature-fusion, deep-learning, channel-attention, image-denoising, image-super-resolution, multi-layer-perceptron, recurrent-neural-network, time-series-forecasting, generative-adversarial-network, data-consistency-layer, spiking-neural-network, surrogate-gradient, vision-transformer, depthwise-separable-convolution, feed-forward-reconstruction, optical-character-recognition, attention-mechanism, convolutional-recurrent-network, point-to-grid-interpolation, chamfer-distance, offline-reinforcement-learning, flow-matching] status: draft updated_at: 2026-11-01 --- ## Overview A residual unit outputs y = h(x) + F(x). F is a learned block of layers and h is a shortcut that carries the input x forward. When h is the identity, the block only has to learn a correction to its input [arxiv:1711.10684 §2.1.2]. The same shape appears in an MLP block, where the shortcut is a linear map [arxiv:2304.08424 §4]. The skip can also span a whole network instead of one block: an MRI reconstruction network outputs a correction that is added to its zero-filled input [arxiv:1709.00753 §III-E]. Variants differ in five places: what F is (convolutions or an MLP, and how many layers), where normalisation and activation sit relative to the addition, what the shortcut does when input and output shapes differ, how far the skip reaches, and how the branch and shortcut are combined. In spiking networks, adding two spike trains does not give a spike train, so the sum is replaced by a logical gate [arxiv:2102.04159 §3.3]. A ConvNet block can also scale its residual branch per channel before the addition [arxiv:2201.03545 §2.6]. The segmentation paper does not say how the shortcut is matched when shapes differ [arxiv:1711.10684 §2.1.2], so that case is left to the implementer. An encrypted ResNet-20 does state it: a 1x1 convolution on the shortcut at each of its two stage changes [arxiv:2106.07229 tab. 1]. In offline RL, a residual MLP adds a bounded correction to each action sampled from a fixed flow policy [arxiv:2601.10471 §3.2]. In scene text recognition, the residual part is only the encoder backbone [arxiv:1709.02054 §4.2]. The motivation given varies by paper. One paper argues that residual connections ease training of deep stacks when labels are scarce, but it does not compare against a non-residual network of the same size [arxiv:1711.10684 §2.1.1]. Another paper ablates its connections and finds that removing them lowers accuracy at some forecast horizons but not at the longest one [arxiv:2304.08424 §5.4]. In spiking networks, the authors argue that the ordinary residual block cannot pass its input through unchanged, and that its gradient product shrinks or grows with depth [arxiv:2102.04159 §3.2]. This page rests on thirteen cards from different fields. Each reports results on its own benchmarks, so the page shows the design space, not a ranking. Two of the thirteen (point-cloud completion and scene text) use residual structure only in a weaker sense, and their entries say so. ## Variants ### Full pre-activation residual unit (segmentation) The unit applies batch normalisation and ReLU before each of two 3x3 convolutions, and adds the identity shortcut [arxiv:1711.10684 §2.1.2] [fig. 1]. Downsampling is a stride of 2 in the first convolution of a unit, not pooling, and channel counts change between levels [arxiv:1711.10684 tab. I]. The paper places this unit at every level of a 7-level encoder-decoder and removes U-Net's cropping step, so input and output have the same size [arxiv:1711.10684 §2.1.3] [arxiv:1711.10684 §2.3]. Whether an activation follows the addition is not stated [arxiv:1711.10684 §2.1.2]. Card: [Road Extraction by Deep Residual U-Net](/papers/1711.10684). ### Residual-on-residual with channel gating (denoising) The network chains four enhancement attention modules. Each module adds its output to its input, and the body output is added to the extracted features [arxiv:1904.07396 §3.2] [arxiv:1904.07396 eq. 6]. Inside each module, a channel gate computed from globally pooled features rescales the channels [arxiv:1904.07396 §3.2.1]. The network's output is added to the noisy input [arxiv:1904.07396 §3.2] [arxiv:1904.07396 eq. 3]. In the ablation, the four check-mark columns (LSC, SSC, LC, FA) add to a baseline of 28.45 dB, reaching 28.96 dB with all of them on BSD68 [arxiv:1904.07396 tab. 1]. This is a single run per row. Card: [Real Image Denoising with Feature Attention](/papers/1904.07396). ### Residual blocks after warping (video super-resolution) BasicVSR refines a carried feature state with residual blocks after the state has been warped by estimated optical flow. Each branch uses 30 blocks of 64 channels [arxiv:2012.02181 eq. 2] [arxiv:2012.02181 app. A]. Here the residual blocks are a refinement stage, not the skip path that the other cards study. The card records no ablation of the residual blocks themselves; its ablations concern alignment and the IconVSR components [arxiv:2012.02181 §3.1] [arxiv:2012.02181 tab. 3]. Card: [BasicVSR](/papers/2012.02181). ### Residual MLP block with linear skip (forecasting) Each block has a ReLU hidden layer, dropout on the output layer only, a fully linear skip, and layer norm on the sum [arxiv:2304.08424 §4]. The same block serves as the feature projection, inside the dense encoder, and in the temporal decoder [arxiv:2304.08424 §4.1] [arxiv:2304.08424 §4.2]. A global linear map from the look-back to the horizon is added to the output, so a purely linear model is a special case [arxiv:2304.08424 §4.2]. Card: [Long-term Forecasting with TiDE](/papers/2304.08424). ### Global residual over a zero-filled input (MRI reconstruction) The generator is a convolutional autoencoder built from residual blocks. It takes the zero-filled image from an under-sampled k-space and predicts a residual, which is added back to that input, so the network learns the aliasing artefact rather than the image itself [arxiv:1709.00753 §III-E]. The paper stacks a second generator of the same design on the first, with its own weights [arxiv:1709.00753 §III-G]. Training adds a cyclic data-consistency term to the adversarial loss, with weights of 1.0 and 10.0 [arxiv:1709.00753 §III-D] [arxiv:1709.00753 §III-F]. Reported time is 0.061 s (ReconGAN) and 0.106 s (RefineGAN) per 256x256 brain image, against 8.57 s for CSCMRI [arxiv:1709.00753 tab. 1]. The hardware is not stated, and the sampling rate used for timing is not stated either. Card: [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](/papers/1709.00753). ### Bottleneck residual blocks in a detect-then-repaint pipeline (raindrop removal) The detector is five convolutions with six bottleneck residual blocks, batch normalisation and ReLU after every convolution, and a stride of 2 in the second convolution [arxiv:2609.37870 §4.1]. Each bottleneck is a 1x1 convolution down to 64 channels, a 3x3 convolution, and a 1x1 convolution up to 256 channels [arxiv:2609.37870 §4.1]. The reconstruction network has the same design with eight residual blocks and a four-channel input [arxiv:2609.37870 §4.2]. A refine network with two residual blocks follows [arxiv:2609.37870 §4.3]. The card does not describe the shortcut in words, nor whether a block is trained to learn a correction to its input, so this entry records the block layout only. Card: [Learning from synthetic photorealistic raindrop for single image raindrop removal](/papers/2609.37870). ### Spike-element-wise residual block (spiking networks) The branch A is a spiking residual mapping and S is the block input. The output is g(A, S), where g is ADD, AND or IAND [arxiv:2102.04159 eq. 9] [arxiv:2102.04159 tab. 1]. ADD is not binary, so k stacked blocks give outputs of at most k + 1. AND and IAND keep spikes binary [arxiv:2102.04159 §3.3]. Identity is reached by zeroing the weight and bias of the last BatchNorm in the branch for ADD and IAND, which makes A zero [arxiv:2102.04159 §3.3]. In an identity region with A held constant, the gradient through k blocks is exactly 1 [arxiv:2102.04159 eq. 11]. The paper says deep directly trained Spiking ResNet-50, -101 and -152 do not converge without zero initialisation, but it does not say the same for SEW ResNet [arxiv:2102.04159 §4.1]. Card: [Deep Residual Learning in Spiking Neural Networks](/papers/2102.04159). ### Residual blocks in a position-map regressor (3D face) One convolution is followed by 10 residual blocks that reduce a 256x256 face crop to an 8x8x512 feature map. A decoder of 17 transposed convolutions then writes a 256x256x3 map, where each cell holds the x, y and z position of one face point [arxiv:1803.07835 §3.2] [arxiv:1803.07835 §3.1]. The paper does not state the channel widths or strides of the residual blocks, so an implementer must choose them [arxiv:1803.07835 §3.2]. Card: [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](/papers/1803.07835). ### Inverted-bottleneck block with layer scale (ConvNet backbone) The block is a 7x7 depthwise convolution, a LayerNorm, an expansion to four times the channels, a single GELU, a projection back, and a per-channel layer scale. The result is added to the block input, and stochastic depth is applied to the residual branch [arxiv:2201.03545 §2.4] [arxiv:2201.03545 §2.5] [arxiv:2201.03545 §2.6]. Layer scale starts at 1e-6 [arxiv:2201.03545 app. A.1]. The paper reaches this block by changing a ResNet-50 one step at a time, and keeps each step only if it improves ImageNet-1K accuracy [arxiv:2201.03545 §2] [arxiv:2201.03545 tab. 10]. Card: [A ConvNet for the 2020s](/papers/2201.03545). ### Residual correction to a sampled action (offline RL) A multi-step flow-matching policy is trained by flow matching alone and then kept fixed as a behaviour prior [arxiv:2601.10471 §3.2]. A four-layer MLP of width 512 adds a correction to each sampled action, and the executed action is the clipped sum of the base and the correction [arxiv:2601.10471 app. B] [arxiv:2601.10471 alg. 1]. The base action enters with its gradient stopped, so Q-learning updates only the correction [arxiv:2601.10471 §3.2]. The correction takes the sampled action as input, not only the state [arxiv:2601.10471 §3.2] [arxiv:2601.10471 §5]. A penalty on the mean squared correction keeps it near a per-task budget delta [arxiv:2601.10471 §3.2] [arxiv:2601.10471 app. B.1]. On cube-double task 1, the base flow alone scores 10 ± 2 and the same checkpoints with the correction score 62 ± 12, over 50 episodes [arxiv:2601.10471 app. C.3]. Card: [DeFlow: Decoupling Behavior-Prior Modeling and Value Maximization for Offline Policy Extraction](/papers/2601.10471). ### Additive skips in a grid encoder-decoder (point-cloud completion) The encoder has four blocks, each a 4x4x4 3D convolution, batch normalisation, leaky ReLU and 2x2x2 max pooling, with 32, 64, 128 and 256 channels [arxiv:2006.03761 §3.3]. Encoder and decoder maps of the same size are added, in the manner of U-Net connections [arxiv:2006.03761 §3.3]. The card does not describe a block that learns a residual on a shortcut, and it does not explain the word "residual" in the title. This entry therefore records a skip sum only. The paper reports 2.723 against 4.016 for PCN on ShapeNet, in x10^-4 Chamfer [arxiv:2006.03761 tab. 1]. Card: [GRNet: Gridding Residual Network for Dense Point Cloud Completion](/papers/2006.03761). ### Residual encoder backbone with a focusing network (scene text) The encoder is a 32-layer ResNet-style CNN, followed by a bidirectional LSTM [arxiv:1709.02054 §4.2] [arxiv:1709.02054 tab. 1]. The card's text does not describe the shortcut, so the residual part here is the backbone only. The focusing network is a separate loss on the attention, weighted by lambda; lambda of 0 disables it [arxiv:1709.02054 §4.2] [arxiv:1709.02054 tab. 6]. Part of the training data has character pixel labels, and the paper uses about 4 million synthetic word images for that subset [arxiv:1709.02054 §4.2]. Card: [Focusing Attention: Towards Accurate Text Recognition in Natural Images](/papers/1709.02054). ### Residual blocks under encryption (ResNet-20 on CKKS) The network is a standard ResNet-20 with the same stages and blocks, trained in plaintext and then evaluated on an encrypted image [arxiv:2106.07229 §4.1]. Its shortcut uses a 1x1 convolution at each of the two stage changes [arxiv:2106.07229 tab. 1]. The residual addition itself is not written out in the card, so the card records the layer list and the shortcut projection only. Two changes make the block computable under encryption: ReLU becomes a composition of minimax polynomials, and bootstrapping is inserted in the middle and at the end of each ReLU [arxiv:2106.07229 §3.4]. The card reports a runtime of about 4 hours per image on a 112-core CPU [arxiv:2106.07229 tab. 5]. Whether the encrypted runs are real or simulated is not clear from the text [arxiv:2106.07229 §4.1] [arxiv:2106.07229 §4.2]. Card: [Encrypted ResNet-20 with CKKS bootstrapping](/papers/2106.07229). ## When to use which | Situation | Use | Why | |---|---|---| | Deep dense-prediction network, few labels, wish to train deeper stacks | Full pre-activation units in an encoder-decoder with no cropping | The authors argue residual connections help when labels are scarce, without a same-size non-residual control [arxiv:1711.10684 §2.1.1] | | Parameter budget matters in a U-Net-style segmenter | Same as the row above | The text reports about 7.8M parameters against 30.6M for U-Net, which the layer table alone does not confirm [arxiv:1711.10684 §3.4] [arxiv:1711.10684 tab. I] | | Single-pass denoising of real photographs, no per-image tuning | Residual-on-residual with channel gating | 39.23 dB PSNR and 0.9526 SSIM on DnD, blind, against 38.06 dB for CBDNet, author-reported [arxiv:1904.07396 tab. 5]. The public repository has the test model only, so training is hard to reproduce; see the card | | Video super-resolution on long clips, simple recurrent baseline | Bidirectional propagation with residual blocks after flow warping | 63 ms and 6.3M parameters against 378 ms and 20.6M for EDVR, with PSNR 31.42 against 31.09 dB on REDS4 (BI), author-reported [arxiv:2012.02181 tab. 2]. These figures are for the whole model, not the residual blocks alone | | Many related series with known future covariates, need a linear fallback | Residual MLP block with linear skip and a global linear residual | A linear model is a special case of the design [arxiv:2304.08424 §4.2]. Removing residual connections lowered accuracy at horizons 96 to 336 on Electricity [arxiv:2304.08424 §5.4] | | Accelerated MRI from radial masks at 10 to 40%, one forward pass per image, with a physics-consistency term | Global residual over the zero-filled image, with a cyclic data-consistency loss | Over 42 dB average PSNR at 40% sampling, author-reported and plotted, not tabulated [arxiv:1709.00753 §I]. The authors' repository has no licence, so no code from it is used here; see the card's Paper vs code section | | Adherent raindrops on a lens or windscreen, with paired synthetic training data and a detect-reconstruct-refine pipeline | Bottleneck residual blocks in the detector and reconstructor, with a refine stage | 41.29 dB PSNR on the authors' synthetic test set, against 37.79 dB for Qian et al. [arxiv:2609.37870 tab. 1]. On the real-image set of Qian et al., the authors' rough-mask model reaches 30.17 dB against 30.82 dB for Qian et al., which the authors call on par [arxiv:2609.37870 tab. 2] | | Deep spiking classifier trained directly, with spikes passed between layers | SEW block with ADD, or IAND if outputs must stay binary; avoid AND | On ImageNet at T = 4, SEW ResNet-152 reaches 69.26% against 10.03% for Spiking ResNet-152 [arxiv:2102.04159 tab. 2]. On DVS Gesture, AND reaches 70.49% against 97.92% for ADD [arxiv:2102.04159 tab. 4] | | Single-image 3D face shape and dense alignment, one pass and no template fitting | Residual encoder-decoder that regresses a position map | Author-reported NME 3.62 on AFLW2000-3D against 5.42 for 3DDFA [arxiv:1803.07835 tab. 1], and 9.8 ms per image on a GTX 1080 against 75.7 ms for 3DDFA [arxiv:1803.07835 tab. 2]. The 9.8 ms excludes face detection [arxiv:1803.07835 §4.4]. The training code and trained weights are not released, and the paper's training description omits several details [arxiv:1803.07835 §3.3] | | Pure ConvNet backbone for classification or dense prediction, standard layers only | ConvNeXt block: 7x7 depthwise, inverted bottleneck, layer scale | ConvNeXt-T reaches 82.1% against 81.3% for Swin-T on ImageNet-1K at 224 pixels [arxiv:2201.03545 tab. 1]. On an A100 with TF32, ConvNeXt-T runs at 1943.5 images/s against 1325.6 for Swin-T [arxiv:2201.03545 app. E] [arxiv:2201.03545 tab. 12]. The paper notes that depthwise convolutions are slower and use more memory than dense ones at equal FLOPs [arxiv:2201.03545 §4] | | Scene text recognition where attention drifts off characters, and some training images have character pixel labels | Attention decoder on a ResNet-style encoder, plus a focusing network with lambda = 0.01 | Unconstrained accuracy 87.4% on IIIT5k and 70.6% on IC15 at lambda = 0.01, with lambda swept from 0.001 to 0.1 [arxiv:1709.02054 tab. 6] [arxiv:1709.02054 tab. 7]. The baseline without the network is the same ResNet-based model, and the card's Results section compares the two [arxiv:1709.02054 §4.3] | | Dense completion of partial 3D shapes on a voxel-style grid, from one partial scan | Grid encoder-decoder with additive skips, then point read-out | Chamfer 2.723 on ShapeNet against 4.016 for PCN, both x10^-4 [arxiv:2006.03761 tab. 1]. The card lists the decoder stride as a pitfall, so the decoder size must be checked against the paper's description [arxiv:2006.03761 §3.3] | | Improving a pretrained generative policy from offline data with a bounded correction | Residual MLP on a fixed flow prior, with a Lagrangian budget on the correction | Cube-double 40 for DeFlow against 29 for FQL, and puzzle-3x3 43 against 30 [arxiv:2601.10471 tab. 1]. SORL is stronger on several other tasks in the same table, and the card says the comparison is not compute-matched [arxiv:2601.10471 §4.1] [arxiv:2601.10471 §4.3]. The paper does not claim a monotone depth effect [arxiv:2601.10471 app. C.4] | | Residual network that must run on encrypted inputs, with the weights already trained in plaintext | Standard residual blocks with polynomial ReLU and bootstrapping, as in the encrypted ResNet-20 | Author-reported 98.67% agreement with plaintext ResNet-20 on 75 images, and 90.67% accuracy on those images against 89.33% for plaintext [arxiv:2106.07229 tab. 4]. About 4 hours per image [arxiv:2106.07229 tab. 5]. The paper names batching across images and hardware accelerators as the next steps for run time [arxiv:2106.07229 §5], so the variant suits a design study more than a deployment today | | Need the exact published training recipe | Check the card before implementing | The cards record training gaps in most of the thirteen papers, for example [arxiv:2012.02181 app. A] [arxiv:1904.07396 §4.1] [arxiv:1709.00753 §III-D] [arxiv:2609.37870 §4.4] [arxiv:1803.07835 §3.3] [arxiv:2102.04159 tab. 7] [arxiv:2006.03761 §4.3] [arxiv:2106.07229 §4.1] | ## Implementation sketch The shared core is one line of arithmetic. The variants differ inside the block, in the shortcut, and in the combine rule. ```pseudocode # Common core: output = shortcut(x) + F(x) [arxiv:1711.10684 §2.1.2] def residual_unit(x, block, shortcut=identity): # Learned residual F(x): one or more layers [arxiv:1711.10684 §2.1.2] residual = block(x) # Identity shortcut when shapes match; when they do not, a projection # or padding is needed. The segmentation paper does not say which # [arxiv:1711.10684 §2.1.2]; the encrypted ResNet-20 uses a 1x1 conv # [arxiv:2106.07229 tab. 1] return shortcut(x) + residual ``` ```pseudocode # Variant: global residual over a zero-filled input [arxiv:1709.00753 §III-E] def refine_reconstruct(zero_filled, body): # The network predicts a correction, not the image [arxiv:1709.00753 §III-E] correction = body(zero_filled) # The skip spans the whole network: input plus predicted residual # [arxiv:1709.00753 §III-E] return zero_filled + correction ``` ```pseudocode # Variant: full pre-activation block, two BN-ReLU-conv stages # [arxiv:1711.10684 §2.1.2] [arxiv:1711.10684 tab. I] def preact_block(x, filters, stride): # Normalise and activate before each convolution [arxiv:1711.10684 §2.1.2] h = conv3x3(relu(batch_norm(x)), filters, stride=stride) # Second stage keeps the spatial size [arxiv:1711.10684 tab. I] return conv3x3(relu(batch_norm(h)), filters, stride=1) ``` ```pseudocode # Variant: MLP block with linear skip and layer norm [arxiv:2304.08424 §4] def mlp_block(x): # ReLU hidden layer, dropout on the output layer only [arxiv:2304.08424 §4] h = dropout(linear_out(relu(linear_hidden(x)))) # Fully linear skip, then layer norm on the sum [arxiv:2304.08424 §4] return layer_norm(h + linear_skip(x)) ``` ```pseudocode # Variant: spike-element-wise block, combine rule g [arxiv:2102.04159 eq. 9] # [arxiv:2102.04159 §3.3] def sew_block(s, branch, rule="IAND"): # Spiking residual branch A = SN(F(S)) [arxiv:2102.04159 eq. 9] a = branch(s) # ADD is not binary; the output of k blocks is at most k + 1 [§3.3] if rule == "ADD": return a + s # IAND keeps spikes binary; for spikes it equals (1 - A) * S [§3.3] if rule == "IAND": return (1 - a) * s raise ValueError(rule) ``` ```pseudocode # Variant: residual correction on a fixed prior [arxiv:2601.10471 §3.2] [alg. 1] def residual_policy_act(s, prior_sample, residual, bounds): # Base action from the frozen flow prior; its gradient is stopped [§3.2] base = stop_gradient(prior_sample) # Correction is an MLP of the state and the sampled action [§3.2] shift = residual(s, base) # Executed action is the clipped sum [alg. 1] return clip(base + shift, bounds) ``` ## Contradictions No two papers report incompatible results for the same setting. Several cards show gaps between the paper text and the released code, and some text is unclear. None of these is resolved here. - **GRNet, the word "residual".** The title names a residual network, but the body describes plain convolution blocks with additive U-Net-style skips [arxiv:2006.03761 §3.3]. The card does not explain the name. The paper's decoder stride cannot reach the stated 64 cubed grid, while the released code uses stride 2 and adds maps of matching size [arxiv:2006.03761 §3.3]. See the card's Paper vs code section. - **RIDNet, skip paths.** The paper adds each module's output to its input and adds the extracted features to the body output [arxiv:1904.07396 §3.2] [arxiv:1904.07396 eq. 6]. The card's Paper vs code section records that the released model does not include these additions as written. - **TiDE, block count.** The paper describes the encoder as multiple residual blocks and tunes the layer count over {1, 2, 3} [arxiv:2304.08424 §4.1] [arxiv:2304.08424 tab. 7]. The card records that the released code builds one fewer block than the layer count, so a count of one has no residual block, and the published Traffic, Weather and ETTm1 settings use one [arxiv:2304.08424 tab. 8]. See the card's Paper vs code section. - **RefineGAN, residual block size.** The text says `conv_i`, which has stride 1, "reduces the dimension of the tensor by half". Read literally, a stride of 1 does not halve the spatial size, and the intended design is not stated [arxiv:1709.00753 §III-E]. See the card's Pitfalls. - **RIDNet, internal numbers.** The text gives 3.82 dB and 4.14 dB over NC and NI [arxiv:1904.07396 §4.3.4], while the table implies 3.80 dB and 4.12 dB [arxiv:1904.07396 tab. 5]. The card records the mismatch. - **SEW ResNet, block counts.** The paper says there are 7, 15 and 15 SEW blocks in ResNet-18, -34 and -50 [arxiv:2102.04159 §4.1]. The card's Paper vs code section records that the released code builds 8, 16 and 16, and the paper does not explain the difference. - **ConvNeXt, large kernels.** Section 2.5 says the ResNet-200 regime also stops gaining at 7x7 [arxiv:2201.03545 §2.5]. Appendix C says it saturates at 5x5 instead [arxiv:2201.03545 app. C]. Table 11 gives 82.32 for 5x5 and 82.30 for 7x7 [arxiv:2201.03545 tab. 11]. The kernel sits inside the residual branch, not on the shortcut. ## Papers - [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](/papers/1709.00753): a residual encoder-decoder generator that adds its output to the zero-filled image, trained adversarially with a cyclic data-consistency loss and chained with a second generator. - [Road Extraction by Deep Residual U-Net](/papers/1711.10684): full pre-activation residual units in a U-Net-style network with no cropping, reported to beat U-Net on one road dataset. - [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](/papers/1803.07835): a residual encoder-decoder that regresses a UV position map of 3D face points from one image, with a region-weighted loss and no 3D model fitting. - [Real Image Denoising with Feature Attention](/papers/1904.07396): residual-on-residual modules with channel feature attention for single-stage blind denoising of real photographs. - [BasicVSR](/papers/2012.02181): residual blocks refine warped features in a bidirectional recurrent video super-resolution baseline. - [Deep Residual Learning in Spiking Neural Networks](/papers/2102.04159): a spike-element-wise residual block that combines the branch and the input with ADD, AND or IAND, trained directly on ImageNet and DVS data. - [Encrypted ResNet-20 with CKKS bootstrapping](/papers/2106.07229): a ResNet-20 with bootstrapping and polynomial ReLU, evaluated on encrypted CIFAR-10 images with the plaintext weights. - [Long-term Forecasting with TiDE](/papers/2304.08424): residual MLP blocks with a linear skip, used as the building block of a long-horizon forecaster. - [A ConvNet for the 2020s](/papers/2201.03545): an inverted-bottleneck residual block with 7x7 depthwise convolution and layer scale, reached by modernising a ResNet-50 step by step. - [Learning from synthetic photorealistic raindrop for single image raindrop removal](/papers/2609.37870): bottleneck residual blocks in a three-stage detector, reconstructor and refiner, trained on a rendered raindrop dataset. - [Focusing Attention: Towards Accurate Text Recognition in Natural Images](/papers/1709.02054): a ResNet-style encoder with an attention decoder, plus a focusing network trained on part of the data with character pixel labels. - [GRNet: Gridding Residual Network for Dense Point Cloud Completion](/papers/2006.03761): a grid encoder-decoder with additive skips that completes partial point clouds through a gridding step. - [DeFlow: Decoupling Behavior-Prior Modeling and Value Maximization for Offline Policy Extraction](/papers/2601.10471): a fixed flow-matching prior with a small residual MLP that shifts each sampled action toward higher critic value.