Convolutional neural network

Overview

A convolutional layer applies the same small set of learned filters at every position of its input, so each output value depends only on a local neighbourhood. Stacking such layers with nonlinearities and down-sampling gives features that cover a larger area as depth grows. The cited papers use this one recipe at very different scales: a two-layer classifier on 32 x 32 images 2303.14126 fig. 5, a one-dimensional network over 200-point GPS segments 1804.02386 §3.2, a three-dimensional network that predicts a deformation field between two volumes 1807.03361 §2.3, and deep encoder-decoder networks for per-pixel labels 1612.05360 §3.1.

The authors of one cited analysis read convolutions as high-pass filters that keep fine detail, in contrast to self-attention, which they read as a low-pass smoothing step 2202.06709 §3. That reading is the authors' interpretation, not a measured property of every CNN. A 2016 survey gives the same building blocks and the AlexNet recipe as a starting point 1601.06615 §2. A 2018 survey gives the convolution and pooling formulas and the output-size rules, then walks through LeNet to DenseNet 1803.01164 §III. Most cited papers are applications with their own measurements. A 2021 survey of detectors lists the classifier backbones that detectors reuse, such as ResNet and EfficientNet, and copies their ImageNet top-1 from the source papers 2104.11892 §4 2104.11892 tab. 2. A free textbook covers the same building blocks with runnable code, and its convolutional chapter starts on page 239 2106.11342 p. 239. One of the cited papers is a position paper on brain-inspired AI, with no equations or experiments and no convolutional design detail 1706.01040 §3, so it adds no evidence for the choices below.

A 2021 systematic review of machine learning for disease diagnosis names CNNs as the most common method. That is a count of the reviewed papers, not a controlled comparison 2112.15538 §7.

Two 2021 reviews of application areas show the same pattern. Of the 232 articles and proceedings that one review coded for UAV imagery, 91.2% use a CNN 2101.10861 §3. A weed-detection survey notes that pre-trained CNNs are common, while the accuracies it lists come from different crops, datasets and metrics and are not comparable 2103.01415 §10.1.1 2103.01415 §11. Both are counts of the reviewed work, not controlled comparisons.

CNNs can also learn attributes that clinicians cannot see. Standard CNNs trained on medical scans predicted self-reported race with one-vs-all AUC of about 0.96 to 0.99 on chest X-rays 2107.10356 tab. 3, and the authors could not isolate the image features responsible 2107.10356 p. 21.

Variants

Encoder-decoder with residual blocks and summation skips

Four down-sampling levels with 64 to 512 feature maps, a 1024-map bridge at 40 x 40, and a decoder that deconvolves and adds the matching encoder map instead of concatenating it 1612.05360 §3.1 1612.05360 tab. 1. Each level uses residual blocks. Reported on one electron-microscopy volume per dataset, with V_rand 0.97804 against 0.97276 for U-Net on Drosophila 1612.05360 tab. 2 and Dice 0.9461 against 0.9085 on zebrafish 1612.05360 tab. 3. Card: FusionNet.

Fully convolutional segmentation with object-level voting

A VGG16-based fully convolutional network, fine-tuned from ImageNet weights, labels every pixel of 1000 x 1000 px patches. The patches are stitched back into the full image, and each object takes the majority label of its pixels 1706.06480 §4.2 1706.06480 fig. 3. The authors use this to avoid resizing objects, since resizing destroys the texture they need 1706.06480 §4.1. Training is staged: the network grows from FCN-32s to FCN-16s and FCN-8s, with a learning rate that drops at each stage 1706.06480 §4.3.3. On 21 steel image pairs, with an 11/10 image split, the authors report 93.94% object accuracy against 66.50% for an object-based VGG16 1706.06480 tab. 1 1706.06480 §5.1. Class-balanced patches and rotation augmentation account for the rise from 90.97% to 93.94% 1706.06480 tab. 2. Card: Advanced Steel Microstructural Classification.

Inception-style recurrent residual blocks

Each block runs parallel 1 x 1 and 3 x 3 branches whose convolutions are unrolled over time steps, adds a pooled branch, and adds the block input back as a shortcut 1811.04241 §3 1811.04241 eq. 3 1811.04241 eq. 4. The experiments use two time steps of recurrence after one feed-forward convolution 1811.04241 §4. On BreakHis at 40x, the authors report 97.95% image-level binary accuracy with augmentation, against 97.16% without it 1811.04241 tab. 4. Card: IRRCNN.

Small two-layer image classifier

Two convolution blocks with 32 filters and max pooling, a 64-unit dense layer, and one sigmoid output 2303.14126 §3.3 2303.14126 fig. 5. It reaches 93.55% accuracy and F1 0.936 on the authors' 20,000-image test split 2303.14126 tab. 7 2303.14126 tab. 11. The paper's headline 92.98% comes from a different topology with 128 filters 2303.14126 tab. 2. No external baseline is reported 2303.14126 §4.2. Card: CIFAKE.

Two-convolution gesture classifier converted with TensorFlow Lite

Two blocks, each a 3 x 3 convolution, ReLU, 2 x 2 max pooling and dropout, with 32 and then 64 filters, followed by a 128-unit dense layer and a five-way softmax 2110.00791 §2.2 2110.00791 tab. 1. The same design is trained at four square input sizes, from 64 to 256 pixels, and the authors choose 96 x 96 2110.00791 §2. Training uses Adam with batch size 32, at most 25 epochs, and a stop once validation loss has risen for three epochs 2110.00791 §2.2. The trained model is then converted with TensorFlow Lite, which the authors say lowers the precision of the parameters. They also describe structured pruning on a representative dataset, but give no ratio 2110.00791 §3. At 96 x 96 the authors report 98.11% accuracy and 47,864,936 bytes before conversion, against 96.9% and 15,942,712 bytes after 2110.00791 tab. 2. The two accuracies come from different sets, 100 validation images against 32 test images, and the paper does not show that they are comparable 2110.00791 tab. 2. The paper also does not say what "model size" measures 2110.00791 §3. Card: Optimizing Neural Network for Computer Vision task in Edge Device.

One-dimensional CNN over sensor sequences

Each 200-point segment is a 1 x 200 x 4 tensor of speed, acceleration, jerk and bearing rate 1804.02386 §3.2.1. The network uses 1 x 3 convolutions with stride 1, 1 x 2 max pooling, ReLU and a five-way softmax 1804.02386 §3.2. A single model reaches 79.8% test accuracy, and a seven-member bootstrap ensemble reaches 84.8% 1804.02386 tab. 4. Both numbers use a test-set epoch choice, so they are optimistic 1804.02386 §5.1. Card: Transportation modes from GPS.

Convolutional features feeding an LSTM for robot force signals

Each sample is 1000 frames by 24 channels, the joint and force readings of a teleoperated arm sampled at 100 Hz 2402.10220 §III-D 2402.10220 §IV-A. Each channel is z-scored 2402.10220 eq. 6. A stack of four convolution and pooling blocks, then flatten, dropout and dense layers, gives a six-way softmax. The text and the figures disagree on the convolution dimension and pooling type 2402.10220 fig. 7a 2402.10220 §IV-A. For forecasting, the same convolutional encoder feeds an LSTM with four stacked layers of 100, 100, 50 and 50 units and a 200-unit dense layer, which predicts the next 500 frames 2402.10220 fig. 8d 2402.10220 §IV-E. The paper reports a single-stream classifier at 86% accuracy, a dual-stream at 84% and a quadruplet-stream at 89% on its second dataset 2402.10220 §IV-C. The forecaster's RMSE in the table is 1.91e-05 for the quadruplet hybrid, against 0.065 for an LSTM fed the spatial mean of the channels 2402.10220 tab. III. The authors do not say whether the convolutional encoder is frozen or fine-tuned during forecasting 2402.10220 §IV-E.2. Card: Towards Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection from Force Data.

Three-dimensional CNN for weakly supervised registration

A 3D encoder-decoder with four down-sampling and four up-sampling blocks. Each block has two convolution, batch-norm and ReLU layers, and the first convolution uses a 7 x 7 x 7 kernel, while the rest use 3 x 3 x 3 1807.03361 §2.3. The output is a dense displacement field that warps a moving MR volume onto a fixed ultrasound volume. Training uses only sparse anatomical label pairs, scored with a multiscale soft Dice, plus a bending-energy penalty 1807.03361 §2.1 1807.03361 §2.2. Under 12-fold patient-level cross-validation on 108 prostate pairs, the authors report a median landmark error of 3.6 mm and a median gland Dice of 0.87 1807.03361 tab. 1. Training ran for 48 hours on one 12 GB GPU 1807.03361 §3.2. Card: Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration.

One-dimensional CNN encoders for molecules and proteins

A library exposes a 1D CNN over SMILES characters and another over protein residues, each with global max pooling. The two embeddings are concatenated and passed to an MLP 2004.08919 §2.1 2004.08919 app. A. On the DAVIS benchmark the authors report MSE 0.254 and concordance index 0.879 for the CNN+CNN pair, against 0.262 and 0.870 for DeepDTA 2004.08919 fig. 1. These values are read from a low-resolution figure, and the paper does not state the training hyperparameters used for them 2004.08919 §3. Card: DeepPurpose.

Small U-Net with depthwise-separable convolutions and attention

A U-Net whose double convolutions are depthwise-separable, with a CBAM attention block on each skip path, predicts rain maps 30 minutes ahead from 12 five-minute radar frames 2007.04417 §3.1 2007.04417 §4.1. The model has 4.1 million parameters against 17.3 million for U-Net 2007.04417 tab. 1. On the authors' NL-50 test set, both models reach MSE 0.0122, and the model has slightly lower NMSE, 416.10 against 416.38 2007.04417 tab. 4. The training loss in the released code is a per-image sum, not the mean of the paper's equation 2007.04417 eq. 1. Card: SmaAt-UNet.

Deep pretrained-style backbone inside federated learning

A VGG16 backbone with one extra convolution and three dense layers before the output, trained on grayscale 256 x 256 images with Adam at a learning rate of 0.0001 and focal loss 2101.07511 §3.2.2. Clients are grouped by imaging modality, and the server averages weights within and across groups 2101.07511 §3.1 2101.07511 alg. 1. Card: Collaborative federated learning.

CNN backbone with local self-attention at stage ends

The AlterNet design is a pre-activation ResNet-50 in which some convolution blocks at the end of stages are replaced by local multi-head attention 2202.06709 app. E. The paper states a 4 x 4 window for CIFAR; the released code defaults to 7 2202.06709 app. E 2202.06709 §4.1. The authors report that it beats CNN baselines on CIFAR-100 under corruption 2202.06709 §4.2, with the values given only in figures, and is on par with Swin in ImageNet accuracy, a comparison that uses different training pipelines 2202.06709 app. E. Card: How Do Vision Transformers Work?.

Generic scene features from a scene-centric dataset

AlexNet, GoogLeNet and VGG-16 are trained on a dataset of about 2.4 million images in 205 scene categories, and on a 365-category subset of about 1.8 million images 1610.02055 §2.3.1 1610.02055 §3. The authors' VGG model scores 58.90% top-1 on Places205 test and 55.19% top-1 on Places365 test, with 10-crop averaging 1610.02055 tab. 1 1610.02055 tab. 2. For reuse as a fixed feature extractor, they take fc7 activations (4096-d) or the pooled layer of GoogLeNet (1024-d) and train a linear SVM with C = 1 1610.02055 §3.3. The paper does not state the learning rate, batch size or iteration count for these CNNs 1610.02055 §3. Card: Places.

Seven-layer detector for computers without a GPU

Built from Tiny-YOLOv2 by trial and error: the authors vary layer count, filter count and input size, keep the best variant, and retrain it 1811.05588 §3.1. The chosen network has seven convolutions with 16 to 256 filters, 2 x 2 max-pooling after the first five, and a linear 1 x 1 convolution that outputs a grid of five boxes per cell 1811.05588 §4.1. The authors remove batch normalisation, on the argument that it costs time in a small network 1811.05588 §3.5. Trained on PASCAL VOC, it reaches 33.57% mAP at 21 FPS on one laptop without a GPU, against 40.48% and 2.4 FPS for Tiny-YOLOv2 1811.05588 tab. III. The paper gives 12.26% mAP on COCO at the same speed 1811.05588 tab. V. Card: YOLO-LITE.

Structured sparsity: pruning convolution weights at four grain sizes

Magnitude pruning removes convolution weights at one of four grain sizes: single weights, sub-kernel vectors, whole kernels or whole filters. Each grain is scored by the sum of its absolute weights, and the lowest-scoring grains go first 1705.08922 §3.3 1705.08922 eq. 1. The four grains are compared at equal density, with the same method and training schedule, on ImageNet 1705.08922 §4. At matched density, fine-grained pruning gives the highest top-5 accuracy for all five models, and vector pruning lands between it and kernel pruning 1705.08922 tab. 1. Coarser grains need fewer index bits. At the baseline accuracy, vector pruning gives the lowest storage ratio for VGG-16, GoogLeNet and ResNet-50, while fine-grained pruning does so for AlexNet and DenseNet-121 1705.08922 tab. 2. In a simulated SCNN-style dataflow on VGG-16 convolutions, vector pruning needs 30 to 35% fewer output memory references than fine-grained pruning 1705.08922 tab. 3. The results are author-reported. Card: Exploring the Regularity of Sparse Structure in Convolutional Neural Networks.

Pretrained CNN as a starting point for fine-tuning

Eleven ImageNet-pretrained PyTorch CNNs, from AlexNet to MnasNet, are adapted to five small image sets. Each is tuned in one of two ways: only the classifier is retrained, or every layer is 2605.11989 §4. The score is accuracy divided by parameter count, which the authors call accuracy density 2605.11989 eq. 1. Training uses Adam, batch size 10 and learning rates from {1e-2, 1e-3, 1e-4}, with early stopping on validation accuracy 2605.11989 §4. The authors report SqueezeNet as the densest model under full tuning and ResNet-18 under classifier-only tuning, with one pass over the data 2605.11989 §5.1.1 2605.11989 §5.1.2. With ten passes, ResNet-18 is again the most efficient under classifier-only tuning 2605.11989 §5.2.2. The text gives no numeric accuracy, density or time, and the values appear only in plotted figures 2605.11989 fig. 3 2605.11989 fig. 4. Card: A Transfer Learning Evaluation of Deep Neural Networks for Image Classification.

Linear convolutional network (theory of implicit bias)

No nonlinearities: the network is a product of full-width circular convolutions, and its last layer is a dense vector 1806.00468 §2. With exponential loss on separable data, gradient descent on the weights converges in direction to a stationary point of a penalty on the Fourier coefficients, the l_{2/L} quasi-norm, so the depth L changes which separator is chosen 1806.00468 §3. A fully connected linear network of any depth converges to the hard-margin SVM direction instead 1806.00468 §3. The paper has no experiments 1806.00468 §6. Card: Implicit Bias of Gradient Descent on Linear Convolutional Networks.

Classic AlexNet-style stack as the reference design

Five convolutional layers and three fully connected layers, with 96 first-layer filters of size 11 x 11 x 3 1601.06615 §2.1 1601.06615 fig. 1. Max pooling follows layers 1, 2 and 5, dropout is applied to the last two fully connected layers, and the survey names mini-batch SGD with batches of about 100 as the usual optimiser 1601.06615 §2.5 1601.06615 §2.3.1. The learning rate, momentum, dropout rate and weight decay were hand-tuned and are not given in the survey 1601.06615 §2.5. The survey runs no experiments of its own. Its figures of about 3% from augmentation and 1 to 2% from ensembles are summaries of cited studies 1601.06615 §2.4. Card: A Taxonomy of Deep Convolutional Neural Nets for Computer Vision.

Text CNN over word vectors for sentence labels

Each sentence becomes a 50 x 100 matrix of word2vec vectors, padded or cut to 50 tokens 1704.06841 §3. Two groups of two convolution layers follow, each with 256 filters of width 5 and ReLU, and each group ends in max pooling. A dropout of 0.5, a 128-unit dense layer, a second dropout and a 26-way softmax complete the network 1704.06841 §3 1704.06841 fig. 1. The authors report 0.68 accuracy, against 0.28 to 0.51 for four baselines 1704.06841 fig. 2. The sentences come from encyclopedia-style articles with one label per article, the dataset is not released, and there is no separate test split 1704.06841 §3 1704.06841 §4. Card: Medical Text Classification using Convolutional Neural Networks.

Text CNN with FastText embeddings for user-level labels

Follows Kim (2014) with filter size 3 and pooling size 4, trained for 100 epochs with one model per condition. A user's posts are concatenated, cut to 15,000 tokens and embedded with FastText vectors 1806.05258 §5. Across nine conditions it is best only on eating, with binary F1 of 44.53, while FastText is best on 7 of 9 binary tasks 1806.05258 tab. 4. The authors attribute FastText's lead to reading all of a user's posts, where the CNN samples a portion of them 1806.05258 §5. Card: SMHD.

Greedy layer-wise CNN with auxiliary classifiers

Each block is a 3 x 3 convolution with bias and ReLU. Downsampling is an invertible 2 x 2 space-to-depth step at some depths, not pooling 1812.11446 §3.1. Each block is trained with a small auxiliary classifier on the frozen output of the blocks before it, so no end-to-end backpropagation is needed 1812.11446 §3.2 1812.11446 alg. 1. On ImageNet, the single-crop top-5 accuracy is 79.7% for one-hidden-layer blocks, against 79.1% for end-to-end AlexNet, and 88.7% for three-hidden-layer blocks, against 88.0% for end-to-end VGG-11 1812.11446 tab. 3. On CIFAR-10 the authors report 88.3% and 91.7% for one- and three-hidden-layer blocks, against 89% for end-to-end AlexNet and 92.5% for end-to-end VGG 1812.11446 tab. 2. The memory saving is an author claim from CIFAR-10, where end-to-end training uses about five times more GPU memory; no ImageNet memory figures are given 1812.11446 §4.1. Card: Greedy Layerwise Learning Can Scale to ImageNet.

Grid CNN baseline for urban traffic

Grid models stack the input time steps into channels, or apply convolutions to each cell of a city grid 2108.09091 §4. The plain CNN is the grid baseline. On single-step TaxiBJ prediction (six input steps, one output step), it reaches RMSE 23.550, against 18.702 for ST-ResNet, which uses residual CNN blocks over closeness, period and trend inputs, and 17.826 for STDN, which combines local CNNs with LSTM and attention 2108.09091 tab. 4 2108.09091 §4.1 2108.09091 §5.1. The authors say no grid model dominates, with STDN generally strongest 2108.09091 §5.2. Card: DL-Traff.

ConvNeXt-B shared encoder for matching images across views

One ConvNeXt-B backbone, about 88 million parameters, embeds both views, and features are mean-pooled with no attention pooling 2303.11851 §4.2 2303.11851 §3.2. On the VIGOR same-area split, the ConvNeXt-B row reaches R@1 75.81, against 69.89 for the authors' ViT variant of similar size 2303.11851 tab. 4. The ConvNeXt choice rests on this ablation 2303.11851 §4.3.1. Card: Sample4Geo.

Highway layers: gated skip connections

A highway layer outputs a sigmoid-gated mix of a transform H(x) and the untouched input x 1505.00387 §2, eq. 3. The gate bias starts negative, so each layer begins by carrying its input 1505.00387 §2.2. The convolutional version, a 19-layer network trained directly, reaches 92.24% on CIFAR-10 test 1505.00387 tab. 1 1505.00387 §3.2. The authors report that 100-layer fully connected highway nets train with plain SGD where plain nets of that depth degrade 1505.00387 §3.1. The paper reports no variance across seeds 1505.00387 §3.1. Card: Highway Networks.

Patch-trained CNN converted to a whole-image classifier

A patch classifier is trained on lesion-centred patches. Its convolutional layers are then run on a whole image, followed by two convolutional top blocks, global average pooling and a softmax output, with no heatmap in between 1708.09427 §2.2 1708.09427 fig. 1. Training is two stages on image labels: first the new top layers, then all layers 1708.09427 §3.1.2. On the DDSM test split, a four-model ensemble with flip averaging reaches per-image AUC 0.91 1708.09427 §3.1.4 1708.09427 fig. 3. The best single model reaches 0.88 with augmentation 1708.09427 tab. 2. Models transferred to INbreast reach 0.95 each and 0.98 as an ensemble on a 107-image validation split 1708.09427 §3.2.2 1708.09427 tab. 4. The authors' code is non-commercial, so this variant is described from the paper alone. Card: Deep Learning to Improve Breast Cancer Early Detection on Screening Mammography.

Residual U-Net without cropping

The encoder-decoder keeps U-Net's shape, but each level is a residual unit in full pre-activation order, downsampling uses a stride-2 convolution, and the decoder needs no cropping because input and output have the same size 1711.10684 §2.1.3 1711.10684 tab. I. It has 15 convolution layers against 23 in U-Net 1711.10684 §2.1.3. It is trained with mean squared error on 30,000 random 224 x 224 patches, without augmentation 1711.10684 §3.2. On the Massachusetts roads test set the relaxed break-even point is 0.9187, against 0.9053 for U-Net 1711.10684 tab. II. The text puts it at about 7.8M parameters against 30.6M for U-Net 1711.10684 §3.4. The paper does not say how the shortcut is reshaped where stride or filter count changes 1711.10684 §2.1.2. Card: Road Extraction by Deep Residual U-Net.

Unrolled cascade with data-consistency layers

A CNN block is followed by a data-consistency layer that puts back the measured k-space samples. The pair is repeated and trained end to end 1704.02422 §3 1704.02422 §4. Each block uses 3 x 3 convolutions with 64 filters, ReLU and a residual connection 1704.02422 §6. For dynamic sequences, a data-sharing layer adds k-space averages from neighbouring frames as extra input channels 1704.02422 §5 1704.02422 §6. At 3-fold undersampling on 2D frames, the MSE is 0.89 x 10^-3, against 2.12 x 10^-3 for DLMRI 1704.02422 tab. 1. The authors time one frame at 23 ms on a GTX 1080 1704.02422 §7.2.3 and one 3D sequence at 8.21 s 1704.02422 §7.3.2. These timings are the authors' own. Card: Cascaded CNN for dynamic cardiac MRI.

Mixup-trained residual network for character images

Three character-image benchmarks compare a 4-nearest-neighbour classifier, a small two-layer CNN and PreActResNet-18, the last with and without mixup 1812.01718 tab. 1 1812.01718 §3.1. On Kuzushiji-MNIST the mean per-class accuracy is 91.56% for the nearest-neighbour baseline, 95.12% for the small CNN, 97.82% for PreActResNet-18, 98.41% with input mixup and 98.83% with manifold mixup 1812.01718 tab. 1. The paper gives no training details for these baselines 1812.01718 §3.1. Some classes in the two larger datasets have few samples 1812.01718 §2. Card: Deep Learning for Classical Japanese Literature.

DenseNet121 on chest radiographs with uncertain-label handling

The model takes 320 x 320 radiographs and outputs one probability per finding 1901.07031 § Training Procedure. Four ways of handling uncertain labels are compared: ignore them, map them to 0 or to 1, self-train on the model's own soft labels, or use a three-way softmax 1901.07031 § Uncertainty Approaches. No single method wins everywhere. On validation AUROC, U-Ones is best for Atelectasis (0.858) and U-MultiClass for Cardiomegaly (0.854), and the paper reports significance only for those two findings 1901.07031 tab. 3 1901.07031 § Results. Ignoring uncertain labels is a poor fit when uncertainty is common: for Consolidation, 12.78% of studies are uncertain against 6.78% positive 1901.07031 tab. 1. On a 500-study test set, the model lies above all three radiologists on Cardiomegaly, Edema and Pleural Effusion, but not above their majority vote 1901.07031 § Test Set 1901.07031 fig. 3. Card: CheXpert. The paper's CC BY licence does not cover the dataset, so check the access terms before use.

Deformable 3 x 3 operator with shared projection and softmax weights

DCNv3 replaces fixed 3 x 3 weights with learned sampling offsets and modulation weights 2211.05778 §3.1. Three changes make it trainable from scratch at scale: one projection shared by all sampling points, channel groups that each have their own offsets, and a softmax over the points in place of DCNv2's sigmoid 2211.05778 §3.1. A sigmoid in place of the softmax drops the small model's top-1 from 83.5 to 65.7 2211.05778 tab. 6. Larger 5 x 5 or 7 x 7 kernels do not improve top-1 2211.05778 app. B. The 30M-parameter variant reaches 83.5 top-1, against 82.1 for ConvNeXt-T 2211.05778 tab. 2. The largest variant reaches 89.6 at 640 px, but its pre-training uses a joint 427M-image set that the paper does not list 2211.05778 tab. 2 2211.05778 §4.1. The authors report that InternImage-B is slower than ConvNeXt-B despite higher accuracy 2211.05778 app. D. Card: InternImage.

Real-time detector with C3k2 and C2PSA blocks

YOLOv11 is described as a review, with no experiments of its own. C3k2 blocks replace C2f in the backbone, neck and head, and a C2PSA attention block follows SPPF 2410.17725 §4.1.1 2410.17725 §4.1.2. The paper gives no channel counts, strides or kernel sizes per stage 2410.17725 §4. Its performance figures are approximate values read from a third-party chart 2410.17725 §6 2410.17725 fig. 2. Card: YOLOv11: An Overview of the Key Architectural Enhancements.

Fine-tuned AlexNet for small photo sets

An ImageNet-pretrained AlexNet has its last three layers replaced for 12 phone classes and is trained on 650 back-side photos 2606.23286 §4. Each training photo is copied nine times with flips, translation of up to 50 pixels, scale between 0.9 and 1.1, and shear 2606.23286 §5.1. At learning rate 3e-4, SGD with momentum reaches 98.35 ± 2.02% over seven trials, against 95.26 ± 3.04% for Adam and 93.20 ± 3.86% for RMSProp 2606.23286 tab. 1. These are validation accuracies, and no separate test set is described 2606.23286 §4. The paper does not say whether augmentation is applied before or after the split, so copies of one photo may land on both sides 2606.23286 §4. Card: Transfer learning-based method for automated ewaste recycling in smart cities.

Reported CNN use in agricultural studies

This is a survey of 40 agricultural deep-learning studies, not a method 1807.11809 §2. Convolutional networks are the most common architecture, in 17 of the 40 studies 1807.11809 §4.6. Of the 24 studies reporting classification accuracy, 19 are above 90%, but only 8 of the 40 tested on data other than their training data 1807.11809 §4.9 1807.11809 §4.10. The survey warns that the results are not directly comparable 1807.11809 §4.9. Card: Deep learning in agriculture: A survey.

U-Net with an ImageNet-pretrained VGG11 encoder

The encoder is the convolutional part of VGG11, initialised from ImageNet weights, and the decoder mirrors it with transposed convolutions 1801.05746 §2. Training uses binary cross-entropy minus the log of a soft Jaccard term 1801.05746 eq. 4. On 30 validation images of the Inria aerial dataset, validation IoU after 100 epochs is 0.593 with random initialisation and 0.686 with the pretrained encoder 1801.05746 §3. A network pretrained on Carvana reaches 0.687 1801.05746 §3. Both pretrained settings converge faster in the authors' curves [arxiv:1801.05746 fig. 3]. Each setting is one run with no seeds or variance reported. Card: TernausNet.

Parallel CNN and transformer branches with feature exchange

A convolutional branch and a transformer branch share one stem and run side by side. A Feature Coupling Unit in each block sends convolutional features into the transformer as pooled patch tokens, and sends transformer tokens back into the convolutional branch after reshaping and upsampling 2105.03889 §3.2. The paper leaves out where the upsampled features enter the CNN block 2105.03889 §3.2. On ImageNet-1k at 224 px, Conformer-S reaches 83.4% top-1 against 81.8% for DeiT-B [arxiv:2105.03889 tab. 2]. A serial stack of ResNet-26d and DeiT-S reaches 80.2% with 3.7 GMACs, against 81.9% for Conformer-S/32 with 7.0 GMACs [arxiv:2105.03889 tab. 5]. On COCO, Conformer-S/32 in Faster R-CNN reaches 43.1 box AP against 39.4 for ResNet-101 FPN 2105.03889 tab. 3. Its detection throughput is lower: 13.5 FPS against 15.9 for ResNet-101 on a V100 [arxiv:2105.03889 app. C]. Card: Conformer.

Pure CNN with transformer-style block layout for phones

The block separates spatial mixing, a depthwise 3 x 3 convolution that is folded into one kernel at inference, from channel mixing, a pointwise expansion to twice the width 2307.09283 §3.2. The stem, the downsampling blocks and the squeeze-and-excitation placement follow the same ViT-inspired choices 2307.09283 §3.3 2307.09283 §3.4. The authors state that operation counts predict on-device latency poorly, so they measure latency on an iPhone 12 with Core ML 2307.09283 §3.1. With distillation, RepViT-M1.0 reaches 80.0% top-1 at 1.0 ms, against 79.0% at 1.1 ms for EfficientFormerV2-S1 2307.09283 tab. 1. Without distillation the same model reaches 78.6% 2307.09283 tab. 2. The step-by-step ablation starts from 71.5% for MobileNetV3-L with the aligned recipe and ends at 77.4% for the full block 2307.09283 §3.1 2307.09283 §3.4. Card: RepViT.

Random-view 2.5D classifier as a second-stage filter

Each candidate from an existing detector is cropped at several physical scales and under random shifts and rotations, giving about 100 views per candidate 1505.03046 §III-A. A small network reads three orthogonal central slices as one 2.5D patch, and the candidate score is the mean of the per-view probabilities 1505.03046 §II-B 1505.03046 eq. 1. The network has two convolution layers with max pooling, two locally connected layers and a fully connected layer with DropConnect 1505.03046 §II-A 1505.03046 fig. 1. At 3 false positives per volume on lymph nodes, sensitivity is 70% in the mediastinum and 83% in the abdomen, against prior results of 55% and 30% 1505.03046 §III-E. Card: Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation.

Encoder-decoder with side-output supervision for saliency

A VGG16 encoder with its last pooling layer removed feeds three decoders that start from conv3-3, conv4-3 and conv5-3 and upsample by deconvolution back to the input size 1705.02544 §3.3.1 1705.02544 §3.3.2. Each side output has its own pixel-wise cross-entropy loss, and the fused map has one too 1705.02544 eq. 7 1705.02544 eq. 8. A learned fusion layer combines the three side maps 1705.02544 §3.2. Trained on 10,000 SALICON images, it is reported as best on most metrics for four of five eye-tracking benchmarks, but not on MIT300 1705.02544 §4.3 1705.02544 tab. 4. On TORONTO, dropping the side-output losses lowers s-AUC from 0.76 to 0.71 1705.02544 tab. 9. Card: Deep Visual Attention Prediction.

Joint distribution adaptation for a CNN on unlabelled target data

A CNN is trained on labelled source data, then adapted on unlabelled target data. The penalty has a marginal term and one term per class, and the class terms use target pseudo-labels from the network 1804.07265 §III-A 1804.07265 eq. 10. The authors report 97.5% average accuracy over ten fault-diagnosis transfer tasks, against 87.4% with marginal-only adaptation and 80.0% for a plain CNN 1804.07265 tab. III. The base architecture is not stated, and the penalty weight was chosen by a grid search whose validation data is not named 1804.07265 §III-B 1804.07265 §IV-A. Card: Deep Transfer Network with Joint Distribution Adaptation.

Deep linear generator for per-image blur-kernel estimation

The generator has no activation functions: five hidden 64-channel convolutions with kernel sizes 7, 5, 3, 1 and 1, then a 1 × 1 output with stride 2 1909.06581 §4.1 1909.06581 fig. 4. Because the layers are linear, their filters multiply into one kernel, which is the blur estimate 1909.06581 §4.2. It is trained for 3,000 iterations against a patch discriminator, on the one low-resolution image alone 1909.06581 §2. With ZSSR as the super-resolution step, the authors report 30.363 dB at ×2 on DIV2KRK, which uses synthetic blur, against 29.368 dB for the earlier estimator 1909.06581 tab. 1 1909.06581 §5.2. Each image needs a full training run of 61 s on a V100 1909.06581 §5.3. The official code is for academic use only LICENSE.txt, so a commercial version has to be written from the paper. Card: Blind Super-Resolution Kernel Estimation using an Internal-GAN.

Six-dimensional convolution over putative correspondences

Each putative match between two point clouds is a point in six dimensions: the coordinates of its two ends 2004.11540 §3.2. A 6-D convolutional network with residual blocks gives each match an inlier probability [arxiv:2004.11540 fig. 2]. A weighted Procrustes solve then gives a closed-form pose from those weights 2004.11540 §3.3. The per-point features come from a separate descriptor network 2004.11540 §3.1. On 3DMatch, the authors report 91.3% recall against 74.9% for RANSAC with eight million iterations, at 1.21 s per pair including feature extraction 2004.11540 tab. 1. The paper leaves the weight clip threshold and the refinement optimiser unstated 2004.11540 §4.2. Card: Deep Global Registration.

Text CNN with global average pooling, compared with an LSTM

Each sentence is a sequence of 300-dimensional word embeddings over a 15,000-word vocabulary. Four one-dimensional convolutions, one pooling step, a global average pool and a sigmoid output follow 2006.03541 §4.4 2006.03541 fig. 9. The authors say average pooling works better than max pooling, without giving numbers 2006.03541 §4.4. With word embeddings, the CNN reaches accuracy 0.8006 on Sentiment140, against 0.8282 for the LSTM 2006.03541 tab. 2. Processing time on Sentiment140 with embeddings is 7 min 3 s for the CNN and 1 h 4 min for the LSTM 2006.03541 tab. 7 2006.03541 tab. 8. Card: Sentiment Analysis Based on Deep Learning: A Comparative Study.

Three-block CNN on face and speech features with a sarcasm check

The face branch uses three blocks, each a convolution, batch normalisation, ReLU, 2 × 2 max pooling and dropout 2605.30099 §III.D. A separate CNN reads audio features such as MFCCs, chroma and a mel spectrogram [arxiv:2605.30099 §III.E]. For each 5 to 8 s sub-video, the top three emotions from each branch are compared, and the clip is flagged as sarcasm when no emotion is shared 2605.30099 §III.E 2605.30099 alg. 1. The authors report 95.7% accuracy for the CNN with this check, against 70.14% to 95.6% for the compared models 2605.30099 tab. 3. The evidence is thin: the sarcasm claim rests on one public video 2605.30099 §III.F, and the per-class accuracies differ between the paper's Table 1 and Table 2 2605.30099 tab. 1 2605.30099 tab. 2. Card: Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion Detection.

Small CNN pixel classifier on RGB patches

Each pixel is classified from an RGB patch of 5 × 5 to 9 × 9 pixels centred on it. The CNN has one convolution layer with five 3 × 3 filters and ReLU, followed by max pooling 2609.40212 §2.2.2. Networks were selected on a validation split of the training site and tested on a second site 2609.40212 §2.2.2. On the training site the CNN was best, but on the second site the linear and MLP networks were slightly better 2609.40212 §3.2. The best CNN reaches MCC 0.79 on the second site, against 0.82 for a linear network on 9 × 9 patches and 0.61 for the Excess Blue index threshold 2609.40212 tab. 3 2609.40212 tab. 2. Card: Recognition of Urbanized Areas in UAV-Derived Very-High-Resolution Visible-Light Imagery.

Window-based sentence CNN with max pooling over positions

A filter of width h slides over every run of h consecutive word embeddings. Max pooling over positions leaves one value per filter, so the output length does not depend on the sentence 1708.02709 §3.1.1. The survey notes that typical models use hundreds of filters of several widths §3.1.1. For word-level labels, the same window is centred on the word being labelled, and a conditional random field can score the label sequence §3.1.2. The dynamic CNN replaces fixed pooling with dynamic k-max pooling, which keeps the k strongest features in their original order, so narrow filters can relate distant words §3.2. The survey lists two weaknesses: CNNs need a lot of training data, and they model long-range order poorly §3.2. Its benchmark tables copy results from the cited papers, and a multichannel CNN is listed at 88.1% on SST-2 against 87.2% for a bidirectional LSTM 1708.02709 tab. 6. A convolutional sequence-to-sequence model is listed at 26.36 BLEU on WMT14 English-German and 41.29 on English-French 1708.02709 tab. 7. The survey runs no experiments and does not normalise these rows. Card: Recent Trends in Deep Learning Based Natural Language Processing.

Convolutional token embedding and depthwise projections in a Vision Transformer

A three-stage Vision Transformer. Each stage starts with an overlapping convolution that turns the image, or the previous stage's token map, into tokens and shrinks the grid 2103.15808 §3.1. Inside each block, queries, keys and values come from depthwise 3 x 3 convolutions over the token grid, and keys and values are subsampled with stride 2 to cut attention cost 2103.15808 §3.2 2103.15808 §3.2.2. The model drops the position embedding, because the convolutions supply local order 2103.15808 §3.3. CvT-13, with 20M parameters, reaches 81.6% ImageNet-1k top-1 at 224 px, against 81.8% for DeiT-B with 86M 2103.15808 tab. 3. Removing the convolutional embedding costs 0.9 points in the authors' ablation, from 81.6 to 80.7 2103.15808 tab. 6. Stride 1 on keys and values gives 82.3 at 6.55 GFLOPs, against 81.6 at 4.53 GFLOPs for stride 2 2103.15808 tab. 7. The position-embedding ablation is ImageNet-only, and it shows no loss from removing the embedding 2103.15808 tab. 5 2103.15808 §4.4. The fine-tuning learning rate is not stated 2103.15808 §4.1. Card: CvT.

Per-window attention for a k x k convolution

RFAConv replaces a k x k convolution. It expands each window into k² slots, learns a softmax over the slots of each channel at each position, multiplies that by the expanded features, rearranges each window into a k x k block, and finishes with a stride-k convolution 2304.03198 §3.3 2304.03198 eq. 3. The attention is computed per window, so the weights are not shared across overlapping windows the way a spatial attention map is 2304.03198 §3.1 2304.03198 §3.2. On ImageNet-1k, ResNet18 top-1 rises from 69.59% to 71.23% for 0.16M extra parameters 2304.03198 tab. 3. The authors' RFCBAMConv variant reaches 72.15% 2304.03198 tab. 4. On COCO2017 with YOLOv5n, AP rises from 27.5 to 29.0 for RFAConv and to 29.5 for RFCAConv 2304.03198 tab. 7. The authors state the memory cost: the expanded features are k² times the input size, and some devices may not afford it 2304.03198 §4.6. They also say that replacing many convolutions slows inference, so the count should be limited when speed matters 2304.03198 §4.3. Card: RFAConv.

Time series re-encoded as images for a tiled CNN

A one-dimensional series is turned into three square images: a Gramian angular summation field, a Gramian angular difference field and a Markov transition field. The three are stacked as channels 1506.00327 §2 1506.00327 §3.3. A two-layer tiled CNN, pretrained with topographic ICA, feeds a linear SVM 1506.00327 §3.2 1506.00327 fig. 3. Across 20 UCR datasets the authors report the lowest error on 9 of them 1506.00327 tab. 1. The pretraining uses all series, test inputs included, without labels, so the reported test errors are transductive 1506.00327 §3.2. The same angle image can be inverted on its diagonal, and a denoising autoencoder on it lowers interpolation error at the missing points, by 12.18% to 48.02% in the authors' range, against the raw series 1506.00327 §4.2 1506.00327 tab. 2. Card: Imaging Time-Series to Improve Classification and Imputation.

Small CNN scoring projected grasp images

A two-finger grasp candidate is encoded as a 15-channel 60 x 60 image. Three orthogonal projections each give a mean normal, an occupied-depth map and an occluded-depth map 1706.09911 §5.1. A LeNet-style network with two convolution and pooling stages, a 500-unit layer and a two-class softmax scores it 1706.09911 §5. The training set is about 1.5 million labelled candidates from 55 BigBird objects, scanned from 20 views each 1706.09911 §6. The robot trials use the three-channel form, and the occluded channels add about 2 accuracy points at roughly double the encoding time 1706.09911 §7.2 1706.09911 §8.6. The paper's dense-clutter result is 93% (268 of 288), and 53% without the classifier 1706.09911 tab. 3. Card: Grasp Pose Detection in Point Clouds.

Grasp classifier with occlusion channels and CAD pretraining

The conference version of the same grasp pipeline. Each candidate is voxelised into a 60 x 60 x 60 grid and projected into 15 channels 1603.01564 §3.3. Adding the occlusion channels raises recall at 99% precision from 82% to 89% on the 55 BigBird objects 1603.01564 §4.2 1603.01564 fig. 5. Pretraining on rendered CAD depth images reaches at 4000 iterations what a random start reaches at 20000 1603.01564 §4.3. The authors report 93% grasp success from 288 active-sensing attempts, against 73% in their prior work 1603.01564 §5.4. The labels assume frictionless antipodal grasps 1603.01564 §3.4, which the released code relaxes, so check the card's Paper vs code section before reimplementing. Card: High precision grasp pose detection in dense clutter.

Residual encoder-decoder regressing a position map

Residual encoder-decoder regressing a position map

One RGB face crop goes through one convolution and ten residual blocks to an 8 x 8 x 512 bottleneck, then 17 transposed convolutions produce a 256 x 256 map 1803.07835 §3.2. Each map cell stores the x, y and z position of a fixed point on the face, occluded points included, so one output gives landmarks, dense correspondence and shape 1803.07835 §3.1. The loss weights the cells 16 : 4 : 3 : 0 for landmarks, eyes, nose and mouth, other face, and neck 1803.07835 §3.2 1803.07835 tab. 3. Training uses Adam at 0.0001, batch size 16, and 300W-LP with 3DMM-based targets 1803.07835 §3.3. On AFLW2000-3D the authors report mean NME 3.62, against 4.49 for 3DSTN and 5.42 for 3DDFA 1803.07835 tab. 1. The network runs in 9.8 ms per image on a GTX 1080 1803.07835 §4.4. Card: Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network.

Modernised pure ConvNet with large depthwise kernels

The ConvNeXt block is a 7 x 7 depthwise convolution, a LayerNorm, a 1 x 1 expansion by four, one GELU, a 1 x 1 projection, a learned per-channel scale and the residual add 2201.03545 §2.4 2201.03545 §2.5 2201.03545 §2.6 2201.03545 fig. 4. It is reached from a ResNet-50 by a sequence of recipe and design changes, each kept only if it improved ImageNet-1K accuracy 2201.03545 §2 2201.03545 tab. 10. Downsampling uses separate 2 x 2 stride-2 convolutions, each with a LayerNorm, and the stem is a 4 x 4 stride-4 patchify layer 2201.03545 §2.2 2201.03545 §2.6. ConvNeXt-T reaches 82.1% top-1 at 224 px, against 81.3% for Swin-T 2201.03545 tab. 1. Card: A ConvNet for the 2020s.

Sentence CNN as the encoder of a dialog-act classifier with history

Each short text becomes one vector from a convolutional encoder. Filters of height 3 slide over its word vectors, followed by ReLU and max pooling over positions 1603.03827 §2.1.2 1603.03827 tab. 2. A second stage reads the vector of the current text together with the vectors of earlier texts, so the label can depend on the preceding turns 1603.03827 §2.2. The headline CNN accuracies are 65.5% on DSTC 4, 84.6% on MRDA and 73.1% on SwDA, against 66.2%, 84.3% and 69.6% for the LSTM variant 1603.03827 tab. 4. The authors say history at the vector level helps more than history at the label level 1603.03827 §4. For DSTC 4 and SwDA the paper gives only train and test splits, but early stopping needs a validation set 1603.03827 §3.1 1603.03827 §3.2. Card: Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks.

Two-dimensional CNN with temporal difference modules for video

A 2D ResNet runs on a few frames sampled from each video, and two small modules add motion to it. The short-term module adds a feature computed from RGB differences between neighbouring frames to the early stages, and the long-term module compares features of neighbouring sampled segments and uses the result as a gate in later stages 2012.10071 §3.2 2012.10071 §3.3. Adding both modules to a plain 2D baseline changes the top-1 on Something-Something V1 from 46.6% to 52.3% in the authors' ablation 2012.10071 tab. 1. With 8 frames on ResNet-50 the authors report 52.3% top-1 on that dataset at 36 GFLOPs, and 76.6% top-1 on Kinetics-400 with 10 clips and 3 crops 2012.10071 tab. 2 2012.10071 tab. 3. The released code differs from the paper's equations in several places, so the equations alone will not reproduce the numbers; see Contradictions. Card: TDN: Temporal Difference Networks for Efficient Action Recognition.

Continuous convolution with an MLP kernel over point clouds

The kernel is a small MLP of the relative 3D offset between two points, so the layer works on unevenly spaced points and can produce output at new positions 2101.06742 §3.1 2101.06742 §3.2. Factorising the weight tensor keeps the stored kernel values to one per point pair and output channel, not one per input channel as well 2101.06742 §3.3. On indoor scans the authors report 58.27 mIOU, against 48.92 for SEGCloud 2101.06742 tab. 1. On driving data the model on its own scores below the 3D-FCN baseline, at 46.35 mIOU against 49.28, and the best driving result comes from combining it with a 3D-FCN 2101.06742 tab. 2. The driving and flow data are in-house, and no official code was found 2101.06742 §4.2. Card: Deep Parametric Continuous Convolutional Neural Networks.

Local two-dimensional self-attention in a ResNet-like backbone

Local attention takes the place of the spatial convolution in each bottleneck. The image is cut into blocks, and every block attends to one shared window that extends a few pixels beyond it, with no mask 2103.12731 §3.2. The windows are not centred on each pixel, so the layer is equivariant only to shifts that are multiples of the block size 2103.12731 §3.2. Downsampling is a strided attention layer, which replaces attention followed by pooling 2103.12731 §3.2. Under the same recipe, HaloNet-50 reaches 80.5% top-1 against 79.5% for ResNet-50 2103.12731 tab. 3. The largest model, H7, reaches 84.9% top-1 when trained from scratch 2103.12731 §4.1, and pure attention models train more slowly than the matched EfficientNets on the authors' hardware 2103.12731 §5. Keeping convolutions in the first two stages cut the normalised training time from 1.9 to 1.0, at 84.7% against 84.9% 2103.12731 tab. 4. Card: Scaling Local Self-Attention for Parameter Efficient Visual Backbones.

Dual colour-space enhancer with an attention blend

Three blocks for underwater photos. An RGB block of eight 3 x 3 convolutions removes the colour cast. Its output is converted to HSV, where a second block predicts the knots of four piecewise-linear curves for brightness and saturation 2103.07138 §III-A 2103.07138 eq. 1. An attention block sees the raw image and both outputs and predicts per-pixel weights that blend the two results 2103.07138 §III-A. Training uses L1, SSIM, an HSV loss and a VGG-19 perceptual loss 2103.07138 §III-B 2103.07138 eq. 2. Author-reported PSNR on the real-world set is 24.57 dB, against 21.30 dB for UGAN, the next highest PSNR in the table 2103.07138 tab. II. The baselines were run with the authors' pretrained weights, not retrained 2103.07138 §IV-A. The released code differs from the paper in several layers and losses, so read the card's Paper vs code section before implementing. Card: UIEC^2-Net.

Pre-trained CNN features from sentiment, emotion and personality, merged for sarcasm

Three CNNs trained on other labelled data (sentiment, six-way emotion and five binary personality traits) produce fully connected features. These are concatenated with the features of a baseline sarcasm CNN, giving a 1,100-value vector for an SVM. The paper calls this CNN-SVM 1610.08815 §4.1 1610.08815 §5.3. Whether the pre-trained extractors are frozen or fine-tuned is not stated 1610.08815 §4.2. With all feature groups the author-reported F1 is 97.71%, 94.80% and 93.30% on three Twitter sarcasm sets, against 97.60%, 92.32% and 92.20% for the baseline CNN-SVM 1610.08815 tab. 2. Training on one set and testing on another can fall sharply, to 33.05% in one direction 1610.08815 §5.6. Card: A Deeper Look into Sarcastic Tweets.

One-dimensional CNNs as baselines for raw satellite series

Two 1-D convolutional networks, MS-ResNet and TempCNN, classify crop types from 13-band Sentinel-2 series of 70 dates. A Transformer and an LSTM are trained on the same parcels 1910.10536 §4.1 1910.10536 §4.2. In a gradient analysis of one example per class, the Transformer and LSTM show zero gradient on cloudy dates, while the convolutional models show non-zero gradient on some of them 1910.10536 §8.3. The author-reported kappa on raw data is 0.67 ± 0.02 for TempCNN and 0.69 ± 0.03 for MS-ResNet, against 0.71 ± 0.03 and 0.71 ± 0.01 for the Transformer and LSTM on 23 classes 1910.10536 tab. 1. On the 12-class set the gap is wider: 0.71 ± 0.07 for TempCNN and 0.78 ± 0.02 for MS-ResNet, against 0.82 and 0.83 1910.10536 tab. 1. Preprocessing raised every model's scores 1910.10536 §8.1. Card: Self-attention for raw optical Satellite Time Series Classification.

CNN over the last four layers of a language-specific BERT

Each language gets its own BERT-base encoder. The outputs of its last four layers are stacked as four input channels for a CNN. The CNN has 160 filters, 32 for each height from one to five tokens, followed by ReLU, global max-pooling and a sigmoid output 2007.13184 §4.4 2007.13184 fig. 1. The paper gives learning rate 2e-5 and 10 epochs. Its released code runs 6 epochs and uses binary F1 for checkpoint selection, and several training settings in the code are not in the paper 2007.13184 §4.4; see the card's Paper vs code section. Card: KUISAIL at SemEval-2020 Task 12.

Convolutional network under homomorphic encryption

A standard ResNet-20 keeps its trained weights, and the operators are changed so that the whole network runs on encrypted input. ReLU is a composition of minimax polynomials, batch normalisation is a scalar multiply and add, a stride-2 convolution is a stride-1 convolution times a 0/1 window, and the softmax is computed as exp(x/4) normalised 2106.07229 §3.3 2106.07229 §3.4 2106.07229 §3.7. The network uses 1,149 bootstraps 2106.07229 §3.5. Author-reported agreement with the plaintext network is 98.67% on 75 images, where the plaintext network reaches 89.33% and the encrypted one 90.67% 2106.07229 tab. 4. One image takes about 14,694 seconds on a 112-core CPU 2106.07229 tab. 5. The text does not make clear whether the encrypted runs are full runs or simulated 2106.07229 §4.1 2106.07229 §4.2. Card: Privacy-Preserving Machine Learning with Fully Homomorphic Encryption.

Regularised CNN: input, internal and label methods

A survey groups recent CNN regularisers by the place where they act. Input methods change the training images, among them Cutout, Mixup, CutMix and AutoAugment 2201.03299 §3. Internal methods act inside the network, among them Dropout variants, DropBlock, Shake-Shake and ShakeDrop [arxiv:2201.03299 §4]. Label methods soften the target, among them label smoothing, TSLA, SLS and JoCoR 2201.03299 §5. The survey runs no experiments. Its compiled table gives 10.94% error for PyramidNet with ShakeDrop and PBA on CIFAR-100, against a 17.01% baseline 2201.03299 tab. 2, but the rows differ in architecture and schedule, and the card flags printing errors in the table [arxiv:2201.03299 §7.2]. Card: Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks.

Reported CNN use in mineral-exploration remote sensing

A review of machine learning for mapping rock units, alteration, structures and minerals from satellite, airborne and ground-based data. It places convolutional networks in its deep-learning group and names lithology and mineral mapping as their uses, citing UAV, Landsat and AVIRIS studies 2103.07678 §5.5 2103.07678 tab. 2. The authors say few studies apply deep learning to mineral exploration 2103.07678 §5.5. The review runs no experiments, so its few numbers come from cited studies 2103.07678 §6. Card: A review of machine learning in processing remote sensing data for mineral exploration.

When to use which

On small datasets, the same authors report that a ResNet's built-in inductive bias gives lower test error than a ViT at equal training loss 2202.06709 app. C. That is a single-study result, so check it on your own data.

Implementation sketch

The shared core is a stack of convolution, nonlinearity and down-sampling, followed by a classifier head. The variants add skips, branches or recurrence inside the block. Kernel sizes and pool sizes differ by paper and are cited where stated.

# Common CNN core: convolution, nonlinearity, down-sampling  [arxiv:2303.14126 fig. 5]
# Input: images (B, H, W, C); output: one score per class (B, K)

def cnn_classifier(images, filters, dense_units, num_classes, k):
    x = images
    for f in filters:
        # Shared-weight convolution over a k-sized neighbourhood
        # k is paper-specific: 1 x 3 [arxiv:1804.02386 §3.2.2],
        # 1 x 1 and 3 x 3 [arxiv:1811.04241 §3], 3 x 3 x 3 in 3-D [arxiv:1807.03361 §2.3]
        x = relu(conv2d(x, filters=f, kernel=k))
        # Down-sample; pool size is not stated in arxiv:2303.14126
        x = max_pool(x)
    x = flatten(x)
    x = relu(dense(x, units=dense_units))
    # Softmax or sigmoid is applied by the loss  [arxiv:2303.14126 §3.3]
    return dense(x, units=num_classes)

# Residual block: add the block input back to its output  [arxiv:1811.04241 eq. 4]
# Input and output have the same shape (B, C, H, W)

def residual_block(x, body):
    # Shortcut: the input passes through unchanged and is added
    return x + body(x)

# Encoder-decoder skip joins  [arxiv:1612.05360 §3.1]
# The decoder adds the encoder map of the same level instead of concatenating

def decode_level(x, encoder_map):
    x = deconvolve(x, scale=2)
    # Pixel-wise sum; the paper contrasts this with U-Net's concatenation
    return x + encoder_map

# Depthwise-separable convolution  [arxiv:2007.04417 §3.2]
# Per-channel spatial filter, then a 1 x 1 convolution that mixes channels

def dsc(x, channels_out, kernels_per_layer):
    # One spatial filter per output of each input channel
    x = depthwise_conv3x3(x, channels_out=kernels_per_layer * channels_in(x))
    # Pointwise 1 x 1 convolution to the target width
    return conv1x1(x, channels_out=channels_out)

# Highway layer: gated mix of a transform and the input  [arxiv:1505.00387 §2, eq. 3]
# Input and output have the same shape  [arxiv:1505.00387 §2.1]

def highway(x, transform):
    # Gate in (0, 1); its bias starts negative so the layer carries x  [arxiv:1505.00387 §2.2]
    gate = sigmoid(gate_conv(x))
    # Blend the transform with the untouched input  [arxiv:1505.00387 §2, eq. 3]
    return transform(x) * gate + x * (1 - gate)

Contradictions

None known between the cited papers on the same benchmark. Several cards flag numbers that conflict inside a single paper, and these should be read with care:

• CIFAKE quotes 92.98% as its headline, while its final model scores 93.55% 2303.14126 tab. 2 2303.14126 tab. 7.
• Transportation modes from GPS reports a confusion matrix whose printed recall and totals do not match its counts 1804.02386 tab. 5.
• IRRCNN gives 3 trials in one section and 5 in another, and 50 epochs in the text against about 150 in its figures 1811.04241 §4.
• Advanced Steel Microstructural Classification says augmentation adds about 2 points, while its own table shows a gain of about 3 points 1706.06480 §6 1706.06480 tab. 2.
• YOLO-LITE gives three VOC mAP values for its chosen network, 33.81%, 33.77% and 33.57%, in the abstract, a results table and the trial table 1811.05588 tab. III 1811.05588 tab. V. The trial table is the only place where the variant sits beside the others, so this page uses 33.57%.
• A Taxonomy of Deep Convolutional Neural Nets describes dropout as dropping neurons with probability p while scaling activations by p at test time. Those are two conventions, and the survey does not resolve them 1601.06615 §2.3.2. Check the original dropout paper before using it.
• Highway Networks numbers its layers inconsistently. The text maps Highway 1 and Highway 4 to FitNet 1 and FitNet 4, but the table names the FitNet 4 architecture "Highway 2" 1505.00387 §3.2 1505.00387 tab. 1. Use the table's labels.
• Deep Learning to Improve Breast Cancer Early Detection gives 0.88 in its abstract, which is the augmented value. The non-augmented value for the same model is 0.87 1708.09427 tab. 2.
• InternImage gives the stage-width search grid as 48, 64 and 80 in the main text, but as 16, 32 and 64 in its appendix 2211.05778 §3.2 2211.05778 app. B.
• Transfer learning for ewaste recycling describes the frozen layers two ways: all layers except the last three, and the fully connected layer. The two conflict 2606.23286 §4.
• TernausNet gives the encoder as seven convolutions in its text and eight in its released code 1801.05746 §2. Implement the one you need, and check it against your own training run.
• Conformer gives Conformer-S as 83.4% in its ImageNet table but 83.5% with positional embeddings in its ablation table, and as 10.6 GMACs in some tables and 10.3 in others 2105.03889 tab. 2 2105.03889 tab. 6 2105.03889 tab. 8.
• A Survey of Modern Deep Learning based Object Detection Models gives AlexNet eight learnable layers in its text and seven in its backbone table 2104.11892 §4.1 2104.11892 tab. 2.
• RepViT gives a stage ratio of 1:1:7:1 and depth 2:2:14:2 in its text 2307.09283 §3.3. The released code builds 3, 4, 16 and 3 blocks per stage for M0.9, and the paper does not say how it counts depth.
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation gives N = 100 views per candidate in total 1505.03046 §III-A, and also 100 views at each scale 1505.03046 §III-E. The first reading gives 25 views per scale, and the second gives 400 across the four scales.
• Deep Transfer Network with Joint Distribution Adaptation says the results are almost all over 95% 1804.07265 §IV-B, but its table lists 94.3% for F→E 1804.07265 tab. III.
• Deep Global Registration says its method is about twice as fast as RANSAC with 2M iterations 2004.11540 §5.1, while its table gives 1.21 s against 1.39 s 2004.11540 tab. 1.
• Sentiment Analysis Based on Deep Learning says the CNN outperforms other models 2006.03541 §6, but its table has the LSTM with word embeddings ahead on six of eight datasets 2006.03541 tab. 2.
• CvT gives the convolutional-embedding ablation drop as 0.8 points in its text, while table 6 gives 0.9 2103.15808 §4.4 2103.15808 tab. 6. Its text gives the key/value-stride drop as 0.3 points, while table 7 gives 0.7 2103.15808 §4.4 2103.15808 tab. 7.
• Evaluation of Conversational Agents gives 83.18% for Anger in its CNN-only table 2605.30099 tab. 1 and 93.18% after the agreement step 2605.30099 tab. 2, with no reason given for the change.
• Imaging Time-Series to Improve Classification and Imputation states a grid of image sizes 16, 24, 32, 40 and 48 and quantile counts 8, 16, 32 and 64, but Table 1 reports image sizes of 8 and 64 and quantile counts of 24, 40 and 48 1506.00327 §3.2 1506.00327 tab. 1. This page does not choose between them.
• A ConvNet for the 2020s says the ResNet-200 regime stops gaining at 7 x 7 kernels 2201.03545 §2.5, while its appendix says 5 x 5 2201.03545 app. C. Table 11 gives 82.32 for 5 x 5 and 82.30 for 7 x 7 2201.03545 tab. 11.
• Grasp Pose Detection in Point Clouds says no selection strategy gives "18% more grasp failures" 1706.09911 §8.5, while Table 3 gives 93% against 75% success 1706.09911 tab. 3. Those two statements do not give the same number.
• Sequential Short-Text Classification says the CNN beats the LSTM on every dataset 1603.03827 §4. Its grid ties them at 65.8 on DSTC 4, and its headline table gives the LSTM 66.2 against the CNN's 65.5 1603.03827 tab. 3 1603.03827 tab. 4.
• The History Began from AlexNet gives GoogLeNet's cost as 1.53G MACs in its text and 1.43G in its table 1803.01164 §III.B.6 1803.01164 tab. II.
• Towards Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection from Force Data: the abstract gives F1 89% for task classification, but §IV-C reports the same 89% as accuracy for the quadruplet model. §V-A calls it F1 2402.10220 p. 1 2402.10220 §IV-C 2402.10220 §V-A. The paper also gives the forecasting loss as RMSE in §III-G and as MSE in §IV-E 2402.10220 §III-G 2402.10220 §IV-E.
• TDN: the paper's long-term gate is an average of two sigmoid attentions, in (0, 1) 2012.10071 eq. 2 2012.10071 eq. 6. The released code shifts each sigmoid by −0.5, which gives a gate in (−0.5, 0.5), and reverses the sign of the temporal difference base_module.py base_module.py.
• Scaling Local Self-Attention gives a base learning rate of 1.6 in §4.1 and 0.1 in appendix C.3 2103.12731 §4.1 2103.12731 app. C.3.
• A Deeper Look into Sarcastic Tweets gives the Dataset 2 sentiment-feature F1 as 87.00% in §5.4, but its table gives 86.50% for that cell and 87.00% for Dataset 1 1610.08815 §5.4 1610.08815 tab. 2. Its text also calls sentiment the best pre-trained model on Dataset 3, while the table places personality (74.41%) above sentiment (73.50%) 1610.08815 §5.5 1610.08815 tab. 2. The Ptacek et al. Dataset 3 F1 is 62.37% in the text and 63.37% in the table 1610.08815 §5.5 1610.08815 tab. 3. This page uses the tables.
• Self-attention for raw optical Satellite Time Series Classification divides attention scores by the square root of the hidden size in its equation, while its code divides by the square root of the per-head key size 1910.10536 §3.4 1910.10536 eq. 4. With eight heads the two differ by a factor of about 2.8 at a hidden size of 256.
• KUISAIL at SemEval-2020 Task 12 gives 10 training epochs in the paper and 6 in its released code 2007.13184 §4.4. Reported numbers may come from either setting.
• A Review on Deep Learning in UAV Remote Sensing gives 190 Web of Science records in its text, while its per-type counts (136 articles, 46 proceedings and 10 reviews) sum to 192 2101.10861 §3. The paper's totals for analysed studies are consistent with the per-type counts.

Papers

• Places: a scene-centric image dataset and the AlexNet, GoogLeNet and VGG scene CNNs trained on it.
• FusionNet: residual encoder-decoder with summation skips for electron-microscopy segmentation.
• Advanced Steel Microstructural Classification: fully convolutional segmentation of steel micrographs with per-object majority voting.
• Transportation modes from GPS: one-dimensional CNN over GPS-derived motion channels, with a bootstrap ensemble.
• IRRCNN: inception-style recurrent residual blocks for breast histopathology.
• Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration: three-dimensional CNN that predicts a displacement field from sparse label pairs.
• DeepPurpose: drug-target interaction library with CNN and other encoders for molecules and proteins.
• SmaAt-UNet: small attention U-Net with depthwise-separable convolutions for precipitation nowcasting.
• Collaborative federated learning for healthcare: VGG16-based COVID-19 classifier trained with clustered federated averaging.
• How Do Vision Transformers Work?: analysis of self-attention and the AlterNet hybrid of CNN and local attention.
• CIFAKE: small CNN that separates real CIFAR-10 images from Stable Diffusion outputs.
• YOLO-LITE: seven-layer detector built from Tiny-YOLOv2 by trial and error, aimed at computers without a GPU.
• Implicit Bias of Gradient Descent on Linear Convolutional Networks: theory of which separator gradient descent finds in a linear convolutional network of depth L.
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks: magnitude pruning of CNN weights at four grain sizes, compared on ImageNet and in a dataflow simulation.
• A Transfer Learning Evaluation of Deep Neural Networks for Image Classification: eleven pretrained CNNs compared by accuracy density on five small image sets.
• A Taxonomy of Deep Convolutional Neural Nets for Computer Vision: survey of CNN building blocks, the AlexNet recipe and its main variants, with no experiments.
• Medical Text Classification using Convolutional Neural Networks: text CNN over word2vec vectors that labels medical sentences into 26 categories.
• SMHD: Reddit dataset of self-reported mental health diagnoses, with a text CNN baseline beside FastText and bag-of-words models.
• Greedy Layerwise Learning Can Scale to ImageNet: CNN trained one block at a time with shallow auxiliary classifiers, without end-to-end backpropagation.
• DL-Traff: survey and benchmark of urban traffic models, with a plain CNN and ST-ResNet among the grid baselines.
• Sample4Geo: cross-view geo-localisation with one shared ConvNeXt-B encoder and hard-negative sampling.
• Highway Networks: gated skip connections that let very deep plain stacks train.
• Cascaded CNN for dynamic cardiac MRI: CNN blocks interleaved with data-consistency layers for undersampled cardiac MRI.
• Brain Intelligence: Go Beyond Artificial Intelligence: position paper on brain-inspired AI, with no convolutional design content.
• Deep Learning to Improve Breast Cancer Early Detection on Screening Mammography: patch-trained CNN converted to an all-convolutional whole-image mammogram classifier.
• Road Extraction by Deep Residual U-Net: residual U-Net-style encoder-decoder for road masks in aerial images.
• Deep Learning for Classical Japanese Literature: character datasets from cursive Japanese books, with CNN and PreActResNet-18 baselines.
• CheXpert: chest radiograph dataset with uncertainty labels, and DenseNet121 models trained with four ways of handling them.
• InternImage: CNN backbone built on DCNv3, a deformable 3 x 3 operator with shared projections.
• YOLOv11: An Overview of the Key Architectural Enhancements: architecture review of a real-time detector with C3k2 and C2PSA blocks.
• Transfer learning-based method for automated ewaste recycling in smart cities: AlexNet with its output layers fine-tuned on 650 smartphone-back photos.
• Deep learning in agriculture: A survey: survey of 40 agricultural deep-learning studies and their CNN and transfer-learning choices.
• TernausNet: U-Net with an ImageNet-pretrained VGG11 encoder for building masks in aerial images.
• A Survey of Modern Deep Learning based Object Detection Models: survey of detectors and their backbones, with author-reported AP and FPS copied from source papers.
• Conformer: parallel CNN and transformer branches joined by feature coupling units in every block.
• Dive into Deep Learning: open textbook with a chapter on convolutional networks and runnable code in several frameworks.
• RepViT: pure CNN built from MobileNetV3 with lightweight-ViT layout choices, latency measured on an iPhone 12.
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation: random-view 2.5D CNN that scores detector candidates by averaging over views.
• Deep Visual Attention Prediction: VGG16 encoder with three deconvolution decoders and side-output supervision for saliency.
• Deep Transfer Network with Joint Distribution Adaptation: CNN adapted to unlabelled fault data with a joint distribution penalty.
• Blind Super-Resolution Kernel Estimation using an Internal-GAN: deep linear convolutional generator that estimates one image's blur kernel.
• Deep Global Registration: 6-D convolutional network that scores point-cloud correspondences for registration.
• Sentiment Analysis Based on Deep Learning: A Comparative Study: DNN, CNN and LSTM sentiment classifiers compared on eight datasets.
• Reading Race: AI Recognises Patient's Racial Identity In Medical Images: standard CNNs predict self-reported race from medical images, and probes find no single cause.
• Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion Detection: three-block CNNs on face and speech features, with a mismatch check for sarcasm.
• Recognition of Urbanized Areas in UAV-Derived Very-High-Resolution Visible-Light Imagery: vegetation-index thresholds compared with linear, MLP and CNN pixel classifiers.
• Recent Trends in Deep Learning Based Natural Language Processing: survey of deep NLP, with a section on CNN sentence models and copied benchmark tables.
• CvT: Introducing Convolutions to Vision Transformers: Vision Transformer with convolutional token embedding and depthwise convolutional projections.
• RFAConv: Receptive-Field Attention Convolution for Improving Convolutional Neural Networks: drop-in k x k convolution with a per-window softmax over its k² slots.
• Imaging Time-Series to Improve Classification and Imputation: GASF, GADF and Markov transition images of a time series, classified by a tiled CNN and used for imputation.
• Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network: residual encoder-decoder that regresses a 256 x 256 face position map in one pass.
• Grasp Pose Detection in Point Clouds: samples two-finger grasp candidates from point clouds and scores 15-channel grasp images with a small CNN.
• A ConvNet for the 2020s: ConvNeXt, a pure convolutional backbone built by modernising a ResNet-50.
• Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks: LSTM or CNN text encoder with a window over earlier texts, for dialog-act labels.
• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches: survey of deep learning with a section on CNN building blocks and designs from LeNet to DenseNet.
• TDN: Temporal Difference Networks for Efficient Action Recognition: 2D ResNet with short- and long-term temporal difference modules for video.
• Deep Parametric Continuous Convolutional Neural Networks: continuous convolution whose kernel is an MLP over relative 3D positions, for scattered points.
• Scaling Local Self-Attention for Parameter Efficient Visual Backbones: HaloNet, an image backbone built from blocked local self-attention with haloing.
• Optimizing Neural Network for Computer Vision task in Edge Device: small gesture CNN at four input sizes, converted with TensorFlow Lite for a Raspberry Pi.
• Towards Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection from Force Data: CNN classifiers and a CNN+LSTM forecaster on glovebox teleoperation force and joint data.
• UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space: three-block CNN with RGB, HSV and attention branches for underwater images.
• Machine learning based disease diagnosis: A comprehensive review: systematic review of ML and DL for diagnosis, 2012 to 2021, which reports CNNs as the most used model.
• A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks: three CNNs pre-trained on sentiment, emotion and personality, merged with a sarcasm CNN for an SVM.
• Self-attention for raw optical Satellite Time Series Classification: Transformer compared with an LSTM, two 1-D CNNs and a random forest on raw and preprocessed Sentinel-2 series.
• KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media: language-specific BERT with a CNN over its last four layers, for offensive tweets.
• A Review on Deep Learning in UAV Remote Sensing: literature review of deep learning on UAV imagery, coding 232 studies by network, task, sensor and context.
• A Survey of Deep Learning Techniques for Weed Detection from Images: survey of 70 deep-learning weed-detection papers, with their architectures, datasets and metrics.
• Encrypted ResNet-20 inference with RNS-CKKS: ResNet-20 evaluated on encrypted CIFAR-10 images with RNS-CKKS bootstrapping and a polynomial ReLU.
• High precision grasp pose detection in dense clutter: conference version of the grasp classifier, with occlusion channels and CAD pretraining.
• A review of machine learning in processing remote sensing data for mineral exploration: review of machine learning for mineral mapping, with convolutional networks in its deep-learning group.
• Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks: survey of input, internal and label regularisers for CNNs, with a compiled results table.