Paper · 2018-07-26 · cs.CV, eess.IV
Effects of Degradations on Deep Neural Network Architectures
Measures how six standard classifiers, CapsuleNet among them, lose accuracy under six noise, blur and JPEG degradations of growing strength. Proposes V-CapsNet and the NTT block (a fixed low-pass plus a trainable depthwise filter). Author-reported: CapsuleNet is most robust on most degradations; NTT adds robustness for a small clean-accuracy cost.
Capsule networkConvolutional neural networkAdversarial robustnesscomputer-visionimage-classificationrobustnessbenchmarkcapsule-network
arXivPDFCodeMarkdown13 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Image classifiers are usually trained on clean, sharp photos, but cameras in the real world add grain, blur, dropped pixels and compression artifacts. This paper asks how much six well-known networks lose when their input is damaged in one of six such ways, with the damage turned up step by step. Five of the networks are standard convolutional designs; the sixth, CapsuleNet, groups neurons into small vectors and passes information between them with an iterative routing rule. CapsuleNet is the least accurate on clean images but holds up best against most of the damage.
The authors then make two changes. One is a hybrid that puts the early layers of a VGG network in front of CapsuleNet. It is the most accurate on clean images but falls off faster under damage, which suggests that depth makes a network more fragile. The other is a small block for any convolutional network: a fixed smoothing filter followed by a learnable filter. It makes networks more tolerant of noise, at the price of a little clean accuracy. The fixed filter is only described, not released in code.
TL;DR
The paper trains six classifiers on clean images from two datasets it builds (synthetic digits and natural images), then measures validation accuracy while six degradations (Gaussian white and colour noise, salt-and-pepper, Gaussian blur, motion blur, JPEG) grow in strength §III §IV.A. Author-reported: CapsuleNet is the most robust against Gaussian noise, salt-and-pepper and blur, but it is less accurate than the deep CNNs on noise-free images §IV.A §IV.B. The NTT block improves robustness for every network tested, and trades about 6% of clean accuracy for VGG-19 §V.A. Use this card if you need the degradation protocol or the NTT idea. The official repository contains only the baseline training and the degradation sweep; NTT, V-CapsNet and the adversarial experiment have no code there.
Problem
Models are trained and validated on high-quality images, yet inputs in deployment are often degraded by sensor noise, poor light, defocus, motion, wrong exposure or lossy compression §I. Adding noisy training samples helps, but it is hard to cover every noise model, and nonlinear blur kernels in particular are hard to simulate §I §V. Earlier studies of degradations and CNNs were narrower: they omitted some noise models (motion blur, salt-and-pepper) and architectures (ResNet, CapsuleNet), and proposed no defence §II.
The study assumes that a network is trained once on the noise-free training split and then evaluated, without retraining, on degraded copies of the validation split §III. Robustness is therefore measured as accuracy as a function of degradation strength, not as accuracy under a fixed noise budget.
Method
Study protocol
For each architecture and each degradation model, the paper trains on the clean training split, then degrades the validation split at a series of increasing strengths and measures top-1 and top-3 accuracy at each one §III §IV.
# Degradation sweep [§III] [§IV]
# Input: trained models; clean validation images X with labels y
# Output: top-1 and top-3 accuracy for each model at each strength
def degradation_sweep(models, X, y, degradation, strengths):
curves = {}
# Visit the strengths in order, weakest first [§III]
for strength in strengths:
# Degrade every validation image at this strength [§III.C]
X_bad = [degrade(x, degradation, strength) for x in X]
for model in models:
# Scores for each class, one row per image [code: test.py#L264-L269]
scores = model.predict(resize(X_bad, model.input_size))
# Record top-1 and top-3 accuracy [§IV] [code: test.py#L270-L271]
curves[model][strength] = accuracy_top_k(scores, y, k=3)
return curvesDatasets §III.A
- Synthetic digits. Digits 0 to 9 rendered in 16 fonts, each with a random colour, rotation between −30° and +30°, and size between 30 and 240 pixels, placed over a random COCO image. 12,000 images: 10,000 for training and 2,000 for validation, balanced over the ten digits §III.A fig. 1.
- Natural images. 6,899 photos in eight classes: airplane 727, car 968, cat 885, dog 702, flower 843, fruit 1000, motorbike 788, person 986. Split 5,724 training and 1,175 validation §III.A fig. 1. The paper calls the images high-resolution but gives no pixel size. How the split was drawn is not stated.
Architectures §III.B
The six networks are two VGG variants, a residual network, an Inception design, a mobile-oriented network and CapsuleNet. The paper reports their depth and size as follows.
| Architecture | Depth (paper) | Parameters (paper) | Source |
|---|---|---|---|
| VGG-16 | 16 layers | 138.4 M | §III.B |
| VGG-19 | 19 layers | 143.7 M | §III.B |
| ResNet-50 | 50 layers | 25.6 M | §III.B |
| Inception-v3 | 48 layers | 27.2 M | §III.B |
| MobileNet | 28 layers | 4.2 M | §III.B |
| CapsuleNet | 6 layers | 8.2 M | §III.B |
For the four CNNs the full set of convolution layers is kept. The default fully-connected top is replaced by three fully-connected layers with 1024, 1024 and n outputs, where n is the number of classes (10 for synthetic digits, 8 for natural images). A dropout of 0.5 sits between the first two of them §III.B train_deepcnn.py.
For CapsuleNet, two convolution layers are added in front of the original network, each with 9×9 kernels and ReLU. The first has 256 filters and stride 2; the second has 128 filters and stride 1 §III.B train_capsnet.py.
Dynamic routing §IV.A CapsuleNetwork.py
The paper compares CapsuleNet with one routing iteration and a margin loss alone, against three iterations with a reconstruction loss added as a regulariser. The second setting is slightly better §IV.A fig. 5. The routing step itself is described below as the released code implements it.
# Dynamic routing between capsules [code: libs/CapsuleNetwork.py#L82-L96]
# Input: lower capsules u (B, N_in, d_in); weights W (N_out, N_in, d_out, d_in)
# Output: class capsules v (B, N_out, d_out)
def dynamic_routing(u, W, routings):
# Each lower capsule votes for each higher one [code: libs/CapsuleNetwork.py#L83-L86]
u_hat = predictions(u, W) # (B, N_out, N_in, d_out)
# Routing logits start at zero [code: libs/CapsuleNetwork.py#L87-L89]
b = zeros(B, N_out, N_in)
for i in range(routings):
# Softmax over output capsules [code: libs/CapsuleNetwork.py#L92]
c = softmax(b, axis=1)
# Weighted sum of votes, then squash [code: libs/CapsuleNetwork.py#L93]
v = squash(sum_over_inputs(c * u_hat))
# Agreement update, not on last pass [code: libs/CapsuleNetwork.py#L94-L95]
if i < routings - 1:
b = b + dot(v, u_hat)
return vDegradation models §III.C
Each model below is applied to the validation images at increasing strength. Strength grids are those stated in the paper; the released code uses different grids (see Paper vs code).
- Gaussian white noise. Zero-mean noise with standard deviation σ, the same value in every colour channel. σ from 0 to 1 in steps of 0.1 §III.C.
- Gaussian colour noise. As above, but drawn independently for each channel. Same σ grid §III.C.
- Salt-and-pepper. A fraction d of pixels (noise density) is replaced by white or black. d from 0 to 1 in steps of 0.1 §III.C.
- Gaussian blur. Kernel size k from 3×3 to 51×51 in steps of 2. The standard deviation is σ_g = 0.3·((k−1)·0.5−1)+0.8 §III.C.
- Motion blur. A horizontal line kernel of size k from 1×1 to 31×31 in steps of 2 §III.C.
- JPEG compression. Quality q from 30 down to 0 in steps of 2. Lower q means stronger compression §III.C.
NTT block §V.A fig. 9
The NTT (nontrainable–trainable) block is meant to make a network invariant to small smoothing and denoising changes. It has two depthwise stages applied in sequence. The first holds the weights of a low-pass filter and is not trained. The second is a trainable filter. Each stage is a per-channel filter applied at stride 1. Adding the block to a network trades clean accuracy for robustness, and the trade-off is set by network depth and filter size §V.A.
# NTT block [§V.A] [fig. 9]
# Input: feature maps x (B, C, H, W), one filter per channel
# Output: feature maps (B, C, H', W'), H' = H and W' = W at unit stride
def ntt_block(x, lowpass_weights, trainable_weights):
# Fixed low-pass depthwise filter; its weights are never updated [§V.A]
smoothed = depthwise_conv(x, lowpass_weights) # lowpass: size and values not stated
# Trainable depthwise filter, applied after the fixed one [§V.A]
return depthwise_conv(smoothed, trainable_weights)The text does not say which layer the block is inserted before, after or in place of, so the placement is not stated §V.A fig. 10.
V-CapsNet §IV.B fig. 6
V-CapsNet swaps CapsuleNet's opening convolutions for VGG-19's front end, cut off at the start of stage five §IV.B. Its PrimaryCaps convolution produces 256 channels with a 3×3 window and step 2, regrouped into 32 capsule channels of 8-dimensional vectors on a 7×7 grid §IV.B. Class capsules are 16-dimensional, one per class, as in CapsuleNet. The decoder is dropped and only the margin loss is minimised §IV.B. V-CapsNet is the most accurate network in the paper's experiments but decays faster under degradation than CapsuleNet and VGG-19 §IV.B fig. 7. The authors attribute its weaker robustness to its greater depth, as a hypothesis they support with SSIM measurements of feature maps §IV.B fig. 8.
Adversarial perturbations §V.B fig. 12
The paper runs an untargeted fast gradient sign method (FGSM) attack and reports the perturbation size ε needed to force misclassification §V.B. Values of ε are not given in the text; results appear only in fig. 12.
Hyperparameters
Entries marked "paper" come from the text. Entries marked "code" come from the official repository and are not stated in the paper.
| Name | Value | Where | Notes |
|---|---|---|---|
| Noise-free training split | Train on clean images only | §III | No degraded images during training |
| Top-k reported | k = 3 | §IV test.py | Top-1 is reported as well |
| CNN top head | FC 1024 → dropout 0.5 → FC 1024 → FC n (softmax) | §III.B train_deepcnn.py | Paper and code agree |
| CNN initialisation | ImageNet weights, all layers trainable | train_deepcnn.py train_deepcnn.py | Paper silent |
| CNN optimiser | SGD, learning rate 0.0001, momentum 0.5 | train_deepcnn.py | Paper silent |
| CNN batch size | 50 | train_deepcnn.py | Paper silent |
| CNN input size | 299×299×3 | train_deepcnn.py | InceptionV3 default; paper silent |
| CNN augmentation | None; only rescale by 1/255 | train_deepcnn.py | Paper silent |
| CNN epochs and stopping | Up to 100 epochs; stop when validation loss has not improved for 10 | train_deepcnn.py train_deepcnn.py | Paper silent |
| CapsNet input size | 104×104×3 | train_capsnet.py | Paper silent |
| CapsNet convolutions | 9×9, 256 filters, stride 2, ReLU; then 9×9, 128 filters, stride 1, ReLU | §III.B train_capsnet.py | Paper and code agree |
| PrimaryCaps | 9×9 kernel, stride 2, 32 capsule channels of 8 dimensions | train_capsnet.py | Paper silent on the baseline |
| DigitCaps | 10 classes × 16 dimensions, 3 routing iterations | train_capsnet.py train_capsnet.py | Config (b) in §IV.A |
| Margin loss | m⁺ = 0.9, m⁻ = 0.1, λ = 0.5 | train_capsnet.py | Paper: "marginal loss" only |
| Reconstruction decoder | 512 → 1024 → sigmoid over 104·104·3, loss weight 0.0005 with MSE | train_capsnet.py | Paper gives no weight |
| CapsNet optimiser | Adam, learning rate 0.001 decayed by 0.9 per epoch | train_capsnet.py train_capsnet.py | Paper silent |
| CapsNet steps per epoch | 200 training, 40 validation batches of 50 | train_capsnet.py train_capsnet.py | Paper silent |
| Gaussian blur σ | σ_g = 0.3·((k−1)·0.5−1)+0.8 | §III.C | Code lets OpenCV derive σ; see Pitfalls |
| FGSM ε | Not stated | §V.B | Only in fig. 12 |
| NTT low-pass filter | Size and weights not stated | §V.A | Trainable filter size not stated either |
| NTT placement | Not stated | §V.A |
Results
Author-reported. The paper gives almost no numbers in its text. Full accuracy curves are in the figures. The setup is the synthetic digits and natural images datasets above, with the six architectures; the baseline for NTT is the same CNN without the block. The acknowledgements name a TITAN X GPU § Acknowledgements.
| Setting | Metric | Baseline | This paper | Source |
|---|---|---|---|---|
| V-CapsNet, natural images, clean validation | Accuracy | CapsuleNet, improved by 6.2% (unit not stated) | 99.83% | §IV.B |
| VGG-19 with NTT, Gaussian colour and salt-and-pepper noise | Accuracy | VGG-19 without NTT, about 6% lower clean accuracy with NTT | Up to 68% improvement (unit not stated) | §V.A |
| CapsuleNet vs VGG on Gaussian noise | Accuracy decay with σ | VGG (lowest decay among conventional CNNs) | CapsuleNet most robust | §IV.A fig. 3 fig. 4 |
| Salt-and-pepper | Accuracy vs density d | Other CNNs | CapsuleNet ahead for d ≥ 0.2 and up to d ≤ 0.8 | §IV.A fig. 3 |
| JPEG | Accuracy vs q | ResNet and MobileNet | ResNet and MobileNet drop sharply for q ≤ 20; no significant loss for q > 20 for all | §IV.A fig. 3 fig. 4 |
| Routing and reconstruction in CapsuleNet | Accuracy | One iteration, margin loss only | Slight gain with 3 iterations and reconstruction | §IV.A fig. 5 |
| FGSM adversarial attack | ε needed to fool the model | CapsNet and VGG perform poorly | ResNet-50, Inception-v3, MobileNet and V-CapsNet perform well | §V.B fig. 12 |
The figures show curves only. Values behind them are not stated in the text.
Paper vs code
Official code: the repository linked from the author block, at commit
4d9fb70c4548b2c51b25a0bb69eec82268dcb457. It is MIT-licensed and its owner is
the first author. Its scope is the training scripts for the CNN and CapsuleNet
baselines and the degradation sweep (train_deepcnn.py, train_capsnet.py,
test.py, libs/). A text search of the repository for NTT, FGSM, V-CapsNet, depthwise
or low-pass filters finds nothing README.md.
Differences between paper and code:
- Gaussian noise grid. The paper varies σ from 0 to 1 in steps of 0.1 §III.C.
The sweep in the code is
range(0, 256, 5), so σ runs from 0 to 255 in steps of 5 test.py. The noise is added to pixels scaled to [0, 1] DegradationModels.py DegradationModels.py. Any σ of 1 or more therefore pushes most pixels to 0 or 1 after clipping. Which grid produced the published figures cannot be determined from the repository. - Salt-and-pepper grid. Paper: steps of 0.1 §III.C. Code: steps of 0.05, from 0 to 1 test.py.
- Gaussian blur grid. Paper: from 3×3 to 51×51 §III.C. Code starts at 1×1, which is a no-op test.py.
- Motion blur grid. Paper: from 1×1 to 31×31 §III.C. Code starts at 3×3 test.py.
- Blur σ. The paper gives a formula for σ_g §III.C. The code passes σ = 0 to OpenCV, which computes σ from the kernel size itself DegradationModels.py. That OpenCV default appears to be the same expression as the paper's; it was not verified by running OpenCV.
- Rounding. Noisy images are converted to 8-bit by truncation (
astype), not rounding DegradationModels.py DegradationModels.py. The paper does not say how conversion is done. - Order of degradation and resizing. The code degrades each image at its dataset resolution, then resizes it to the network's input size test.py. The paper does not state the order.
- ImageNet initialisation. The CNN training script loads ImageNet weights train_deepcnn.py. The paper does not mention pretraining.
- Natural images. Both training scripts and the test script are configured
for
synthetic_digitsonly train_deepcnn.py train_capsnet.py test.py. The repository has no natural-images configuration. The dataset is said to be on Kaggle README.md. The 99.83% for V-CapsNet cannot be re-run from this repository as it stands. - Routing configuration (a). The paper's single-iteration, margin-only setting is not a switch in the code. The defaults are three iterations with the reconstruction loss on train_capsnet.py train_capsnet.py.
- Missing methods. V-CapsNet, NTT and the FGSM experiment have no code in the repository README.md test.py.
- Matches. The CNN head (1024, dropout 0.5, 1024, n) and the two extra CapsNet convolutions match the text §III.B train_deepcnn.py train_capsnet.py. The JPEG grid (30 down to 0 in steps of 2) matches §III.C test.py.
Pitfalls
- One validation split does three jobs. The best checkpoint is chosen on the validation split train_capsnet.py, early stopping uses it train_capsnet.py, and every robustness curve is measured on the same split test.py. There is no held-out test set, so the reported numbers are optimistic.
- Fixed seeds repeat the noise.
imdegradere-seeds NumPy and Python'srandomon every call DegradationModels.py. With a fixed seed, every image therefore gets the same random draws. WithRANDOM_SEED = None, which is the shipped value, each call draws from fresh entropy and runs are not repeatable test.py. - Large σ saturates. Noise is added on the [0, 1] scale, so for σ ≥ 1 nearly all pixels are clipped to 0 or 1. Curves that look like a gentle decline may really be a change of regime. Check the grid before comparing with the figures.
- Strengths are not comparable across families. Noise σ, blur kernel size, density d and JPEG quality q are different scales. Accuracy versus strength therefore compares shapes of curves, not "robustness" per unit.
- Learning-rate decay is steep. Under the code's schedule the CapsNet learning rate is multiplied by 0.9 each epoch train_capsnet.py. After 50 epochs it is about 0.5% of the starting value (0.9^50 ≈ 0.005), so late epochs barely change the weights.
- Fixed batch counts. CapsNet training draws exactly 200 batches per epoch and validates on 40 train_capsnet.py train_capsnet.py, not one pass over the data. The paper does not mention this.
- Hard-coded CNN defaults.
train_deepcnn.pyis set to InceptionV3 with 10 classes and a learning rate of 0.0001 train_deepcnn.py. Those are the defaults of the committed file; the repository does not say which settings produced each figure. - No data augmentation for CNNs. Only rescaling is applied train_deepcnn.py. Results may differ from an implementation with augmentation, which the paper does not describe.
- Unclear headline units. "6.2%" and "up to 68%" do not say whether they are percentage points or relative gains §IV.B §V.A.
- The NTT block is underspecified. Low-pass weights, filter sizes and placement are not stated in text §V.A. Re-implementing it from the text alone is guesswork.
- Adversarial results are qualitative. No ε values, attack norm or number of iterations are given in the text §V.B. Treat the claim that the NTT block makes attacks easier to detect as a figure-only observation fig. 12.
- Version. The source fetched is the arXiv version recorded in
raw/arxiv/1807.10108.json(v6; metadata updated 2025-02-18). Earlier versions were not compared.
Lineage
- Builds on: CapsuleNet with dynamic routing, cited as Sabour et al. 2017 §III.B. The routing code is adapted from Xifeng Guo's Keras CapsNet implementation, per the file headers CapsuleNetwork.py.
- Compared architectures: VGG-16 and VGG-19, ResNet-50, Inception-v3 and MobileNet, each taken from its original paper §I §III.B.
- Compared against: Dodge and Karam 2016 on image quality and deep networks §II, and Pei et al. 2018, arXiv:1810.05552 §II. The paper says both studied fewer noise models and networks, and proposed no defence §II.
- Related: the paper's adversarial experiment uses FGSM, cited to Kurakin et al. 2016 §V.B.
- Superseded by: not known from content on this wiki.
Source and license
Adapted from “Effects of Degradations on Deep Neural Network Architectures” by Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, Umapada Pal (arXiv:1807.10108), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/prasunroy/cnn-on-degraded-images at commit 4d9fb70c4548 (MIT). Any quoted code is shown under that license.