wikillm

Paper · 2021-09-09 · cs.CV, cs.AI

Towards Transferable Adversarial Attacks on Vision Transformers

Dual attack for transferable untargeted L-inf adversarial examples on ViTs. Pay No Attention (PNA) drops gradients through attention weights; PatchOut updates a random 130 of 196 patches per iteration. Author-reported: 58.67% mean attack success over eight ViTs, against 52.28% for SGM, the best averaged baseline.

Adversarial robustnessVision Transformer (ViT) backbonesSelf-attentionadversarial-attacktransferabilityvision-transformerattention-gradientpatch-dropout

arXivPDFCodeMarkdown14 min read

AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

An adversarial example is an image changed by a little noise that a classifier gets wrong. Attackers often build the noise on a model they can inspect and hope it also fools other models. That hope is called transferability. Attacks tuned for convolutional networks transfer poorly to vision transformers, which look at an image as a set of square patches that attend to each other.

The paper attacks the two parts that make transformers different. First, it stops the attack's learning signal from flowing back through the attention weights, so the noise does not fit the particular attention pattern of the model being attacked. Second, at each step it changes only a random subset of the patches, similar to dropout, so the noise does not over-fit one model. The two parts are combined in one attack. The authors report that it transfers better than earlier methods across transformers, and also to some convolutional networks.

TL;DR

  • Untargeted attack with an L-inf budget, crafted on white-box ViTs and tested on black-box ViTs, CNNs and robustly trained CNNs §4.1.
  • Pay No Attention (PNA) treats the attention weights as constants in backpropagation §3.2. PatchOut uses a random subset of T = 130 of 196 patches per iteration §3.3 §4.2.
  • Adds an L2 penalty on the perturbation with weight 0.1 §4.2 eq. 7.
  • Author-reported: 58.67% average MASR over eight ViTs tab. 1 §4.2; 42.61% on normal CNNs and 22.38% on robust CNNs tab. 2 §4.2.
  • The official repository has no LICENSE file. Nothing is copied from it. Several details differ from the paper; see Paper vs code.

Problem

Transfer attacks craft examples on a surrogate model and use them against an unseen victim §1. The paper reports that attacks designed for CNNs degrade significantly when applied to ViTs, and that TI and ATA perform poorly there §2.3 §4.2. It argues the architecture needs its own treatment: self-attention and patch tokens §1. The earlier ViT-specific Self-Ensemble attack depends on class tokens to build an ensemble, and its Token Refinement step fine-tunes on ImageNet training data §2.3. The authors contrast their method with it, calling theirs generalizable across ViTs and easy to implement §2.3.

Method

The method has two parts, applied together and in the same iterative loop.

  1. Pay No Attention (PNA). In each attention head, the attention matrix A is a softmax over scaled query-key products §3.2 eq. 2. The output is A times the values eq. 3. PNA sets the derivative of A with respect to the input to zero, so the gradient flows only through the value path. This is a stop-gradient on the attention weights §3.2 eq. 5. The authors say it limits interactions between patches in the perturbation, which they relate to transferability §1 §3.2.
  2. PatchOut. At iteration i, a random subset of T patches is chosen. The perturbation is applied only to those patches, through a 0/1 mask §3.3 eq. 6. The loss is the cross-entropy of the masked input plus lambda times the L2 norm of the full perturbation eq. 7.
  3. Iteration. The attack runs I = 10 steps with step size alpha = eps / I §4.1 alg. 1. The perturbation is clipped to [-eps, eps] after each step alg. 1.
  4. Combination. PNA and PatchOut are independent switches in the ablation §4.3 tab. 4. The method can also be run with momentum (MI) or with the skip-gradient decay (SGM) §4.2 tab. 3.
Pseudocode
# Dual attack: PNA gradients and PatchOut masks  [alg. 1]
# Input:  white-box ViT f, image x in [0, 1], label y
# Output: adversarial image x_adv in [0, 1]
def dual_attack(f, x, y, eps=16/255, steps=10, T=130, lamb=0.1):
    # Step size alpha = eps / steps  [§4.1] [code: our_method.py#L92]
    alpha = eps / steps
    # Start with zero perturbation  [alg. 1]
    delta = zeros_like(x)
    for i in range(steps):
        # Mask for this iteration: T random patches  [§3.3] [eq. 6]
        mask = patchout_mask(i, T)  # (1, 3, 224, 224), values 0 or 1
        # Classify the masked perturbation  [eq. 7]
        # [code: our_method.py#L172-L178]
        logits = f(normalise(x + mask * delta))
        # Cross-entropy, plus lambda times the L2 norm of all of delta
        # [eq. 7] [code: our_method.py#L180-L187]
        loss = cross_entropy(logits, y) + lamb * l2_norm(delta)
        # Gradient with respect to delta; PNA removes the attention path
        # [§3.2] [code: our_method.py#L109-L117]
        grad = gradient(loss, delta)
        # Sign step, then clip to the eps-ball  [alg. 1]
        # [code: our_method.py#L190] [code: BaseAttack in our_method.py#L74-L75]
        delta = clip(delta + alpha * sign(grad), -eps, eps)
        # Keep the image in its valid range  [code: our_method.py#L191]
        delta = clip(x + delta, 0, 1) - x
    return x + delta
Pseudocode
# Pay No Attention: attention weights are constants in backprop  [§3.2]
def pna_attention(tokens):
    # Queries, keys and values are linear maps of the tokens  [eq. 2] [eq. 3]
    queries = tokens @ W_Q
    keys = tokens @ W_K
    values = tokens @ W_V
    # Softmax attention weights, scaled by sqrt(D_h)  [eq. 2]
    weights = softmax(queries @ keys.T / sqrt(D_h))
    # Stop gradient through the weights, so dA/dZ = 0  [§3.2] [eq. 5]
    weights = stop_gradient(weights)
    # The output still depends on values, so gradients flow there  [eq. 3]
    return weights @ values
Pseudocode
# PatchOut mask for iteration i: T of the N = 196 patches  [§3.3] [§4.1]
def patchout_mask(i, T):
    # The shuffle is seeded with the iteration index, so every image and
    # every run uses the same subset  [code: our_method.py#L148-L151]
    patch_ids = shuffle(range(196), seed=i)
    # Keep the first T ids: sampling without replacement
    # [code: our_method.py#L151]
    chosen = patch_ids[:T]
    # Mask is 1 on the chosen 16 x 16 patches, 0 elsewhere  [eq. 6]
    # [code: our_method.py#L155-L158]
    return mask_from_patches(chosen)

Key equations, as in the paper. Equation numbers 4 and 5 are unlabelled in the source; their numbers follow from counting the equation environments in order §3.2.

\arg\max_{\delta} J(f(x+\delta), y), \quad \lVert\delta\rVert_\infty < \epsilon eq. 1

A = \operatorname{softmax}\!\left(Z W^{Q} (Z W^{K})^{T} / \sqrt{D_h}\right) eq. 2

Z' = A\,(Z W^{V}) eq. 3

\frac{\partial Z'}{\partial Z} = (\mathbb{I} \otimes A)\,\frac{\partial (Z W^{V})}{\partial Z} + ((Z W^{V})^{T} \otimes \mathbb{I})\,\frac{\partial A}{\partial Z} eq. 4

\frac{\partial Z'}{\partial Z} \approx (\mathbb{I} \otimes A)\,((W^{V})^{T} \otimes \mathbb{I}) eq. 5

M_p^i = \begin{cases} 1 & \text{if } x_p^i \in x_s \\ 0 & \text{otherwise} \end{cases} eq. 6

\arg\max_{\delta} J(f(x + M \odot \delta), y) + \lambda \lVert\delta\rVert_2, \quad \lVert\delta\rVert_\infty < \epsilon eq. 7

The paper's Algorithm 1 is written out in the pseudocode above alg. 1.

Hyperparameters

Values are from the paper unless the Where column names the code. "Not stated" means the paper and the repository do not give it.

NameValueWhereNotes
Norm bound epsilon (L-inf)16§4.1 eq. 1Unit not stated in the paper. Code uses 16/255 on [0, 1] images our_method.py
Iterations I10§4.1 §4.3Code default steps=10 our_method.py
Step size alpha1.6 (= eps / I)§4.1Code uses eps / steps our_method.py. Alg. 1 writes no sign; code uses sign(grad)
Patches per iteration T130 (of 196)§4.2 fig. 4Code default 130 our_method.py. Optimum in the T sweep §4.3
Patch size P16§4.1Code crop_length=16 our_method.py
Image size224 x 224§4.1Code our_method.py. The paper states N = 196 patches §4.1
L2 weight lambda0.1§4.2 fig. 4Code lamb=0.1 our_method.py. Unit of delta for the norm not stated
Loss Jcross-entropy§4.1 eq. 1Code nn.CrossEntropyLoss, mean reduction our_method.py
Pixel rangenot statedn/aCode clamps to [0, 1] after every step our_method.py
PNA gradient gate0 on attention path§3.2 eq. 5Code gamma=0 on each block's attn_drop our_method.py
SGM decay, for SGM + Ours0.5 on MLP gradients§4.2Code gamma=0.5 our_method.py
MI momentum decaynot stated§4.2 ("original settings")Code default decay=1.0 our_method.py
Input normalisationnot statedn/aCode: ImageNet mean/std for six models; 0.5/0.5 for vit_base_patch16_224 and tnt_s_patch16_224 dataset.py
Surrogates (white-box)ViT-B/16, PiT-B, CaiT-S-24, Visformer-S§4.1Attack names in the README ReadMe.md
Victims8 ViTs, 4 CNNs, 3 robust CNNs§4.1CNN code not in the repository (see Paper vs code)
Evaluation imagesone per class from ImageNet 2012 val, correctly classified by all models§4.1Ablation uses 2,000 images §4.3 fig. 1
Batch sizenot statedn/aREADME example uses 1 ReadMe.md
Ablation T sweep10, 40, 70, 100, 130, 160, 196§4.3 fig. 4ViT-B/16 white-box only
Ablation lambda sweep0.001, 0.01, 0.1, 1, 10§4.3 fig. 4T fixed at 130

Results

Author-reported. Setup: ImageNet 2012 validation set, one image per class §4.1. The four white-box surrogates are ViT-B/16, Visformer-S, CaiT-S-24 and PiT-B §4.1. Baselines run with their original settings, integrated into BIM §4.2. TR is not compared because it fine-tunes on ImageNet §4.2. Hardware is not stated.

Headline numbers:

SettingMetricBest baseline (average)This paperSource
8 ViT victimsMASR (%)SGM 52.2858.67tab. 1 §4.2
4 normal CNNsASR (%)MI 35.6342.61tab. 2 §4.2
3 robust CNNsASR (%)MI 21.5322.38tab. 2 §4.2

Table 1: MASR (%) against ViT victims tab. 1. Each entry is averaged over the white-box surrogates. The first four columns are the surrogates themselves.

MethodViT-B/16PiT-BCaiT-S-24Visformer-SDeiT-BTNT-SLeViT-256ConViT-B
FGSM15.5719.8020.4319.3722.0822.7818.8025.58
BIM20.7722.1722.6322.7033.5332.1320.4535.30
MI41.2345.2347.1345.9756.0355.2343.7558.25
DI32.5745.1343.0747.7748.0855.1843.2549.35
TI19.3317.6716.5019.0025.1328.1813.7027.53
SIM34.9732.7335.1731.1344.1346.7336.4345.68
SGM38.8741.6052.3048.8060.5364.3351.1360.68
IR21.3322.7024.0023.4334.0033.4321.3036.38
TAP25.2724.7333.4032.2043.2039.7830.0342.20
ATA3.471.130.972.673.683.372.023.72
SE29.0521.2531.4024.9045.2337.8721.7346.03
Ours (PNA + PatchOut + L2)46.1052.4059.8758.6063.8567.2557.6263.70

Table 2: MASR (%) against CNN victims tab. 2. The first four columns are normally trained CNNs. The last three are robustly trained CNNs; their labels follow the source macros for the ens3, ens4 and ens variants.

AttackInception v3Inception v4Inception-ResNet v2ResNet v2-152Inception v3 (ens3)Inception v3 (ens4)Inception-ResNet v2 (ens)
FGSM20.7818.8016.3819.0514.6213.9810.38
BIM17.8814.7712.4011.828.026.204.45
MI39.6537.4332.1733.2824.8022.1017.68
DI32.7831.7526.4025.0017.7315.2211.12
TI23.2723.6015.2820.8818.8020.8013.92
SIM30.5527.6324.1724.4819.4517.6713.40
SGM38.4234.0027.2527.2518.7016.5311.68
IR17.6515.8312.0812.488.056.504.78
TAP29.5826.1020.6719.2312.5810.626.85
ATA3.252.532.032.071.200.850.92
SE18.4016.4712.3312.639.136.735.10
Ours47.9545.1238.4538.9326.2022.8518.10

Table 3: average ASR (%) when the dual attack is combined with MI and SGM, white-box ViT-B/16 tab. 3 §4.2.

MethodViTsNormal CNNsRobust CNNs
MI49.1035.6321.53
MI + Ours59.7048.7930.13
SGM52.2831.7315.64
SGM + Ours63.4852.3131.37

Table 4: ablation, average ASR (%), white-box ViT-B/16, 2,000 images tab. 4 §4.3. The set of victims averaged for the ViTs column is not stated.

PatchOutL2PNAViTsNormalRobust
---26.9710.635.50
yes--34.9212.557.07
-yes-37.7916.5510.00
--yes42.4718.7510.97
yesyes-48.8224.5514.67
-yesyes49.6922.3613.54
yesyesyes59.1536.2323.87

Figure-based results, described in words: the toy experiment shows that bypassing all attention gradients raises BIM's ASR on ViT-B/16 from 29.92% to 42.47% §1 fig. 1. The 42.47% equals the PNA-only ViT entry in tab. 4. Stacking several ten-patch perturbations reaches a higher ASR than perturbations optimised on the whole image fig. 2 §1. The T sweep peaks at T = 130 and the lambda sweep peaks at lambda = 0.1 §4.3 fig. 4. Adversarial examples are described as hardly perceptible §4.2 fig. 3.

Paper vs code

Repository: zhipeng-wei/PNA-PatchOut at 85f23c7. The owner is the first author, and the paper's abstract links it. There is no LICENSE file, so no code is quoted in this card. The CNN attack code and CNN model loading are absent (see below).

  • Update rule. Algorithm 1 applies the gradient without a sign alg. 1. The code applies grad.sign() (a BIM-style step) our_method.py our_method.py.
  • Pixel range. Algorithm 1 clips only to the eps-ball alg. 1. The code also clamps the image to [0, 1] after each step our_method.py.
  • PatchOut sampling. The paper says the patches are "randomly sampled" at each iteration §3.3. The code seeds the shuffle with the iteration index, so the same 130 patches are used for every image and every run [code: our_method.py#L148-L151]. A repeatable, with-replacement alternative is commented out our_method.py.
  • PNA. The paper sets dA/dZ = 0 §3.2 eq. 5. The code zeroes the gradient entering each block's attn_drop our_method.py. The two match if attn_drop sits after the softmax, which is a timm internal and is not in this repository. This was not checked.
  • L2 term. Both paper and code use the full delta in the penalty eq. 7 our_method.py. The code's norm runs over the whole batch tensor.
  • Hooks for other models. Hooks are registered only for ViT-B/16, DeiT-B, the PiT, CaiT and Visformer branches our_method.py. For TNT, LeViT and ConViT no hook is added, so PNA is silently off our_method.py.
  • Preprocessing. The paper says "resize all images to 224 x 224" §4.1. The code reads pre-resized PNGs from clean_resized_images.zip; the resize step is not in the repository dataset.py.
  • Normalisation. Not in the paper. The code uses two schemes by model name dataset.py.
  • Victims. The paper evaluates 8 ViTs, 4 CNNs and 3 robust CNNs §4.1. The repository's get_model handles only the 8 ViT names model.py model.py. For any other name it returns an unbound variable. The driver our_attacks.py imports cnns_method our_attacks.py. That module is not in the repository. __pycache__ holds .pyc files for cnns_method, defence, dct_utils and head_drop_utils, but no sources. Checked by directory listing. The driver therefore cannot import as shipped.
  • Metric. The paper defines ASR as the share of examples misclassified by the victim §4.1. evaluate.py writes top-1 accuracy on the adversarial images evaluate.py evaluate.py. ASR is 100 minus that value only if the clean image is classified correctly by the same model. The repository does not filter for that. MASR averaging is not in the repository. run_evaluate.sh evaluates all eight ViTs, including each surrogate run_evaluate.sh.
  • Checkpoints. CORR_CKPTS lists checkpoint file names model.py, but it is never used. Weights come from timm's pretrained=True [code: model.py#L26-L29].
  • Dataset path. dataset.py expects the class-name JSON in a transformer/ subfolder under ROOT_PATH dataset.py. The repository keeps that file at the top level, so the path must be fixed by hand.
  • Script name. The README calls our_attack.py; the file is our_attacks.py ReadMe.md.
  • SGM branch. For CaiT and Visformer, the MLP and qkv hooks are exclusive (elif) our_method.py. The default 1,0 is unaffected.

No discrepancies found for: alpha = eps / I §4.1 our_method.py; eps = 16/255 §4.1 our_method.py; T = 130 and lambda = 0.1 §4.2 our_method.py; patch size 16 and image size 224 §4.1 our_method.py; SGM decay 0.5 for the MLP gate §4.2 our_method.py.

Pitfalls

  • Units of epsilon, alpha and the L2 term. The paper gives eps = 16 and alpha = 1.6 with no unit. The code uses [0, 1] images, so these are 16/255 and 1.6/255 our_method.py our_method.py. The gradient of the L2 term has norm lambda, whatever the unit. The cross-entropy gradient scales with the unit of delta. So the balance between the two terms depends on the unit, which the paper does not state.
  • Batch size changes the balance. The loss is a mean over the batch [code: our_method.py#L165]. The L2 norm is taken over the whole batch tensor [code: our_method.py#L183]. With batch size B > 1, the cross-entropy gradient per image is divided by B, while the L2 gradient is not. The README uses B = 1 ReadMe.md. The paper does not state a batch size.
  • Quantised output. Images are saved as 8-bit PNG, clipped to [0, 1] [code: our_method.py#L55-L64], at astype(uint8) our_method.py. A step of 1.6/255 is rounded to multiples of 1/255. The evaluated examples are the quantised PNGs, not the float tensors.
  • Deterministic mask. Because the seed is the iteration index, the patch subsets are fixed across images and runs. The paper says the patches are randomly sampled but does not say whether the subset differs per image §3.3. Each reseed also resets Python's global random state, which may affect other code in the same process.
  • Aggregate checks by hand. The 58.67% in §4.2 equals the mean of the eight "Ours" entries in tab. 1. That mean includes the four surrogate columns. The 42.61% and 22.38% in §4.2 match the averages of tab. 2 (normal: four columns; robust: three columns).
  • Relative, not absolute, gains in the introduction. The 15.86% for ViTs in §1 matches the mean of the relative gains over MI (19.5%) and SGM (12.2%) computed from tab. 1 averages. The robust figure of 23.52% matches the same calculation only with rounded table values, 22.38 / 21.53 / 15.64. The normal CNN figure of 27.68% does not match; the same calculation gives 26.9%.
  • Combination claims. For robust CNNs, §4.2 gives 28.54% (MI) and 102.49% (SGM). From tab. 3 the relative gains are 39.9% and 100.6%. 28.54% equals the gain divided by the combined score, (30.13 - 21.53) / 30.13. 102.49% matches neither form I tried.
  • Surrogate columns. §4.1 says the remaining models serve as black-box victims when one model is the surrogate §4.1. The caption of tab. 1 says the ASRs are averaged over surrogates, and those include the model itself tab. 1. Whether a surrogate's own white-box result enters its column is therefore ambiguous.
  • Image count. The paper says "one image per class" §4.1, which is about 1,000 images. The ablation and figure 1 use 2,000 images §4.3 fig. 1. The image count in the shipped archive was not checked.
  • Attention hooks depend on the implementation. PNA is implemented as a backward hook on attn_drop our_method.py. Its equivalence to the paper's stop-gradient depends on timm's attention code. That code is not in the repository, and this was not checked.
  • Momentum decay. The MI variant uses decay 1.0 in code. The paper does not state a value our_method.py.
  • Table 4 victims. The ViTs average in tab. 4 is not defined in the paper, so its values are not directly comparable with tab. 1.

Lineage

  • Builds on: BIM [kurakin2016adversarial], MI [dong2018boosting], and the skip gradient method SGM [wu2020skip] and interaction-reduced attack IR [wang2020unified], as cited in §2.1 and §3.2. The dropout-like patch subsets are described as akin to dropout and diversity input §1. No cards are linked for these works.
  • Compared against: MI, DI, TI, SIM, SGM, IR, TAP, ATA and SE §4.2. TR is not compared §4.2.
  • Journal version: a later IEEE Transactions on Image Processing paper (volume 32, 2023), listed in the repository README ReadMe.md.
  • Superseded by: not stated.

Source and license

Adapted from “Towards Transferable Adversarial Attacks on Vision Transformers” by Zhipeng Wei, Jingjing Chen, Micah Goldblum, Zuxuan Wu, Tom Goldstein, Yu-Gang Jiang (arXiv:2109.04176), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/zhipeng-wei/PNA-PatchOut at commit 85f23c782845 (none (no LICENSE file in repository)). Any quoted code is shown under that license.