---
kind: concept
slug: backdoor-attack
title: Backdoor attacks (trojans) and their repair
summary: >-
  A backdoor hides a trigger in a model's training data, so triggered inputs get the attacker's label while clean inputs behave normally. A survey sorts attacks and defenses into one taxonomy with no experiments. One neuron-pruning method repairs a backdoored model after training, without fine-tuning. All numbers are author-reported, from two cards only.
tags: [security, backdoor, trojan, model-repair]
papers: ["2012.10544", "2110.14430"]
related: [data-poisoning, adversarial-robustness, network-pruning, federated-learning]
status: draft
updated_at: 2026-10-31
---

## Overview

A backdoor, also called a trojan, is planted by poisoning training inputs with
a trigger. The trained model then maps any test input that carries the trigger
to an attacker-chosen label, while clean inputs behave as usual
[arxiv:2012.10544 §3]. The threat arises when training data or training itself
comes from a party the trainer does not control. Training may be outsourced to
a third-party platform, which can return a model with a backdoor inside
[arxiv:2110.14430 §1].

Backdoors sit next to training-only data poisoning, which changes the trained
model without any trigger at test time [arxiv:2012.10544 §1] [arxiv:2012.10544 §3].
See [data-poisoning](/concepts/data-poisoning) for that side. The two cards here
cover two different jobs. The survey maps attacks and defenses. The ANP paper
proposes no attack. It treats a backdoored network as something to repair
after training, and tests it against several known backdoor attacks
[arxiv:2110.14430 §1] [arxiv:2110.14430 app. A].

## Variants

### Attack families (survey taxonomy, no numbers)

The survey splits backdoor attacks into model-agnostic ones and
model-specific ones [arxiv:2012.10544 §3.2] [arxiv:2012.10544 §3.3]. A
model-agnostic backdoor is described as working across models trained on the
same poisoned set [arxiv:2012.10544 §3.2]. Model-specific attacks use the
architecture or a pre-trained model. They have their own transfer-learning and
federated cases [arxiv:2012.10544 §3.3.3] [arxiv:2012.10544 §3.3.4]. Clean-label
backdoors borrow the feature-collision idea from poisoning attacks
[arxiv:2012.10544 §2.2] [arxiv:2012.10544 §3.3.2]. The survey also lists
application areas [arxiv:2012.10544 §3.1]. It gives no effectiveness figures for
any of these attacks. Card: [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](/papers/2012.10544).

### Defenses (survey taxonomy)

The survey groups defenses by when they act [arxiv:2012.10544 §4]:

- **Detection of poisoned data** [arxiv:2012.10544 §4.1]. The survey warns that
  outlier-based detection can be bypassed by adaptive poisons placed close
  together [arxiv:2012.10544 §4.1.1].
- **Detection of poisoned models** [arxiv:2012.10544 §4.2]. Trigger
  reconstruction needs clean samples and full model access
  [arxiv:2012.10544 §4.2.1].
- **Repair after training** [arxiv:2012.10544 §4.3]. This is the setting of the
  ANP method below.
- **Training-time defenses**: certified smoothing, majority votes, differential
  privacy and input preprocessing [arxiv:2012.10544 §4.4]. The survey says the
  empirical protection from differential privacy is far weaker than its
  worst-case theory [arxiv:2012.10544 §4.4.3].
- **Federated defenses**: robust aggregation, robust training and post-training
  repair [arxiv:2012.10544 §4.5].

The survey calls certified defenses far from meaningful guarantees at realistic
scale [arxiv:2012.10544 §4.6]. It compares no methods with numbers.

### Backdoor repair after training by neuron pruning

Adversarial Neuron Pruning (ANP) assumes the defender has a backdoored network
and a small clean set, and does not know the trigger [arxiv:2110.14430 §3.1].
It gives each neuron a mask in [0, 1]. The masks are optimised while weights and
biases are perturbed in the worst case, within a budget. Neurons whose mask
falls below 0.2 are then pruned. The remaining weights are not changed and
there is no fine-tuning [arxiv:2110.14430 §3.3] [arxiv:2110.14430 §4.1].

The motivating experiment finds that backdoored networks misclassify more under
such perturbations, and mostly into the target label. The authors attribute
this to backdoor neurons that are dormant on clean inputs
[arxiv:2110.14430 §3.2].

Author-reported results on CIFAR-10 with ResNet-18 and 1% of the training
images (500 images) across six attacks [arxiv:2110.14430 §4.1] [arxiv:2110.14430 tab. 1]:

- Attack success rate (ASR) falls below 1% on five attacks. On the sixth it is
  3.98%.
- Average clean-accuracy drop is 1.47 points.
- Fine-tuning at two learning rates drops clean accuracy by 3.53 and 7.04
  points on average. Its ASR on Blend is 47.17% at the lower rate and 10.20% at
  the higher one [arxiv:2110.14430 tab. 1].
- ANP takes about 2.5 times the fine-tuning time at the same iteration count.
  The paper names no hardware [arxiv:2110.14430 §4.2] [arxiv:2110.14430 tab. 2].

Reproduction caveats:

- The companion card's code review found that the released code covers only
  three of the six attacks, and that its noise handling differs from the
  algorithm box. It also found that the pruning comparison is written with
  "less than or equal to" where the text says "smaller than". Read
  [the ANP card](/papers/2110.14430) before rerunning it.
- The clean-accuracy numbers at very small clean sets are the weak point. See
  the table in the next section.

Card: [Adversarial Neuron Pruning Purifies Backdoored Deep Models](/papers/2110.14430).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You need the threat model or a literature map before choosing a method | Survey card [arxiv:2012.10544] | It covers attacks and defenses in one taxonomy. It has no experiments, so it gives no performance guidance [arxiv:2012.10544 §5]. |
| You have a backdoored model from a third party, no training pipeline and about 1% of clean data | Post-training pruning such as ANP [arxiv:2110.14430] | One pass with no fine-tuning. The authors report ASR below 1% on five of six attacks [arxiv:2110.14430 tab. 1]. |
| The clean set is very small (about 50 images) and clean accuracy matters most | Check the trade-off first | With 50 images, ANP's clean accuracy is 85.66–89.54%, below fine-pruning's 89.27–91.29% on all six attacks. ANP's ASR on Blend is 5.70%, against 99.95% for fine-pruning [arxiv:2110.14430 tab. 9]. |
| Compute or wall-clock time is tight | Fine-tuning, with the accuracy cost in mind | Fine-tuning was faster in the authors' table (93.80 s against 241.5 s for ANP, hardware not stated) [arxiv:2110.14430 tab. 2]. It drops clean accuracy more [arxiv:2110.14430 tab. 1]. |
| You use a different architecture, such as MobileNet-V2 | Tune the budget and threshold on your model | The paper's good threshold band of about 0.2 to 0.5 was seen for ResNet-18, ResNet-50 and VGG-19. MobileNet-V2 needed a smaller budget of 0.2 and lost clean accuracy once more than 5% of neurons were pruned [arxiv:2110.14430 §4.3] [arxiv:2110.14430 fig. 3]. |
| Your attack is not one of the three the released code covers | Rerun the attack in your own setup first | The paper reports six attacks, but the released code covers three. Results for the others cannot be checked from that code [arxiv:2110.14430 tab. 1]. |
| The trigger is known, or reconstruction is possible with full model access | Either; no direct comparison in the cards | The survey says trigger reconstruction needs clean samples and full model access [arxiv:2012.10544 §4.2.1]. ANP needs no trigger knowledge [arxiv:2110.14430 §1]. Neither card compares the two on numbers. |

Pruning budget matters. With 20 mask-optimisation iterations, pruning about 4%
of neurons with the smallest masks gave ASR of 6.28%, and ASR rose again after
about 10% were pruned. With 100 or more iterations the authors report that this
effect disappears [arxiv:2110.14430 §4.4] [arxiv:2110.14430 fig. 4]. Use at
least 100 iterations unless you have checked the curve on your model.

## Implementation sketch

The common core of post-training repair is a per-neuron mask, a worst-case
perturbation inside a budget, and a threshold prune. This follows the ANP
algorithm and threshold rule [arxiv:2110.14430 alg. 1] [arxiv:2110.14430 §3.3].
The other methods in the ANP results table are baselines there, not sketched
here [arxiv:2110.14430 tab. 1].

```pseudocode
# Post-training backdoor repair with a per-neuron mask
#   [arxiv:2110.14430 alg. 1] [arxiv:2110.14430 §3.3]
# Input:  backdoored net, clean set D (1% of train data in the paper),
#         trade-off alpha, budget eps, prune threshold t (0.2 in the paper)
# Output: repaired net, neurons with mask below t removed

def repair(net, D, alpha, eps, t):
    # Start every mask at 1, i.e. the unpruned network  [alg. 1, line 2]
    masks = ones(num_neurons(net))
    for batch in D:
        # Random start in [-eps, eps], then one signed-gradient step
        #   [alg. 1, lines 5-9] [§3.2]
        delta, xi = worst_case_perturbation(net, masks, batch, eps)
        # Mix clean loss and worst-case loss  [eq. 8] [alg. 1, line 10]
        loss = alpha * clean_loss(net, masks, batch) \
             + (1 - alpha) * adv_loss(net, masks, delta, xi, batch)
        masks = sgd_step(masks, loss)
        # Keep masks in [0, 1]  [alg. 1, line 11]
        masks = clip(masks, 0, 1)
    # Prune low-mask neurons, with no fine-tuning  [§3.3] [§4.1]
    return prune(net, masks < t)
```

Where variants differ: the survey's other repair and training-time defenses
have no pseudocode in these cards [arxiv:2012.10544 §4.3] [arxiv:2012.10544 §4.4].
Batch norm handling is also a choice. The ANP paper perturbs batch-norm scale
and shift rather than weight and bias [arxiv:2110.14430 §3.3].

## Contradictions

- **Within the ANP paper, at 50 clean images.** The appendix text says ANP keeps
  ASR below 5% at this size, but its table gives 5.70% on Blend
  [arxiv:2110.14430 app. C.3] [arxiv:2110.14430 tab. 9]. Report the table value.
- **Survey warning and ANP's single setting.** The survey warns that reported
  poisoning results are hard to compare across settings, and that many methods
  work less well when the victim's architecture, optimiser or augmentation
  differs [arxiv:2012.10544 §2.8] [arxiv:2012.10544 §5]. ANP's numbers come from
  one architecture and one dataset in the paper [arxiv:2110.14430 §4.1]. The
  two cards do not conflict, but the ANP numbers do not show how well the method
  transfers.

## Papers

- [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](/papers/2012.10544): survey of poisoning, backdoor and federated threats, and their defenses. No experiments.
- [Adversarial Neuron Pruning Purifies Backdoored Deep Models](/papers/2110.14430): post-training repair of a backdoored network by learned neuron masks and pruning, with author-reported results on CIFAR-10.