Adversarial training

Overview

Adversarial training is a min-max problem. An inner step searches for a perturbed input inside a small ball around each clean input and maximises the loss there. An outer step updates the parameters on those perturbed inputs 2112.08304 eq. 1 2112.08304 §1. The paper argues that the quality of the inner search governs the robustness the outer step can reach, and that no criterion existed for measuring that quality for common attacks 2112.08304 §1 2112.08304 §2.1.

Only one source is on record for this page, so the variants below all come from one paper. Other cards mention adversarial training in passing, and this page does not use them.

Variants

Standard PGD adversarial training

The baseline runs a fixed-length projected gradient ascent (PGD) on each batch and trains on the result. The paper's setting is 10 steps with a random start, and a step size of ε/4 2112.08304 §5.1. The paper attributes this recipe to Madry et al. 2112.08304 §2.2. Card: On the Convergence and Robustness of Adversarial Training.

Curriculum adversarial training

The curriculum baseline starts with weak attacks and makes them stronger over training. Here the difficulty knob is the PGD step count, as in the Cai et al. curriculum that the paper compares against 2112.08304 §4.3 2112.08304 §5.1. The paper says more steps can overshoot, which is its motivation for a different difficulty measure 2112.08304 §4.3. Card: On the Convergence and Robustness of Adversarial Training.

Dynamic adversarial training with a FOSC threshold

The paper's own method controls the inner attack per example. It defines a first-order stationarity score, FOSC, for an inner point. A value of zero means the point is a stationary point of the inner problem, and a small value means the point is close to the inner maximum 2112.08304 §3.1 2112.08304 eq. 4. The score has a closed form: ε times the L1 norm of the input gradient, minus the inner product of the displacement from the clean input with that gradient 2112.08304 §3.1.

Training then uses a threshold c_t that falls linearly from c_max to zero at epoch T′. Each example keeps being attacked only while its FOSC is above c_t 2112.08304 §4.1 2112.08304 alg. 1. The paper uses c_max = 0.5 for both datasets 2112.08304 §5.1. Its CIFAR-10 run uses T′ = 100 of 120 epochs and its MNIST run uses T′ = 40 of 50 2112.08304 §5.1.

An observation in the paper motivates this. FOSC is much lower early in training, which the authors read as overfitting to strong attacks, and a one-step FGSM warm-up in the first 20 epochs gives higher final robustness 2112.08304 §3.2 2112.08304 fig. 3. Card: On the Convergence and Robustness of Adversarial Training.

When to use which

The numbers are author-reported, come from one setup, and show single values without any spread. Compare them only within the same table.

Implementation sketch

Only the schedule and the stop rule are needed to see the dynamic variant. The card's full pseudocode is in the paper card. The sketch below follows the card's statement of the rule, not the printed indicator in alg. 1 (see Contradictions).

# Threshold for epoch t  [§4.1]
# Falls linearly from c_max to 0 at T_prime, then stays at 0
def threshold(t, c_max, T_prime):
    return max(c_max - t * c_max / T_prime, 0.0)

# Per-batch attack with a per-example stop  [alg. 1] [§4.1]
def fosc_pgd(model, x0, y, c_t, eps, alpha, K):
    x = x0
    active = ones_like(y)                  # 1 = still being attacked
    for step in range(K):
        grad = input_gradient(model, x, y)
        # Ascent only on active examples, then project to the eps-ball
        x = clip(x + active * alpha * sign(grad), x0 - eps, x0 + eps)
        # Stop an example once its FOSC is at or below the threshold
        active = fosc(grad, x, x0, eps) > c_t  # keep attacking if above
        if not any(active):
            break
    return x

# FOSC, closed form  [§3.1]
def fosc(grad, x, x0, eps):
    return eps * abs(grad).sum(-1) - ((x - x0) * grad).sum(-1)

The outer loop is ordinary training on the returned batch alg. 1. Two points differ between variants. The standard baseline has no threshold, so every example gets the same number of PGD steps §5.1. The curriculum baseline changes the step count over epochs instead of the stop rule §4.3.

Contradictions

Only one card supports this page, so no two papers disagree here. Within the paper, the following conflicts are open:

• Stop condition in alg. 1. The printed indicator for the control vector is 1 where FOSC is at most c_t, while its comment says the element becomes 0 where FOSC is smaller than c_t 2112.08304 alg. 1. The paper's text says "smaller than" 2112.08304 alg. 1, while the code freezes examples at or below the threshold 2112.08304 § Paper vs code. An implementation that follows the printed formula attacks the wrong examples.
• c_max. The paper suggests estimating c_max from FGSM examples, but every run uses 0.5 2112.08304 §4.1 2112.08304 §5.1. No estimation code is in the repository 2112.08304 § Paper vs code.
• Clean accuracy. Standard PGD training reaches 66.31% clean accuracy on CIFAR-10, and the dynamic schedule 72.17%, against 89.39% for an undefended model 2112.08304 tab. 1. The paper does not discuss this gap.
• Weight decay. The paper states weight decay of 1e-4, but the code passes a Keras learning-rate decay argument of the same value 2112.08304 §5.1 2112.08304 § Paper vs code. Reruns must pick one reading.

Papers

• On the Convergence and Robustness of Adversarial Training: proposes the FOSC score and a dynamic schedule that lowers its threshold over training. It also gives a convergence result and compares against standard PGD and a step-count curriculum.