Expectation over transformation

Overview

Expectation over transformation asks an attack to work on average across the changes a physical object undergoes before a camera records it. Instead of scoring a perturbed input once, the attacker draws a set of transformations, applies each one to the perturbed input, and optimises the mean of the results 2203.03818 §3.4 2203.03818 eq. 5.

Only one card on record uses the technique. Its authors credit it to Athalye et al. 2203.03818 §3.2, but that paper has no card here, so this page cannot describe the original work. Everything below comes from the shadow attack.

Variants

Shadow attack with EOT and a stabilisation stage

Card: Shadows can be Dangerous.

The attack runs in two stages, and only the physical attack uses EOT. The digital stage optimises one fixed image 2203.03818 §3.3.

• Transforms. Each sample applies downsampling, a brightness change, a perspective change, motion blur and a random shadow coefficient k. The objective is the mean over 10 transformed samples plus the unperturbed image 2203.03818 §3.4 2203.03818 eq. 5.
• Stabilisation stage. Stage 1 finds a wrong label. Stage 2 reruns the search to maximise the expected probability of that label over the same transforms, while the predicted label must stay the same 2203.03818 §3.4 2203.03818 eq. 6.
• Reported result. In one outdoor video of 220 frames, the authors report 100.00% misclassification, and 95.91% of frames predicted as speed limit 35 2203.03818 §4.3. This is one sign and one video, and it is author-reported.
• Code. The released code applies the random transforms only in the first half of the search iterations, and it draws the shadow coefficient from a range that the paper's text does not give. Read the card's Paper vs code section before copying a schedule 2203.03818 §3.4.

When to use which

Implementation sketch

Language-agnostic pseudocode of the core. The paper's transform ranges are not given in its text, so the sampler is left as a parameter.

# Expected-objective fitness over sampled transforms  [arxiv:2203.03818 eq. 5]
# Input:  candidate x_adv, true label y, sampler sample_T, count n
# Output: mean probability of label y over the transformed copies

def eot_fitness(x_adv, y, sample_T, n):
    # n random transforms, plus the unperturbed candidate  [arxiv:2203.03818 §3.4]
    transforms = [sample_T() for _ in range(n)] + [identity]
    probs = []
    for t in transforms:
        # Transform the candidate, then query the model once per copy
        probs.append(model_prob(t(x_adv), y))
    return mean(probs)

Contradictions

None known. Only one card is on record for this concept.

Papers

• Shadows can be Dangerous: a black-box physical shadow attack on traffic signs that uses EOT and a second stage to keep a targeted wrong label stable.