---
kind: concept
slug: expectation-over-transformation
title: Expectation over transformation
summary: >-
  Expectation over transformation (EOT) averages an attack's objective over sampled transformations, such as scale, lighting and blur, so a physical perturbation is optimised for many views of the object. One card on record uses it in a black-box shadow attack on traffic signs, with author-reported outdoor results. No other card covers the technique yet.
tags: [adversarial-attacks, physical-world, robustness]
papers: ["2203.03818"]
related: [adversarial-robustness, adversarial-training, particle-swarm-optimization]
status: draft
updated_at: 2026-11-08
---

## Overview

Expectation over transformation asks an attack to work on average across the
changes a physical object undergoes before a camera records it. Instead of
scoring a perturbed input once, the attacker draws a set of transformations,
applies each one to the perturbed input, and optimises the mean of the results
[arxiv:2203.03818 §3.4] [arxiv:2203.03818 eq. 5].

Only one card on record uses the technique. Its authors credit it to Athalye
et al. [arxiv:2203.03818 §3.2], but that paper has no card here, so this page
cannot describe the original work. Everything below comes from the shadow
attack.

## Variants

### Shadow attack with EOT and a stabilisation stage

Card: [Shadows can be Dangerous](/papers/2203.03818).

The attack runs in two stages, and only the physical attack uses EOT. The
digital stage optimises one fixed image [arxiv:2203.03818 §3.3].

- **Transforms.** Each sample applies downsampling, a brightness change, a
  perspective change, motion blur and a random shadow coefficient k. The
  objective is the mean over 10 transformed samples plus the unperturbed image
  [arxiv:2203.03818 §3.4] [arxiv:2203.03818 eq. 5].
- **Stabilisation stage.** Stage 1 finds a wrong label. Stage 2 reruns the
  search to maximise the expected probability of that label over the same
  transforms, while the predicted label must stay the same
  [arxiv:2203.03818 §3.4] [arxiv:2203.03818 eq. 6].
- **Reported result.** In one outdoor video of 220 frames, the authors report
  100.00% misclassification, and 95.91% of frames predicted as speed limit 35
  [arxiv:2203.03818 §4.3]. This is one sign and one video, and it is
  author-reported.
- **Code.** The released code applies the random transforms only in the first
  half of the search iterations, and it draws the shadow coefficient from a
  range that the paper's text does not give. Read the card's Paper vs code
  section before copying a schedule [arxiv:2203.03818 §3.4].

## When to use which

| Situation | Use | Why |
|---|---|---|
| The attack must survive printing, camera distance and viewpoint, and only output scores are visible | EOT over sampled transforms, as in the shadow attack | The paper combines EOT with a black-box search for this threat model [arxiv:2203.03818 §3.2] [arxiv:2203.03818 §3.4] |
| The attack targets a fixed digital image | Plain objective, no sampled transforms | The digital stage in the paper optimises a single image [arxiv:2203.03818 §3.3] |
| The goal is one chosen wrong label, not any wrong label | Add a second stage that keeps the chosen label stable across transforms | The paper says a targeted misclassification is hard to reach with a shadow's small perturbation space, and uses this stage for it [arxiv:2203.03818 §3.4] |
| The number of transform samples must be chosen | No guidance on record | The card records ablations over polygon edges and restarts, not over the number of EOT samples [arxiv:2203.03818 tab. 4] [arxiv:2203.03818 tab. 5] |

## Implementation sketch

Language-agnostic pseudocode of the core. The paper's transform ranges are not
given in its text, so the sampler is left as a parameter.

```pseudocode
# Expected-objective fitness over sampled transforms  [arxiv:2203.03818 eq. 5]
# Input:  candidate x_adv, true label y, sampler sample_T, count n
# Output: mean probability of label y over the transformed copies

def eot_fitness(x_adv, y, sample_T, n):
    # n random transforms, plus the unperturbed candidate  [arxiv:2203.03818 §3.4]
    transforms = [sample_T() for _ in range(n)] + [identity]
    probs = []
    for t in transforms:
        # Transform the candidate, then query the model once per copy
        probs.append(model_prob(t(x_adv), y))
    return mean(probs)
```

## Contradictions

None known. Only one card is on record for this concept.

## Papers

- [Shadows can be Dangerous](/papers/2203.03818): a black-box physical shadow attack on traffic signs that uses EOT and a second stage to keep a targeted wrong label stable.