Gradient leakage

Overview

In federated training, parties send gradients rather than raw records. A common belief is that larger batches make the gradients harder to invert, because each gradient mixes more records 2110.15122 §1. Gradient leakage is the attack that tests this belief: the attacker starts from random dummy inputs and labels, then changes them until the gradients they produce match the gradients it observed.

The invariant shared by the attacks is that the observed gradients are a fixed set of numbers, while the unknown inputs number K times the input size, where K is the batch size. Recovery therefore depends on extra structure. The attack in CAFE gets that structure from the batch indices: in vertical federated learning, the server sends each batch's row ids to every party, so the server knows which rows each gradient came from [arxiv:2110.15122 §3.1] 2110.15122 §3.3.

Variants

Index-aligned inversion in vertical FL (CAFE)

The attack runs in three stages, each a fitting problem 2110.15122 §3.3.

1. Gradient at the first-layer outputs. A matrix V is fitted so that its sums over each known batch match the first-layer bias gradient 2110.15122 eq. 6 2110.15122 alg. 1. The authors prove this objective is strongly convex when K < N, for fixed model parameters 2110.15122 §3.3 2110.15122 app. B.
2. First-layer inputs. A matrix of estimated inputs is fitted so that its product with V matches the first-layer weight gradient [arxiv:2110.15122 eq. 8] 2110.15122 alg. 2.
3. Data and labels. Dummy rows and dummy labels are optimized against three terms: gradient matching over all parameters, a truncated total-variation term on the dummy inputs, and a term that pulls the dummy rows' first-layer inputs toward the stage-2 estimate 2110.15122 eq. 9 2110.15122 alg. 4.

The two schedules are a nested loop, which runs stages 1 and 2 to a fixed budget before stage 3, and a single loop, which does one update of each stage per iteration 2110.15122 alg. 3 2110.15122 alg. 4. The text says the single loop needs fewer iterations 2110.15122 §4.2. Table 5 shows a mixed picture: the single loop took 7300, 6600 and 12400 iterations on CIFAR-10, MNIST and Linnaeus 5, while the nested row for Step III took 1635 and 2468 on CIFAR-10 and MNIST, and the nested Step II and III rows hit the 20000 cap on Linnaeus 5 2110.15122 tab. 5.

Author-reported results at batch size 40 with four workers: PSNR 31.83 on CIFAR-10, 43.15 on MNIST and 33.22 on Linnaeus 5, against 3.86 to 18.94 for five baselines the authors implemented 2110.15122 tab. 2. Removing the internal-representation term hurts most in the authors' ablation 2110.15122 tab. 4. The paper reports a maximum of 800 leaked images at batch size 100, 4 workers, on its own hardware 2110.15122 §4.

Earlier attacks (named only)

The paper compares against DLG, iDLG, inverting gradients, SAPAG, R-GAP, a theory-oriented attack, GradInversion and a leakage evaluation framework 2110.15122 §2 2110.15122 tab. 1. Those papers have no card here, so this page describes none of their methods. The DLG objective is the baseline that stage 3 generalizes 2110.15122 §3.3. Links for reading: DLG on arXiv and GradInversion on arXiv.

When to use which

The CAFE attack applies when its conditions hold. Check these against your setup before deciding how much weight the paper's results carry.

Defense comparison. The paper compares fake gradients with gradient clipping to norm 3 under differential privacy at several epsilon values. At 8000 iterations the training loss is 2.69 under every DP setting, 1.15 with fake gradients and 0.46 with true gradients 2110.15122 tab. 11. The authors say the noise needed to block leakage also stops the loss from falling 2110.15122 app. F.3.

Implementation caveats. Steps 1 and 2 are proven only for fixed model parameters 2110.15122 §3.3. The paper's worked example has layer width 1024 with N = 800, which fits the Theorem 2 condition N < d2; the released code uses width 256 and the card's Pitfalls section explains why the guarantee then does not cover it. The released code also differs from the paper on learning rates, the TV gate and the PSNR definition, and covers MNIST only. Treat reproduction numbers as needing re-measurement.

Implementation sketch

The shared core is a loop over batches that fits the three stages in turn. The single-loop form is shown; the nested form runs the first two stages to convergence before each stage-3 update 2110.15122 alg. 3.

# Known to the server: batch row ids s, observed aggregated gradients [§3.1]
# Stage 1: fit V so its batch sums match the first-layer bias gradient [eq. 6-7]
# Stage 2: fit H_hat so H_hat^T V matches the first-layer weight gradient [eq. 8]
# Stage 3: fit dummy inputs x_hat and labels y_hat to the observed gradients,
#          a truncated TV term, and H_hat [eq. 9]
def attack_iteration(s, grads, V, H_hat, x_hat, y_hat):
    V = update_stage1(V, s, grads.first_bias)
    H_hat = update_stage2(H_hat, V, s, grads.first_weight)
    x_hat, y_hat = update_stage3(x_hat, y_hat, s, grads, H_hat)
    return V, H_hat, x_hat, y_hat

Optimizer details are not fixed by the paper for stages 2 and 3 [arxiv:2110.15122 alg. 2] 2110.15122 alg. 4. The per-dataset learning rates and loss weights are in 2110.15122 tab. 9.

Contradictions

None known across papers: this page cites one card. Within that card the authors call fake gradients similar in training trajectory to true gradients 2110.15122 §4.4, while the same paper's Table 11 gives a training loss of 1.15 against 0.46 at 8000 iterations 2110.15122 tab. 11. The batch-size results are not monotonic on CIFAR-10: K = 40 (PSNR 31.83) scores below K = 20 (35.70), and the paper gives no explanation 2110.15122 tab. 3. The text's claim that the single loop needs fewer iterations [arxiv:2110.15122 §4.2] is not borne out for the nested Step III row in Table 5, which needed fewer iterations on CIFAR-10 and MNIST 2110.15122 tab. 5.

Papers

• CAFE: Catastrophic Data Leakage in Vertical Federated Learning: index-aligned three-stage inversion for vertical FL, with a fake-gradient defense.