Differentially private SGD (DP-SGD)

Overview

DP-SGD is stochastic gradient descent with a differential-privacy guarantee. Each step clips every example's gradient to a fixed L2 norm, averages them, and adds Gaussian noise before the parameter update 2101.04535 §2.3. The proven guarantee is an (epsilon, delta) bound on the whole training run. The first card attributes the algorithm and its moments accountant to Abadi et al. 2016 2101.04535 §2.3.

The two cards measure different things. Adversary Instantiation does not propose a new DP-SGD. It asks how tight the proven bound is, by building an attacker that estimates epsilon from experiments. It gives no guidance on choosing training settings beyond the ones it used. Analyzing Leakage of Personally Identifiable Information in Language Models uses DP-SGD as one defence in a language-model pipeline and measures how much personal text the model still emits. Its reference setting is epsilon 8, which the authors justify by deployed systems rather than by a tuned utility target 2302.00539 §IV-C. Every number on this page comes from these two cards and is author-reported.

Variants

The first card compares training settings and attacker models rather than algorithm variants. The entries below are settings, not separate methods, and the last one comes from the second card.

Standard DP-SGD training

The experiments train a 4-layer CNN on MNIST and CIFAR-10 with learning rate 0.15 and clipping norm 1.0. A 3-layer fully connected network is trained on Purchase with learning rate 1.1 and clipping norm 0.05 2101.04535 app. A. The noise multiplier was tried at 0.5, 0.6, 0.9 and 1.0, and the best practical epsilon was kept 2101.04535 app. A. Test accuracy by target epsilon (1, 2, 4, 10) is 95 to 98.5% on MNIST, 53 to 59% on CIFAR-10 and 37 to 90% on Purchase 2101.04535 app. A. Card: Adversary Instantiation.

Black-box audit (API access)

The attacker sees only the final model and cannot change the data. The crafter adds one random sample, and a distinguisher thresholds the model's loss on that sample 2101.04535 §4.1. On CIFAR-10 with target epsilon 4 and delta 1e-5, the empirical lower bound is 0.31 against a proven 4 2101.04535 §4.1.

Poisoning audits

A static poison input, chosen with shadow models, raises the empirical bound above the black-box setting on all three datasets, shown in a figure 2101.04535 §4.2 fig. 4. Poisoning an intermediate model gives only a slight further gain over static poisoning with the final model 2101.04535 §4.3 fig. 5. Adaptive poisoning, which replaces the poison during training, reached 0.37 on Purchase at target epsilon 2, against 0.25 for static poisoning 2101.04535 §4.4.

Gradient and dataset attacks

The gradient attack inserts a fixed watermark gradient and infers it from the weight trajectory. The card reports 0.3 against a proven epsilon of 1, which the paper calls almost tight 2101.04535 §4.5 fig. 7. The dataset attack builds a malicious dataset and reaches 3.6 against a proven 4 2101.04535 §4.6 fig. 8. That is about 90% of the proven value, the largest share in the first card.

Language-model fine-tuning with DP-SGD

The second card fine-tunes GPT-2 small (124M) on ECHR court cases. The DP models use target epsilon 8, delta 1/N for N training records, clipping norm 1.0, 4 epochs and batch size 64, with AdamW and the dp-transformers library over Opacus. The learning rate is not stated 2302.00539 §IV-C. The authors note that the guarantee protects who contributed a record, not a personal detail that appears in many records or in a small group 2302.00539 §II-D.

Under DP at epsilon 8, observed extraction precision on ECHR is about 3% for all three GPT-2 sizes. On Enron and Yelp-Health it stays between 6% and 12% 2302.00539 tab. 3. Reconstruction stays at or below 1%, which the authors do not consider practical 2302.00539 §IV-F. Sentence-level membership inference on ECHR has ROC AUC 0.96 undefended, 0.82 with scrubbing only, and 0.505 with DP and scrubbing together 2302.00539 §IV-H. The second card is the source for these numbers, and it describes DP in this pipeline as one layer beside data scrubbing 2302.00539 §II-D.

When to use which

The cards support choices about what to audit and what to expect, not what privacy level to deploy. Their numbers are lower bounds on leakage for the attacks they tested, so they cannot certify that a model is private.

Implementation sketch

Neither card states the full training recipe for the step itself. The first card does not state the batch size, the epoch count, or where the noise is added relative to batch averaging. The second card gives batch 64 and 4 epochs for its language model but no learning rate 2302.00539 §IV-C. The sketch below follows the first card's description of the step 2101.04535 §2.3 and marks the choices it leaves open.

# One DP-SGD step on a minibatch  [§2.3]
# eta: learning rate; sigma: noise multiplier; b: clipping norm
def dpsgd_step(params, batch, eta, sigma, b):
    # Clip each example's gradient to L2 norm at most b  [§2.3]
    clipped = [clip(grad(params, x, y), b) for (x, y) in batch]
    total = sum(clipped)
    # Gaussian noise with scale sigma * b  [§2.3] [app. A]
    # Adding noise before dividing by the batch size is an assumption;
    # the card does not state the order.
    noise = normal(0, (sigma * b) ** 2, size=total.shape)
    # Average over the batch, then take the step  [§2.3]
    return params - eta * (total + noise) / len(batch)

clip(v, b) scales v by min(1, b / ||v||_2) §2.3. The loss, the batch sampling scheme and the number of steps are not stated app. A. To audit, run the attack game many times and convert the error rates to a lower bound on epsilon with the Clopper-Pearson bounds in eq. 5 §3.3.

Contradictions

No two cards report conflicting numbers for the same setting. Several points need care, and none is resolved here.

• Different leakage measures. The first card reports an empirical lower bound on epsilon for image and tabular training. The second reports the share of personal text extracted, and membership AUC, for GPT-2 on text. At target epsilon 4 the first card gets 0.31 for black-box CIFAR-10 2101.04535 §4.1. At target epsilon 8 the second gets about 3% extraction for ECHR 2302.00539 tab. 3. These numbers do not measure the same thing and should not be compared directly.
• Gradient attack tightness. The first card reports 0.3 against a proven epsilon of 1 and quotes the paper's phrase "almost tight" 2101.04535 §4.5. The gap is large, and the card does not explain the phrase in terms of the numbers.
• Trial counts. Section 4.1 says 1,000 trials, while the appendix gives 1,000 for most experiments and 1,000,000 for the malicious dataset 2101.04535 §4.1 2101.04535 app. A.
• Noise multiplier selection. The best practical epsilon was kept from four tried noise multipliers 2101.04535 app. A. The card notes that choosing settings by the measured epsilon makes the reported bounds optimistic.

Papers

• Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning: builds a DP-SGD attack game and reports empirical epsilon lower bounds for black-box, poisoning, gradient and dataset attackers.
• Analyzing Leakage of Personally Identifiable Information in Language Models: measures PII extraction, reconstruction and inference from GPT-2 fine-tuned with and without DP-SGD and scrubbing.