Model inversion

Overview

Model inversion runs a trained classifier or embedding network backwards. The victim model is never changed. The input is the variable being optimized: starting from noise, the attacker follows gradients until the model's output matches a chosen target, either a training class or an arbitrary embedding vector 1801.07698 §3.3. The survey defines the general goal the same way: given a pre-trained model and possibly some prior knowledge, produce reconstructions that resemble the private training set, without access to that set 2402.04013 §2.1.

Matching the output alone does not give natural-looking inputs, so each variant adds some kind of prior. What the prior needs differs:

• ArcFace reads the BatchNorm statistics already stored in the model, so it needs no extra data 1801.07698 §3.3, eq. 8.
• Generative attacks need a generator, usually trained on public data that resembles the private set 2402.04013 §4.1.2.
• Black-box attacks need no gradients from the model, but many queries 2402.04013 §4.1.4.

The survey also reports the field's history. Early work targeted tabular models, and it did not transfer well to deep networks on RGB images 2402.04013 §1, §3. Compared with membership inference, which reveals narrow aspects of the training data, inversion can reconstruct training samples 2402.04013 §1.

Variants

Closed-set inversion with a BatchNorm prior (ArcFace)

The model is inverted toward a class. Each BatchNorm layer already holds a running mean and variance over the training set. The generated batch is penalized when its own per-layer activation statistics drift from those stored values. This gives a prior that needs no data and no generator [arxiv:1801.07698 §3.3, eq. 8]. Simple pixel regularizers (total variation and L2 norm of the image) are added as well 1801.07698 §4.5. The objective is the ArcFace loss plus \lambda times the statistics term, with \lambda = 0.05 1801.07698 §3.3.

L_8= \sum_{i=0}^L \|\tilde \mu_i^r - \mu_i\|_2^2 + \|\tilde \sigma_i^r - \sigma_i\|_2^2 1801.07698 eq. 8

Here \tilde\mu_i^r, \tilde\sigma_i^r are the mean and standard deviation of the generated batch's activations at BN layer i, and \mu_i, \sigma_i the values stored in that layer. Reported setup: Adam, learning rate 0.25, 20K steps, a batch of 256 images of different identities at 112×112 on one V100, TV and L2 weights 1e-3 and 1e-4 1801.07698 §4.5. Card: ArcFace: Additive Angular Margin Loss for Deep Face Recognition.

Open-set inversion (embedding target)

The classification loss is swapped for an L2 distance between the embedding of the generated image and a target embedding. Any feature vector can then be turned into a face, not only the class centers learned during training 1801.07698 §3.3, §4.5. In the reported experiments the open-set inversions are slightly worse than closed-set ones on every metric (FID 73.16 vs 70.39, cosine similarity 0.5849 vs 0.6248 for ArcFace with m=0.5) 1801.07698 tab. 10, tab. 11. Card: ArcFace.

Effect of the training loss of the inverted model

The same procedure was applied to ResNet50 models trained on MS1MV3 with different losses, and the generated faces were scored by a separate ResNet100 ArcFace model trained on IBUG-500K 1801.07698 tab. 10, tab. 11:

1801.07698 tab. 10, tab. 11

All three margin losses beat plain softmax, with ArcFace best on every column. Real LFW faces reach 99.81% with the inverting R50 model, and its own inversions 97.75% when verified by that same model 1801.07698 §4.5, so a model judging its own inversions looks far more favorable than an independent scorer does.

Generative inversion for image classifiers

The survey's central formulation, for a classifier f_{\theta}, a generator G and target class c, searches the generator's latent space instead of the pixels 2402.04013 §4.1.1:

\mathbf{z}^{*} = \arg\min_{\mathbf{z}} \mathcal{L}_{iden}(f_{\theta}(G(\mathbf{z})), c) + \lambda \mathcal{L}_{prior}(\mathbf{z}; G) 2402.04013 eq. 3

The latent starts from a Gaussian, and the reconstruction is G(\mathbf{z}^{*}) 2402.04013 §4.1.1. The identity loss is usually cross-entropy on the target label. The prior term is an image-realism regularizer, often a discriminator loss 2402.04013 §4.1.1, §4.1.4. The survey sorts the methods by four design choices:

• Generative prior 2402.04013 §4.1.2. GMI trains a GAN on public data with images similar to the private data; in early work this limited resolution to 64×64. Pre-trained StyleGAN or BigGAN generators allow higher resolution. DiffusionMI replaces the GAN with a conditional diffusion model whose conditions are pseudo labels from the target model.
• Data initialization 2402.04013 §4.1.3. Sample many latents and keep those whose generated images the target classifies with high confidence. C2FMI trains an inverse network from confidence to feature space to obtain coarse initial latents. GMI and SecretGen initialize from blurred or masked versions of the private image.
• Attack process 2402.04013 §4.1.4, tab. 2. Momentum SGD or Adam in the white-box case. PPA swaps cross-entropy for a Poincaré distance 2402.04013 eq. 4. PLGMI uses a max-margin loss 2402.04013 eq. 5. LOMMA bypasses the softmax and maximizes the target logit. AlignMI averages losses over a local neighbourhood of each sample, without training. Several methods drop the discriminator loss, because it can pull results towards the public data distribution.
• Search space 2402.04013 §4.1.4. The usual space is the latent \mathcal{Z}. StyleGAN maps to the more disentangled \mathcal{W} space. Mirror and SMILE clip in a \mathcal{P} space, within [\mu - \sigma, \mu + \sigma]. IF-GMI optimizes the intermediate features of the generator layer by layer, each under an l_1-ball constraint 2402.04013 eq. 7. PPDG fine-tunes the GAN prior on selected pseudo-private samples.

Additional techniques, such as pseudo-label guidance, augmentation and selecting results by the target's confidence after augmentation, are described in 2402.04013 §4.1.5.

Black-box inversion

When only confidence scores or hard labels are visible, the survey covers several routes 2402.04013 §4.1.4:

• Conversion to a white-box surrogate (SecretGen, LOKT).
• Reinforcement learning with Soft Actor-Critic (RLBMI).
• Label-only gradient estimation (BREPMI), which estimates the gradient from N random directions at radius r 2402.04013 eq. 6.
• Genetic algorithms (Mirror, C2FMI).
• An optimization-free mapping from predictions to styles (P2I-MI) and conditional diffusion sampling (DiffusionMI).

Black-box attacks need many queries, which is the motivation for SMILE 2402.04013 §4.1.4.

Generative models as the target

For generative models the survey describes direct extraction of training data. GAN training images can be recovered by sampling the generator repeatedly. For diffusion models, sampling many images from a text prompt and keeping the largest clique of similar images gives memorized training images. SIDE extends this to unconditional diffusion models, using a time-dependent classifier as a surrogate condition 2402.04013 §4.2. The survey reports, citing Carlini et al., that diffusion models leak nearly twice as much training data as GAN models 2402.04013 §4.2.

Text and graph models

For language models, the survey groups attacks into embedding optimization, inverse mapping from sentence embeddings back to text, token search by perplexity, and prompt recovery from next-token probabilities or output sequences 2402.04013 §5.1, eq. 8. For graph models, attacks optimize a dummy adjacency matrix by gradient descent on the output distance 2402.04013 eq. 9, maximize mutual-information terms in a chain-based variant 2402.04013 eq. 10, or train an autoencoder and apply its decoder to the target's output embeddings 2402.04013 §5.2. No card in this wiki gives implementation detail for these.

Defenses

The survey splits defenses into two families 2402.04013 §6.

• Model output processing 2402.04013 §6.1. Additive noise, including adversarial perturbation of confidences, an exponential mechanism from differential privacy, and Laplacian noise. Purification trains an autoencoder to reduce the spread of confidences, against an adversarial inversion model. For language models, output filtering detects and refuses sensitive content. These defenses mostly target black-box APIs and do not protect open-source or offline models.
• Robust model learning 2402.04013 §6.2. Mutual-information penalties (MID, BiDO), label smoothing with a smoothing factor that can also be negative, an anti-adversarial term (AT-AT), and scaling the last-stage skip connections (SSF). Other methods are transfer learning that fine-tunes only the last layers, GAN-based misleading fine-tuning, low-rank feature filtering (LoFt), trigger injection (Trap-MID) and training-data de-duplication.

The survey gives no common benchmark and no numerical comparison of defenses, and no certified-robustness guarantees exist for them 2402.04013 §6, §8. The ArcFace card does not study defenses.

When to use which

Implementation sketch

Common core of the variants above. The ArcFace variant uses pixels as z and no generator; generator variants map latents through G.

# Model inversion, common core  [arxiv:2402.04013 eq. 3] [arxiv:1801.07698 §3.3]
# Input:  frozen model f, target t (class label or embedding), steps T,
#         optional generator G (None for pixel-space inversion)
# Output: reconstructed input x*

def invert(f, t, T, G=None):
    # Start from Gaussian noise: pixels, or latents when G is given
    # [arxiv:1801.07698 §3.3] [arxiv:2402.04013 §4.1.1]
    z = sample_gaussian(batch_shape)
    for step in range(T):
        # Generator variants map latents to images first [arxiv:2402.04013 §4.1.1]
        x = G(z) if G is not None else z
        # Task loss: training loss for a class target, L2 for an embedding
        # [arxiv:1801.07698 §3.3]
        loss = task_loss(f, x, t)
        # Prior: BN-statistics match (eq. 8), generator prior, or none
        # [arxiv:1801.07698 eq. 8] [arxiv:2402.04013 §4.1.4]
        loss = loss + lam * prior(f, x)
        # Pixel variant also penalizes total variation and L2 norm [arxiv:1801.07698 §4.5]
        if G is None:
            loss = loss + a * tv(x) + b * l2(x)
        # Optimizer step: Adam in the ArcFace setup, Adam or momentum SGD
        # in the survey's white-box attacks [arxiv:2402.04013 §4.1.4]
        z = optimizer_step(z, loss)
    # Generator variants augment, re-score and keep the best [arxiv:2402.04013 §4.1.5]
    return select_best(G(z), f, t) if G is not None else z

Notes:

• Reported ArcFace values: \lambda = 0.05, Adam with learning rate 0.25, T = 20K, TV 1e-3, L2 1e-4, B = 256 1801.07698 §3.3, §4.5.
• 1801.07698 eq. 8 uses standard deviations, while BN buffers hold variances; take the square root 1801.07698 §3.3.
• The ArcFace source does not say whether the batch statistics are taken at the BN input or output. The input is the natural match for the stored running statistics; this is our reading, not the paper's.
• Running the model in eval mode keeps the stored BN statistics fixed. This is our note; neither source states it.
• The survey states no training hyperparameters for any method, including \lambda, learning rates and step counts. Read each source paper for those values (survey card). Do not fill them in from ArcFace's values.
• Loss choice changes gradient scale. Max-margin loss works on logits and cross-entropy on softmax outputs 2402.04013 §4.1.4, so retune the step size after switching.

Contradictions

None known between the two cards on any shared claim. One discrepancy sits between a paper and its code. The survey's Poincaré loss 2402.04013 eq. 4 is written with unit constants, while the text says to replace 1 with 0.9999, and the released PPA code applies that change to the target vector instead 2402.04013 §4.1.4. Implement the code's behaviour to reproduce the toolbox.

Papers

• ArcFace: Additive Angular Margin Loss for Deep Face Recognition: inverts face recognition models with the ArcFace loss plus a BatchNorm-statistics prior, in closed-set and open-set forms, and shows margin-trained models invert to better faces than softmax ones.
• Survey of model inversion attacks and defenses on DNNs (Fang et al.): taxonomy of model inversion attacks by attacker access, modality and task, covering generative, black-box, text and graph methods, plus defense families; reports no new benchmark numbers.