--- kind: concept slug: model-inversion title: Model inversion summary: >- Model inversion recovers inputs from a trained network by optimizing noise until the model assigns a target class or embedding. A BatchNorm-statistics prior needs no training data (ArcFace); generative variants add a GAN or diffusion prior. A 2024 survey covers black-box, text and graph attacks plus defenses. Cost: many gradient steps or queries. tags: [model-inversion, data-free, privacy, computer-vision] papers: ["1801.07698", "2402.04013"] related: [membership-inference, differential-privacy, diffusion-models, face-recognition, angular-margin-loss] status: draft updated_at: 2026-10-19 --- ## Overview Model inversion runs a trained classifier or embedding network backwards. The victim model is never changed. The input is the variable being optimized: starting from noise, the attacker follows gradients until the model's output matches a chosen target, either a training class or an arbitrary embedding vector [arxiv:1801.07698 §3.3]. The survey defines the general goal the same way: given a pre-trained model and possibly some prior knowledge, produce reconstructions that resemble the private training set, without access to that set [arxiv:2402.04013 §2.1]. Matching the output alone does not give natural-looking inputs, so each variant adds some kind of prior. What the prior needs differs: - ArcFace reads the BatchNorm statistics already stored in the model, so it needs no extra data [arxiv:1801.07698 §3.3, eq. 8]. - Generative attacks need a generator, usually trained on public data that resembles the private set [arxiv:2402.04013 §4.1.2]. - Black-box attacks need no gradients from the model, but many queries [arxiv:2402.04013 §4.1.4]. The survey also reports the field's history. Early work targeted tabular models, and it did not transfer well to deep networks on RGB images [arxiv:2402.04013 §1, §3]. Compared with membership inference, which reveals narrow aspects of the training data, inversion can reconstruct training samples [arxiv:2402.04013 §1]. ## Variants ### Closed-set inversion with a BatchNorm prior (ArcFace) The model is inverted toward a class. Each BatchNorm layer already holds a running mean and variance over the training set. The generated batch is penalized when its own per-layer activation statistics drift from those stored values. This gives a prior that needs no data and no generator [arxiv:1801.07698 §3.3, eq. 8]. Simple pixel regularizers (total variation and L2 norm of the image) are added as well [arxiv:1801.07698 §4.5]. The objective is the ArcFace loss plus $\lambda$ times the statistics term, with $\lambda = 0.05$ [arxiv:1801.07698 §3.3]. $$ L_8= \sum_{i=0}^L \|\tilde \mu_i^r - \mu_i\|_2^2 + \|\tilde \sigma_i^r - \sigma_i\|_2^2 $$ [arxiv:1801.07698 eq. 8] Here $\tilde\mu_i^r, \tilde\sigma_i^r$ are the mean and standard deviation of the generated batch's activations at BN layer $i$, and $\mu_i, \sigma_i$ the values stored in that layer. Reported setup: Adam, learning rate 0.25, 20K steps, a batch of 256 images of different identities at 112×112 on one V100, TV and L2 weights 1e-3 and 1e-4 [arxiv:1801.07698 §4.5]. Card: [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](/papers/1801.07698). ### Open-set inversion (embedding target) The classification loss is swapped for an L2 distance between the embedding of the generated image and a target embedding. Any feature vector can then be turned into a face, not only the class centers learned during training [arxiv:1801.07698 §3.3, §4.5]. In the reported experiments the open-set inversions are slightly worse than closed-set ones on every metric (FID 73.16 vs 70.39, cosine similarity 0.5849 vs 0.6248 for ArcFace with $m=0.5$) [arxiv:1801.07698 tab. 10, tab. 11]. Card: [ArcFace](/papers/1801.07698). ### Effect of the training loss of the inverted model The same procedure was applied to ResNet50 models trained on MS1MV3 with different losses, and the generated faces were scored by a separate ResNet100 ArcFace model trained on IBUG-500K [arxiv:1801.07698 tab. 10, tab. 11]: | Model (margin) | Close-set FID | Close-set cos sim | Open-set FID | Open-set cos sim | Open-set LFW acc. | |---|---|---|---|---|---| | Softmax | 75.59 | 0.5612 | 77.85 | 0.5504 | 90.14 | | SphereFace (1.35) | 73.18 | 0.5919 | 75.16 | 0.5687 | 92.05 | | CosFace (0.35) | 71.64 | 0.6176 | 74.02 | 0.5762 | 92.69 | | ArcFace (0.5) | 70.39 | 0.6248 | 73.16 | 0.5849 | 93.30 | [arxiv:1801.07698 tab. 10, tab. 11] All three margin losses beat plain softmax, with ArcFace best on every column. Real LFW faces reach 99.81% with the inverting R50 model, and its own inversions 97.75% when verified by that same model [arxiv:1801.07698 §4.5], so a model judging its own inversions looks far more favorable than an independent scorer does. ### Generative inversion for image classifiers The survey's central formulation, for a classifier $f_{\theta}$, a generator $G$ and target class $c$, searches the generator's latent space instead of the pixels [arxiv:2402.04013 §4.1.1]: $$ \mathbf{z}^{*} = \arg\min_{\mathbf{z}} \mathcal{L}_{iden}(f_{\theta}(G(\mathbf{z})), c) + \lambda \mathcal{L}_{prior}(\mathbf{z}; G) $$ [arxiv:2402.04013 eq. 3] The latent starts from a Gaussian, and the reconstruction is $G(\mathbf{z}^{*})$ [arxiv:2402.04013 §4.1.1]. The identity loss is usually cross-entropy on the target label. The prior term is an image-realism regularizer, often a discriminator loss [arxiv:2402.04013 §4.1.1, §4.1.4]. The survey sorts the methods by four design choices: - *Generative prior* [arxiv:2402.04013 §4.1.2]. GMI trains a GAN on public data with images similar to the private data; in early work this limited resolution to 64×64. Pre-trained StyleGAN or BigGAN generators allow higher resolution. DiffusionMI replaces the GAN with a conditional diffusion model whose conditions are pseudo labels from the target model. - *Data initialization* [arxiv:2402.04013 §4.1.3]. Sample many latents and keep those whose generated images the target classifies with high confidence. C2FMI trains an inverse network from confidence to feature space to obtain coarse initial latents. GMI and SecretGen initialize from blurred or masked versions of the private image. - *Attack process* [arxiv:2402.04013 §4.1.4, tab. 2]. Momentum SGD or Adam in the white-box case. PPA swaps cross-entropy for a Poincaré distance [arxiv:2402.04013 eq. 4]. PLGMI uses a max-margin loss [arxiv:2402.04013 eq. 5]. LOMMA bypasses the softmax and maximizes the target logit. AlignMI averages losses over a local neighbourhood of each sample, without training. Several methods drop the discriminator loss, because it can pull results towards the public data distribution. - *Search space* [arxiv:2402.04013 §4.1.4]. The usual space is the latent $\mathcal{Z}$. StyleGAN maps to the more disentangled $\mathcal{W}$ space. Mirror and SMILE clip in a $\mathcal{P}$ space, within $[\mu - \sigma, \mu + \sigma]$. IF-GMI optimizes the intermediate features of the generator layer by layer, each under an $l_1$-ball constraint [arxiv:2402.04013 eq. 7]. PPDG fine-tunes the GAN prior on selected pseudo-private samples. Additional techniques, such as pseudo-label guidance, augmentation and selecting results by the target's confidence after augmentation, are described in [arxiv:2402.04013 §4.1.5]. ### Black-box inversion When only confidence scores or hard labels are visible, the survey covers several routes [arxiv:2402.04013 §4.1.4]: - Conversion to a white-box surrogate (SecretGen, LOKT). - Reinforcement learning with Soft Actor-Critic (RLBMI). - Label-only gradient estimation (BREPMI), which estimates the gradient from $N$ random directions at radius $r$ [arxiv:2402.04013 eq. 6]. - Genetic algorithms (Mirror, C2FMI). - An optimization-free mapping from predictions to styles (P2I-MI) and conditional diffusion sampling (DiffusionMI). Black-box attacks need many queries, which is the motivation for SMILE [arxiv:2402.04013 §4.1.4]. ### Generative models as the target For generative models the survey describes direct extraction of training data. GAN training images can be recovered by sampling the generator repeatedly. For diffusion models, sampling many images from a text prompt and keeping the largest clique of similar images gives memorized training images. SIDE extends this to unconditional diffusion models, using a time-dependent classifier as a surrogate condition [arxiv:2402.04013 §4.2]. The survey reports, citing Carlini et al., that diffusion models leak nearly twice as much training data as GAN models [arxiv:2402.04013 §4.2]. ### Text and graph models For language models, the survey groups attacks into embedding optimization, inverse mapping from sentence embeddings back to text, token search by perplexity, and prompt recovery from next-token probabilities or output sequences [arxiv:2402.04013 §5.1, eq. 8]. For graph models, attacks optimize a dummy adjacency matrix by gradient descent on the output distance [arxiv:2402.04013 eq. 9], maximize mutual-information terms in a chain-based variant [arxiv:2402.04013 eq. 10], or train an autoencoder and apply its decoder to the target's output embeddings [arxiv:2402.04013 §5.2]. No card in this wiki gives implementation detail for these. ## Defenses The survey splits defenses into two families [arxiv:2402.04013 §6]. - *Model output processing* [arxiv:2402.04013 §6.1]. Additive noise, including adversarial perturbation of confidences, an exponential mechanism from differential privacy, and Laplacian noise. Purification trains an autoencoder to reduce the spread of confidences, against an adversarial inversion model. For language models, output filtering detects and refuses sensitive content. These defenses mostly target black-box APIs and do not protect open-source or offline models. - *Robust model learning* [arxiv:2402.04013 §6.2]. Mutual-information penalties (MID, BiDO), label smoothing with a smoothing factor that can also be negative, an anti-adversarial term (AT-AT), and scaling the last-stage skip connections (SSF). Other methods are transfer learning that fine-tunes only the last layers, GAN-based misleading fine-tuning, low-rank feature filtering (LoFt), trigger injection (Trap-MID) and training-data de-duplication. The survey gives no common benchmark and no numerical comparison of defenses, and no certified-robustness guarantees exist for them [arxiv:2402.04013 §6, §8]. The ArcFace card does not study defenses. ## When to use which | Situation | Use | Why | |---|---|---| | Want a representative image per training class, model has BatchNorm layers, no data available | Closed-set inversion with the training loss and the BN prior | Needs only weights, labels and BN buffers; best FID and cosine similarity in the reported setup [arxiv:1801.07698 tab. 10] | | Want a face from an arbitrary embedding (e.g. probing what a stored template reveals) | Open-set inversion with an L2 embedding loss | Works for any feature vector, at a small quality cost [arxiv:1801.07698 §3.3, tab. 11] | | Model has no BatchNorm layers | A generative prior, or pixel regularizers alone | The BN prior in [arxiv:1801.07698 eq. 8] reads stored BN statistics; no card here compares the alternatives | | Only labels or scores are visible through an API | Black-box families (surrogate, label-only gradients, genetic, RL) | Gradients are not available; expect many queries [arxiv:2402.04013 §4.1.4] | | Public data resembling the private data exists and higher resolution is needed | Generative inversion with a pre-trained StyleGAN or BigGAN | Pre-trained generators allow higher resolution than early public-data GANs [arxiv:2402.04013 §4.1.2] | | Choosing which face model to invert | A margin-trained model (ArcFace) if available | Softmax-trained models give the worst inversions in [arxiv:1801.07698 tab. 10, tab. 11] | | Evaluating inversion quality | An independent recognition model plus FID | The inverting model rates its own outputs much higher (97.75% vs real 99.81%) than an independent model does [arxiv:1801.07698 §4.5]; accuracy depends on the evaluator, so scores are not comparable across papers [arxiv:2402.04013 §4.1.6] | | Tight compute budget | Expect a high cost | 20K Adam steps per batch of 256 in the ArcFace setup [arxiv:1801.07698 §4.5]; generative attacks also take time [arxiv:2402.04013 §4.1.2] | | Protecting a model served behind an API | Output processing (§6.1 of the survey) | Targets exactly the black-box setting; does not help offline models [arxiv:2402.04013 §6.1] | | Protecting a model you train | Robust model learning (§6.2 of the survey) | Changes training itself; no common benchmark compares the options [arxiv:2402.04013 §6.2] | ## Implementation sketch Common core of the variants above. The ArcFace variant uses pixels as $z$ and no generator; generator variants map latents through $G$. ```pseudocode # Model inversion, common core [arxiv:2402.04013 eq. 3] [arxiv:1801.07698 §3.3] # Input: frozen model f, target t (class label or embedding), steps T, # optional generator G (None for pixel-space inversion) # Output: reconstructed input x* def invert(f, t, T, G=None): # Start from Gaussian noise: pixels, or latents when G is given # [arxiv:1801.07698 §3.3] [arxiv:2402.04013 §4.1.1] z = sample_gaussian(batch_shape) for step in range(T): # Generator variants map latents to images first [arxiv:2402.04013 §4.1.1] x = G(z) if G is not None else z # Task loss: training loss for a class target, L2 for an embedding # [arxiv:1801.07698 §3.3] loss = task_loss(f, x, t) # Prior: BN-statistics match (eq. 8), generator prior, or none # [arxiv:1801.07698 eq. 8] [arxiv:2402.04013 §4.1.4] loss = loss + lam * prior(f, x) # Pixel variant also penalizes total variation and L2 norm [arxiv:1801.07698 §4.5] if G is None: loss = loss + a * tv(x) + b * l2(x) # Optimizer step: Adam in the ArcFace setup, Adam or momentum SGD # in the survey's white-box attacks [arxiv:2402.04013 §4.1.4] z = optimizer_step(z, loss) # Generator variants augment, re-score and keep the best [arxiv:2402.04013 §4.1.5] return select_best(G(z), f, t) if G is not None else z ``` Notes: - Reported ArcFace values: $\lambda = 0.05$, Adam with learning rate 0.25, $T$ = 20K, TV 1e-3, L2 1e-4, $B$ = 256 [arxiv:1801.07698 §3.3, §4.5]. - [arxiv:1801.07698 eq. 8] uses standard deviations, while BN buffers hold variances; take the square root [arxiv:1801.07698 §3.3]. - The ArcFace source does not say whether the batch statistics are taken at the BN input or output. The input is the natural match for the stored running statistics; this is our reading, not the paper's. - Running the model in eval mode keeps the stored BN statistics fixed. This is our note; neither source states it. - The survey states no training hyperparameters for any method, including $\lambda$, learning rates and step counts. Read each source paper for those values ([survey card](/papers/2402.04013)). Do not fill them in from ArcFace's values. - Loss choice changes gradient scale. Max-margin loss works on logits and cross-entropy on softmax outputs [arxiv:2402.04013 §4.1.4], so retune the step size after switching. ## Contradictions None known between the two cards on any shared claim. One discrepancy sits between a paper and its code. The survey's Poincaré loss [arxiv:2402.04013 eq. 4] is written with unit constants, while the text says to replace 1 with 0.9999, and the released PPA code applies that change to the target vector instead [arxiv:2402.04013 §4.1.4]. Implement the code's behaviour to reproduce the toolbox. ## Papers - [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](/papers/1801.07698): inverts face recognition models with the ArcFace loss plus a BatchNorm-statistics prior, in closed-set and open-set forms, and shows margin-trained models invert to better faces than softmax ones. - [Survey of model inversion attacks and defenses on DNNs (Fang et al.)](/papers/2402.04013): taxonomy of model inversion attacks by attacker access, modality and task, covering generative, black-box, text and graph methods, plus defense families; reports no new benchmark numbers.