---
kind: concept
slug: gan-inversion
title: GAN inversion
summary: >-
  Finds the GAN latent code that reproduces a real image. Routes: conditional-GAN encoders; per-image optimisation of StyleGAN codes and noise maps; pivotal tuning of the generator (under 3 minutes per face); a GAN texture prior in 3D face fitting; text or point edits on inverted faces. Trade-offs: training effort, per-image cost, fidelity against editability.
tags: [gan, image-editing, latent-space, encoder, stylegan, inpainting, text-driven-editing, 3d-face-reconstruction, interactive-editing]
papers: ["1611.06355", "1911.11544", "2103.17249", "2106.05744", "1902.05978", "2305.10973"]
related: [generative-adversarial-network, latent-space-editing, ddim-inversion, text-guided-image-editing, differentiable-rendering, face-recognition, dense-correspondence]
status: draft
updated_at: 2026-11-01
---

## Overview

A GAN generator turns a noise vector z into an image, but it has no encoder.
For a real photo you do not know which z would have produced it, so you cannot
reconstruct the photo or change one thing about it [arxiv:1611.06355 §1]
[§4.1]. Inversion means finding that missing map from image back to a code,
or a code plus a tuned generator, that reproduces the image
[arxiv:2106.05744 §1].

The inversion papers differ in what they train and what they optimise for
each image:

- a pair of encoders, trained once after the GAN [arxiv:1611.06355 §4]
- a per-image optimisation of a W+ code and per-layer noise maps, with the
  generator fixed [arxiv:1911.11544 §4] [§5]
- a per-image optimisation of a W code, then a short fine-tune of the whole
  generator [arxiv:2106.05744 §3]
- a per-image optimisation of a 512-value texture code inside a 3D face model,
  with shape, camera and lighting fitted in the same loop
  [arxiv:1902.05978 §3.1] [§3.4]

Two further papers use a GAN after an inversion step. StyleCLIP edits by text.
For a real photo it first needs an inverted code, which it obtains with e4e
[arxiv:2103.17249 §4]. DragGAN edits by dragging points, and for a real photo
it first inverts the image with PTI [arxiv:2305.10973 §4.1].

Every route that edits ends with the same kind of step: change a code, or a
region of it, and decode it again. The exceptions are StyleCLIP's global
direction, which edits a style code without an input photo, and GANFIT, which
fits a 3D face and does not describe an edit [arxiv:1902.05978 §3].

## Variants

### Two encoders after a conditional GAN (IcGAN)

The generator is a conditional GAN with labels concatenated to the noise. Its
training follows DCGAN and uses the matching-aware discriminator of Reed et al.
[arxiv:1611.06355 §4.3]. Once it is trained, two encoders are fitted:

- **E_z** maps an image to z. It is trained on images the generator produced
  from known z and labels, with a squared loss [arxiv:1611.06355 §4.1] [eq. 3].
- **E_y** maps an image to its attribute vector. It is trained on real images,
  because the authors found that generated images gave noisier results
  [arxiv:1611.06355 §4.1] [eq. 4].

Three ways of coupling the two encoders were compared [arxiv:1611.06355 §4.1]:

- **SNG**: one network with shared layers and two output heads.
- **IND**: two independent networks. This was the best option in the paper.
- **IND-COND**: two networks, with the z-encoder also taking the output of the
  y-encoder as input.

On 150K generated CelebA test images, the reconstruction loss L_e was 0.429
for IND, 0.432 for IND-COND and 0.500 for SNG [arxiv:1611.06355 §5.3]. The
paper also shows IND reconstructions visually [arxiv:1611.06355 fig. 4a].

- **Reconstruction:** encode the image to (z, y) and decode with the same y
  [arxiv:1611.06355 §4].
- **Editing:** keep the z from the encoder and replace y with the target labels
  [arxiv:1611.06355 §5.4]. The paper reports that z tends to hold pose,
  illumination and background, and y holds distinctive face features
  [arxiv:1611.06355 §5.4] [fig. 6b].
- **Qualitative results:** on CelebA, reconstructions generally show the
  requested attribute. Faces in unusual conditions, such as looking away or
  off-centre, are noisier, and a moustache is often not generated, especially
  for women [arxiv:1611.06355 §5.4] [fig. 5b]. On MNIST, the handwriting style
  of a real digit carries over to other digits [arxiv:1611.06355 §5.4]
  [fig. 5a].

### Optimised W+ code with noise maps (Image2StyleGAN++)

The generator is a pretrained StyleGAN whose weights stay fixed
[arxiv:1911.11544 §5]. The image is embedded in two stages. First a W+ code is
optimised. Then per-layer noise maps are optimised with W+ held fixed.
Optimising both at once gives high-frequency artifacts [arxiv:1911.11544 §4.2]
[fig. 2]. The reported schedule is 5000 W+ steps and 3000 noise steps, with a
mean face code as the start for faces and a random code otherwise
[arxiv:1911.11544 §7.1].

Reconstruction reaches 39 to 45 dB PSNR, against about 20 dB for the earlier
W+-only method. Both numbers are author-reported [arxiv:1911.11544 §1]
[§7.1] [fig. 4]. The same optimiser supports local edits:

- **Masked embedding:** the loss covers only part of the image, so a missing
  region can be filled in, or a rough scribble can steer a region
  [arxiv:1911.11544 §7.3] [§7.4].
- **Activation mixing:** the GAN's intermediate activations are copied from one
  embedding into another under two masks. This gives edits, such as swapping
  features, that blend into the rest of the photo [arxiv:1911.11544 §5]
  [eq. 5] [§7.6].

For inpainting at 256, 512 and 1024 px, the paper reports better SSIM, MSE and
PSNR than partial and gated convolution [arxiv:1911.11544 tab. 1]. The
evaluation uses 10 ground-truth images and 7 masks [arxiv:1911.11544 §9.1].

### Pivotal tuning (PTI)

The pivot is a code in StyleGAN2's native W space, found with the generator
frozen. The paper uses W rather than W+, because the tuning step that follows
removes the distortion [arxiv:2106.05744 §3.1]. The inversion optimises w and
the noise maps against an LPIPS loss, and only w is kept [eq. 1].

Then the whole generator is tuned for up to 350 steps, so that the fixed pivot
reproduces the input image [arxiv:2106.05744 §3.2] [§4.5]. A locality term keeps
the tuned generator close to the original on random codes near the pivot,
which limits side effects on other faces [arxiv:2106.05744 §3.3] [eq. 7]. Edits
then run on the tuned generator, with existing edit directions and
StyleCLIP [arxiv:2106.05744 §4.2].

On CelebA-HQ the paper reports LPIPS 0.09 against 0.34 for the best baseline,
and identity similarity 0.9 against 0.85 [arxiv:2106.05744 tab. 1]. Both are
author-reported. Tuning takes under 3 minutes per identity on one RTX 2080
[arxiv:2106.05744 §4.5] [§5].

### GAN texture prior in a 3D face model (GANFIT)

GANFIT fits a 3D morphable face to one photo, and the GAN supplies only the
skin texture [arxiv:1902.05978 §3.1]. The GAN is a progressive-growing network
trained on about 10,000 UV textures at 512 × 512. Its 512-value latent is one of
the parameters that the fit optimises [arxiv:1902.05978 §3.1] [§4.1] [eq. 5].
The shape model has 158 identity and 29 expression bases [arxiv:1902.05978
§4.1]. Camera (7 values) and lighting (9 values) are optimised too
[arxiv:1902.05978 §3.2].

The fit runs in two stages. The first stage uses the landmark loss alone and
moves shape, expression and camera so the mesh overlays the face. The second
stage optimises all parameters with a weighted sum of an identity loss on a
face-recognition network, a content loss on its intermediate layers, a
full-resolution pixel loss, a landmark loss and a regulariser [arxiv:1902.05978
§3.4] [eq. 12]. The reported weights are 2.0 for identity, 50.0 for content,
1.0 for pixels and 0.001 for landmarks, with Adam at learning rate 0.01
[arxiv:1902.05978 §4.1].

On the MICC Florence benchmark, the author-reported mean error is 0.94 to 0.95
for the method, against 1.48 to 1.50 for the best baseline in each setting
[arxiv:1902.05978 tab. 1]. Fitting takes about 30 s per image on a GTX 1080 Ti
[arxiv:1902.05978 §4.1]. The card records that the released repository holds
only the MICC evaluation scripts, so the hyperparameters come from the paper
alone; see [the card](/papers/1902.05978).

### Text-driven edits on a real face (StyleCLIP)

StyleCLIP is not an inversion method. It edits StyleGAN2 images from a text
prompt, scoring each result with the CLIP cosine distance to the prompt
[arxiv:2103.17249 §4] [eq. 1]. Real photos are first inverted with e4e
[arxiv:2103.17249 §4] [§5] [§6]. e4e is a separate encoder, and this wiki has no
card for it: [Tov et al., e4e](https://arxiv.org/abs/2102.02766). The three
routes below therefore take their input codes from a step outside this page.

All three routes use a frozen StyleGAN2 generator [arxiv:2103.17249 §4 fn. 1].

- **Per-image latent optimisation.** Gradient descent on the W+ code of one
  photo, for each prompt, for 200 to 300 iterations [arxiv:2103.17249 §4]. The
  loss adds an L2 penalty to the source code and an optional identity term
  [eq. 1] [eq. 2]. The table reports 98 s of inference per edit on one GTX
  1080Ti [tab. 1]. The text says "several minutes" [§4] [§5]. The authors call
  the method sensitive to its parameters and leave it out of the quantitative
  comparison [§5] [§7].
- **Latent mapper, one network per prompt.** Three small networks, one for each
  group of W+ layers (coarse, medium and fine), output a step that is added to
  the input code [arxiv:2103.17249 §5] [eq. 3]. Training takes 10 to 12 h, and
  one edit then takes 75 ms on a GTX 1080Ti [tab. 1]. The authors report that it
  does better than the global direction on identity-linked prompts such as
  "Trump" [§7] [fig. 10].
- **Global direction in style space S.** One direction per prompt, found from
  per-channel CLIP relevance [arxiv:2103.17249 §6] [eq. 6] [eq. 7]. It does not
  depend on the input image. Pre-processing takes 4 h, and an edit takes 72 ms
  [tab. 1]. The edit moves the style code to G(s + αΔs), so α is a strength
  dial [§6]. The authors report that it removes wrinkles where the mapper does
  not [§7] [fig. 10].

### Point dragging after inversion (DragGAN)

DragGAN is an editing method, not an inversion method. The user places handle
points on a StyleGAN2 image and gives a target for each. The optimisation then
moves each handle toward its target [arxiv:2305.10973 §3]. It takes one
optimiser step on the first six layers of the W+ code per iteration. The loss
pushes the block-6 features near each handle toward the target
[arxiv:2305.10973 §3.2] [eq. 1]. After each step, every handle is moved to the
nearest feature match in a small square around it [arxiv:2305.10973 §3.3]
[eq. 2]. No tracking network is used. The loop stops when each handle is within
a tolerance of its target, and usually takes 30 to 200 iterations
[arxiv:2305.10973 §3.1] [§3.4].

For a real photo, the image is first inverted with PTI, and then the same loop
runs [arxiv:2305.10973 §4.1]. The paper gives no inversion settings beyond the
citation [arxiv:2305.10973 §4.1] [fig. 5], so results depend on the inversion. On face landmarks,
the author-reported mean distance for one point is 2.44, against 11.64 for
UserControllableLT [arxiv:2305.10973 tab. 1]. The unit of that distance is not
stated [arxiv:2305.10973 §4.2].

## When to use which

The constraints below come from each card's setup. The cards time different
hardware, or do not state the hardware, so run times are not compared across
variants.

| Situation | Use | Why |
|---|---|---|
| You can train a conditional GAN with labels, and the edit is an attribute change | Two encoders (IcGAN) | Editing changes the labels and keeps z [arxiv:1611.06355 §5.4]. IND had the lowest reported loss [arxiv:1611.06355 §5.3]. |
| The generator must stay frozen, for example a shared pretrained StyleGAN | Optimised W+ with noise maps | The method assumes fixed weights and uses the noise inputs [arxiv:1911.11544 §5] [§6]. |
| You need inpainting, scribble-guided or region-only edits | Optimised W+ with masks | Masks restrict the embedding to a region [arxiv:1911.11544 §7.3] [§7.4], and tab. 1 reports the inpainting comparison. |
| You need the closest copy of one face that still responds to edits | PTI | LPIPS and identity scores are better than the reported baselines [arxiv:2106.05744 tab. 1]. |
| You need 3D shape and skin texture from one unconstrained photo, and can afford about 30 s per image | GANFIT | Shape, texture latent, camera and lighting are fitted together against identity and content losses [arxiv:1902.05978 §3.4] [§4.1]. The lowest reported shape error on MICC is author-reported [arxiv:1902.05978 tab. 1]. See [the card](/papers/1902.05978) for what is not released. |
| You need text-described edits on a real face, and can train once per prompt | StyleCLIP latent mapper | Edits take 75 ms after 10 to 12 h of training on a GTX 1080Ti [arxiv:2103.17249 tab. 1]. The card does not time the e4e inversion that a real photo also needs. |
| You need text-described edits with one shared direction for many photos | StyleCLIP global direction | The step does not depend on the input image [arxiv:2103.17249 §6] [tab. 1]. The paper gives no default for the threshold β, and the figures use values from 0.11 to 0.16 [§6] [fig. 6]. The card records failures on large domain gaps [app. B]. |
| You need text-described edits without training, and can wait about 98 s per edit | StyleCLIP per-image optimisation | No training is needed [arxiv:2103.17249 tab. 1]. The authors call the method sensitive to its parameters [§5]. |
| You need to move points on a face or scene image to targets, and can wait about 2 s per edit | DragGAN | No tracking network is needed [arxiv:2305.10973 §3]. One-point landmark distance is 2.44 against 11.64 for UserControllableLT [arxiv:2305.10973 tab. 1]. A real photo needs PTI first [§4.1]. |
| You need many identities in one generator, or cannot store a generator per identity | Not PTI without testing | The regulariser is tested with 12 identities [arxiv:2106.05744 §4.3], and tuning time grows linearly with identities [§4.5]. The released code saves one full generator per identity; see [the card](/papers/2106.05744). |
| You need a result in well under a second per image | StyleCLIP mapper or global direction, after their setup | Their edit times are 75 ms and 72 ms [arxiv:2103.17249 tab. 1]. The three inversion routes run hundreds to thousands of steps per image [arxiv:1911.11544 §7.1] [arxiv:2106.05744 §4.5], and IcGAN's per-image time is not in the card. |
| You need high resolution | Image2StyleGAN++ for inpainting | Inpainting is tested at up to 1024 px [arxiv:1911.11544 tab. 1]. IcGAN's experiments are at 64×64 [arxiv:1611.06355 §4.3] [§5.1]. The PTI card does not state a resolution. |
| You need inversion of a diffusion model | Not covered here | See [DDIM inversion](/concepts/ddim-inversion) for that family. |

Costs the cards record:

- IcGAN: the GAN and both encoders are trained in sequence, and the encoders
  need labelled real images for E_y [arxiv:1611.06355 §4.1]. The paper trains
  on CelebA with 18 of 40 attributes [arxiv:1611.06355 §5.1].
- Image2StyleGAN++: a per-image optimisation over a frozen generator, run on
  one 32 GB V100. The run time per image is not stated [arxiv:1911.11544 §6].
- PTI: a full copy of the generator per identity, and a tuning run per image
  [arxiv:2106.05744 §4.5].
- GANFIT: about 30 s per image on one GTX 1080 Ti [arxiv:1902.05978 §4.1]. The
  GAN's own training time is not stated [arxiv:1902.05978 §4.1].
- StyleCLIP: the mapper trains for 10 to 12 h, the global direction needs 4 h of
  pre-processing, and the per-image route runs 200 to 300 steps per edit
  [arxiv:2103.17249 tab. 1] [§4]. The card's timings are on one GTX 1080Ti. The
  e4e inversion is not costed.
- DragGAN: 2.0 s per edit for one point [arxiv:2305.10973 tab. 1]. The hardware
  for those times is not stated, although the paper names a single RTX 3090
  [arxiv:2305.10973 §1].

## Implementation sketch

The common core is the same for every route: estimate a code, optionally tune
the generator, then edit the code and decode. The blocks below show the parts
where the variants differ.

```pseudocode
# Shared shape of all routes: image -> code -> (optional tuning) -> edit
def invert_and_edit(image, G, edit_fn, tune_generator):
    # Stage A: estimate a code for the real image  [see variants above]
    code = estimate_code(image, G)

    # Stage B (PTI only): tune G so the fixed code reproduces the image
    if tune_generator:
        G = pivotal_tune(G, code, image)  # [arxiv:2106.05744 §3.2]

    # Stage C: edit the code, or only a region of it, and decode
    # [arxiv:1611.06355 §5.4] [arxiv:1911.11544 §5]
    return G(edit_fn(code))
```

```pseudocode
# W+ then noise embedding, as in reconstruction  [arxiv:1911.11544 §7.1]
def embed_w_plus_then_noise(image, G):
    # Start from the mean face code, or a random code for non-faces  [§7.1]
    w = initial_w_plus(image)
    n = standard_normal_noise()  # noise maps, N(0, I)  [§7.1]
    # Stage 1: W+ only, noise fixed. Perceptual + masked MSE  [§4.2] [eq. 2]
    w = optimise(w, steps=5000, loss=w_plus_loss(G, w, n, image))
    # Stage 2: noise only, W+ fixed. Masked MSE  [§4.2] [eq. 3]
    n = optimise(n, steps=3000, loss=noise_loss(G, w, n, image))
    return w, n
```

```pseudocode
# Pivotal tuning: invert into W, then tune G  [arxiv:2106.05744 §3]
def pivotal_tune(G_pre, w_pivot, image):
    # The pivot is fixed; only the generator weights change  [§3.2]
    G_t = copy(G_pre)
    for step in range(350):  # upper bound  [§4.5]
        # Reconstruct the image at the pivot  [eq. 2]
        loss = lpips(image, G_t(w_pivot)) + l2(image, G_t(w_pivot))
        # Locality: a random code at distance alpha = 30 from the pivot
        # (alpha and the weight 0.1 from [§4.5])
        w_r = random_code_at_distance(w_pivot, alpha=30)
        target = G_pre(w_r)  # original generator is the target  [eq. 5]
        loss += 0.1 * (lpips(target, G_t(w_r)) + l2(target, G_t(w_r)))
        G_t = adam_step(G_t, loss, lr=3e-4)  # learning rate  [§4.5]
    return G_t
```

Notes on where the variants differ:

- IcGAN has no per-image loop. After training, each new image needs only the
  two encoder passes [arxiv:1611.06355 §4.1].
- The optimisation routes run a loop for each image. Image2StyleGAN++ keeps the
  noise maps and PTI discards them after inversion [arxiv:1911.11544 §4]
  [arxiv:2106.05744 §3.1].
- GANFIT's Stage A is a joint fit, not an encoder pass. A landmark-only stage
  comes first, then shape, camera, lighting and the texture latent are updated
  together [arxiv:1902.05978 §3.4]. Its output is a 3D face, so there is no
  Stage C edit.
- DragGAN replaces Stage C with an iterative loop. It optimises the first six
  W+ layers, re-locates each handle, and repeats until the handles reach their
  targets [arxiv:2305.10973 §3.1] [§3.3]. For a real photo, Stage B is PTI
  [arxiv:2305.10973 §4.1].
- StyleCLIP's global direction has no Stage A. It edits a style code as
  G(s + αΔs) and does not depend on the input image [arxiv:2103.17249 §6]
  [tab. 1]. Its per-image and mapper routes start from an e4e code for the
  photo, as described in its variant above.
- The locality term is part of the method [arxiv:2106.05744 §3.3], but the
  released code leaves it switched off by default. The card lists this and
  other code-versus-paper differences; see [the card](/papers/2106.05744).

## Contradictions

The IcGAN paper contains three internal inconsistencies. No other card covers
the same results, so they cannot be checked against another source:

- The coupling of the two encoders is named IND-COND in its definition
  [arxiv:1611.06355 §4.1] but IND-CND in the results text [arxiv:1611.06355
  §5.3].
- The text reports an overall discriminator accuracy near 86% [arxiv:1611.06355
  §5.2], while the best generator accuracy in the table is 89.83% [arxiv:1611.06355
  tab. 3].
- Two generator rows in the table (layers 2 and 3) both show 82.49%
  [arxiv:1611.06355 tab. 3]. This may be a typo.

The paper says encoder training uses the same optimiser as the GAN
[arxiv:1611.06355 §4.3]. The card's Paper vs code section notes that the
released code uses different encoder optimiser settings. See
[the card](/papers/1611.06355) for the details.

Across papers, the optimisation routes disagree about the same step:

- Image2StyleGAN++ says optimising W+ and noise maps together gives
  high-frequency artifacts, because the perceptual loss does not suit the
  noise maps [arxiv:1911.11544 §4.2] [fig. 2].
- PTI optimises w and the noise maps together in its inversion, against an
  LPIPS loss [arxiv:2106.05744 §3.1] [eq. 1]. It then keeps only w.
- GANFIT fits its texture latent jointly with shape, camera and lighting, after
  a landmark-only stage [arxiv:1902.05978 §3.4]. It has no noise maps. Its
  ablation covers the loss terms, not the stages [arxiv:1902.05978 §4.4], so it
  does not test the staging question.

The papers do not settle this. They also pick different starting spaces. PTI
says W+ reduces distortion but weakens edits [arxiv:2106.05744 §1] [§2]. Its
table gives an edit magnitude of 11.15 for W+ against 14.86 for its own method
[arxiv:2106.05744 tab. 2], with units not stated. Image2StyleGAN++ builds its
embedding on W+ [arxiv:1911.11544 §4], and its card reports no edit
comparison against W. StyleCLIP also edits in W+ and says learned directions
there are harder to disentangle [arxiv:2103.17249 §3]. Its card reports no
comparison against W.

The reconstruction numbers are not comparable. Image2StyleGAN++ reports PSNR
in dB [arxiv:1911.11544 §1] [fig. 4]. PTI reports LPIPS, MSE, MS-SSIM and
identity similarity [arxiv:2106.05744 tab. 1]. GANFIT reports a point-to-plane
shape error in millimetres on 3D scans [arxiv:1902.05978 tab. 1], and DragGAN
reports a landmark distance whose unit is not stated [arxiv:2305.10973 tab. 1].
None of these measures a 2D reconstruction on the same scale.

PTI's table marks MS-SSIM with a down arrow, but its own value is the one the
caption calls best. The card treats this row as unclear [arxiv:2106.05744
tab. 1]. It is not a disagreement between papers.

StyleCLIP's card gives two times for its per-image route: "several minutes" in
the text [arxiv:2103.17249 §4] [§5], and 98 s in its table [tab. 1]. This page
quotes the table, and the text figure is not resolved.

## Papers

- [Invertible Conditional GANs for image editing](/papers/1611.06355): trains a
  conditional GAN, then two encoders for noise and attributes, to reconstruct
  and edit real face images.
- [Image2StyleGAN++: How to Edit the Embedded Images?](/papers/1911.11544):
  embeds a photo into StyleGAN's W+ space and then into noise maps, with masks
  for local edits and inpainting.
- [Pivotal Tuning for Latent-based Editing of Real Images](/papers/2106.05744):
  inverts a face into W, then fine-tunes StyleGAN2 so the code reproduces it,
  with a locality term.
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](/papers/1902.05978):
  fits a 3D face model to one photo, with a GAN supplying the skin texture and
  a face-recognition network the identity loss.
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](/papers/2103.17249):
  edits StyleGAN2 images from text prompts, by per-image optimisation, a
  per-prompt latent mapper, or a global direction in style space.
- [Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold](/papers/2305.10973):
  moves handle points of a StyleGAN2 image to targets by optimising the latent
  code, with nearest-neighbour point tracking and no tracker network.