---
kind: concept
slug: latent-space-editing
title: Latent-space editing
summary: >-
  Edits an image by moving its code in a pretrained generator's latent space, then decoding. Ten cards cover trained encoders, flows, CLIP-guided optimisation, per-photo tuning, noise-map embedding, face-age autoencoders, StyleCLIP, point dragging and motion. Trade-offs: labels and training versus minutes per image. Author-reported numbers.
tags: [image-editing, latent-space, generative-models, gan-inversion]
papers: ["1611.06355", "2008.02401", "2204.08583", "2108.00946", "1911.11544", "2106.05744", "2203.08063", "1702.08423", "2103.17249", "2305.10973"]
related: [generative-adversarial-network, gan-inversion, text-guided-image-editing, ddim-inversion, image-to-image-translation, vision-language-pretraining, diffusion-models, domain-adaptation, normalizing-flows, neural-style-transfer, disentangled-representation, variational-autoencoder, dense-correspondence]
status: draft
updated_at: 2026-11-01
---

## Overview

Editing in latent space means three steps: find a code that reproduces the
input, move the code, and decode it again. The hard part is the middle step.
Naive vector arithmetic in a GAN's latent space can change the target attribute,
but it also pushes the code away from the codes the decoder was trained on and
can change the person's identity [arxiv:2008.02401 §1].

The ten covered papers take different routes. Eight edit images, one
works on human motion, which uses the same kind of latent space, and one
trains a generator from text alone rather than editing a given image:

- A trained encoder pair on a conditional GAN. Two encoders, learned after the
  GAN, map a real image to its noise vector and its attribute vector. Changing
  the attribute vector edits the image [arxiv:1611.06355 §4] [§4.1].
- A learned map on a pretrained GAN's latent. A normalizing flow, conditioned on
  attributes, decides where an edited code should go [arxiv:2008.02401 §3].
- Optimisation in a pretrained decoder's latent grid, with no training. A joint
  image-text model scores the decoded image against a prompt, and the grid moves
  to raise the score [arxiv:2204.08583 §1] [§2].
- Changing the generator instead of the code. A pretrained GAN is fine-tuned
  from two text prompts alone, so the new generator makes the target domain.
  The paper claims the latent space stays the source's, so latent-space methods
  still apply [arxiv:2108.00946 §4.1] [§5.2].
- Tuning the generator to one photo. The photo is inverted into StyleGAN2's W
  space, then the whole generator is fine-tuned so that this fixed code
  reproduces the photo [arxiv:2106.05744 §3.1] [§3.2].
- Optimising noise as well as the latent. A StyleGAN photo is embedded into W+,
  then into per-layer noise maps. Masks limit the embedding to one region, so
  local edits and fills are possible [arxiv:1911.11544 §4] [§5].
- A conditional autoencoder for face age. An encoder gives a short identity
  code, and a generator renders it at a chosen age bin. Changing the age label
  with the code held fixed changes the age [arxiv:1702.08423 §4.1] [§5.2].
- Text prompts on a pretrained StyleGAN2. CLIP's cosine distance to a prompt
  steers the generator through three routes: per-image latent optimisation, a
  small per-prompt mapper, and one global direction in style space
  [arxiv:2103.17249 §4] [§5] [§6].
- Dragging handle points on a StyleGAN2 image. Each step optimises the first
  six W+ layers so that features under each handle move toward its target. A
  nearest-neighbour search then re-finds each handle, with no tracking network
  [arxiv:2305.10973 §3].
- A motion model rather than an image model. A transformer auto-encoder on
  human motion is aligned to CLIP's text and image spaces, so a text prompt can
  be decoded directly into a motion [arxiv:2203.08063 §3] [§4.2].

## Variants

### Trained encoder pair on a conditional GAN (IcGAN)

The method adds a learned inverse to a conditional GAN. The GAN is trained
first. Then E_z maps an image to the noise vector z, and E_y maps a real image
to its attribute vector y [arxiv:1611.06355 §4.1]. Editing keeps z and replaces
y, so the generator redraws the same photo with other attributes
[arxiv:1611.06355 §5.4]. Inversion is one forward pass through each encoder, so
no per-image optimisation is needed [arxiv:1611.06355 §4.1].

- **Training:** the cGAN follows DCGAN. It uses Adam at learning rate 0.0002,
  β1 0.5, batch size 64 and 25 epochs [arxiv:1611.06355 §4.3]. The paper's
  encoder optimiser is the same as the cGAN's, but the released code uses other
  values; see the card's Paper vs code section.
- **Encoder design:** of three ways to couple the two encoders, two independent
  networks (IND) gave the lowest reconstruction loss L_e on 150K generated CelebA
  test images: 0.429, against 0.432 for IND-COND and 0.500 for a shared-layer
  encoder [arxiv:1611.06355 §5.3].
- **Where the label enters:** the attribute vector goes in at the generator's
  input and after the first convolution of the discriminator. The authors chose
  these positions from their comparison of insertion points [arxiv:1611.06355
  §5.2] [tab. 3].
- **Labels for E_y:** E_y is trained on real images, not generated ones, because
  generated images gave noisier attribute estimates [arxiv:1611.06355 §4.1].
- **Reported results:** edits are judged qualitatively. Reconstructions generally
  show the requested attribute, and unusual poses are noisier [arxiv:1611.06355
  §5.4]. The generator table reports attribute accuracy of 74% to 90%, measured
  with a classifier the paper describes only by architecture [arxiv:1611.06355
  §5.2] [tab. 3].

Card: [IcGAN](/papers/1611.06355). The card's Paper vs code section lists the
differences between the paper and the released code, including the encoder
optimiser and the loss normalisation.

### Attribute-conditioned flow on StyleGAN W (StyleFlow)

The method fits one conditional continuous normalizing flow between StyleGAN's W
codes and a standard normal. The attributes enter the flow's velocity field, so
one network covers all attributes jointly [arxiv:2008.02401 §5.1] [§5.2]. The
training set is 10k generated samples, labelled with an attribute classifier
[arxiv:2008.02401 §5.1]. The paper trains with Adam at learning rate 1e-3 and
batch size 5 for 10 epochs [arxiv:2008.02401 §5.3].

- **Sampling:** draw noise, run the flow in reverse with the target attributes,
  and decode the resulting W code [arxiv:2008.02401 §6.1].
- **Editing:** project the image to W+, measure its attributes, invert the flow
  under the source attributes, and decode under the target attributes. Only the
  W+ rows that the edit covers are replaced; the rest come from the source
  [arxiv:2008.02401 §6.2] [§6.2.3].
- **Sequential edits:** a fast mode skips re-projection after each edit and the
  paper says it can produce sudden jumps. An accurate mode re-projects all 18
  vectors after every edit [arxiv:2008.02401 §6.2].
- **Reported results:** on sequential pose, light and expression edits of
  generated images, FID is 53.15 for this method, against 64.69 for GANSpace,
  67.08 for InterFaceGAN and 82.49 for Image2StyleGAN [arxiv:2008.02401 tab. 1].
  Baselines were partly retrained by the authors [arxiv:2008.02401 §7.3].
- **Cost:** inference is 0.61 s with four flow blocks and 0.21 s with six, on a
  Titan XP. The paper does not say what one timed call covers
  [arxiv:2008.02401 §5.3] [tab. 4].

Card: [StyleFlow](/papers/2008.02401).

### CLIP-guided latent optimisation (VQGAN-CLIP)

A pretrained VQGAN decoder turns a grid of codebook vectors into an image. CLIP
embeds the image's augmented crops and the prompt, and the loss is a squared
spherical distance between them [arxiv:2204.08583 §2] [§2.2] [§2.3]. Each
iteration decodes the grid, averages the loss over many random crops, and takes
one Adam step on the grid. The paper uses learning rate 0.15 and 400 iterations
[arxiv:2204.08583 §2].

- **Editing:** the start point is the encoded input image instead of random
  codes. The paper's only editing comparison with Open-Edit is visual, with no
  metric reported [arxiv:2204.08583 §2] [§4.1].
- **Regularisation:** an optional L2 penalty on the latent is described for a
  regularised variant. The paper gives no numeric weight, and names a decay of
  0.005 that the released repository does not reproduce [arxiv:2204.08583 §2.4].
- **Reported results:** a human study on 12 prompts gave a mean of 4.6 out of 5,
  against 3.3 for the best GLIDE baseline. The baselines were picked best of five,
  this method was best of one [arxiv:2204.08583 tab. 1] [§3.2].
- **Cost:** no training, so 0 V100-hours [arxiv:2204.08583 tab. 2]. Runtime is
  188.3 s per image on a V100 and 654.3 s on a P100 [arxiv:2204.08583 tab. 2].
  A Colab run takes under 3 minutes and under 11 GB of GPU memory
  [arxiv:2204.08583 §6.1].

Card: [VQGAN-CLIP](/papers/2204.08583). The card's Paper vs code section lists
where the released defaults differ from the paper's values.

### Text-only generator adaptation (StyleGAN-NADA)

This route does not edit a given image. It retrains a copy of a pretrained
StyleGAN2 to draw a new domain. A frozen copy supplies the source images, and
the trained copy is pushed so that its change in CLIP space points along the
text direction from source to target prompt. The loss is one minus the cosine
between the two directions [arxiv:2108.00946 §4.1] [eq. 2]. For shape changes,
only a few layers are updated per iteration, chosen by a short latent
optimisation [arxiv:2108.00946 §4.2].

- **Training cost:** texture-type changes take about 300 iterations, roughly
  3 minutes on one V100. Animal changes take 2000 iterations plus a latent
  mapper, about 6 hours in total [arxiv:2108.00946 app. I].
- **Reuse of latent-space tools:** the paper says inversion, latent-direction
  editing and conditional encoders built for the source generator work on the
  adapted one without retraining [arxiv:2108.00946 §5.2] [fig. 8] [fig. 9]
  [fig. 10]. This is the link to the other variants.
- **Reported results:** dog to cat FID with 10 target images is 41.22 with
  zero-shot pre-training, against 45.13 for the best few-shot baseline shown
  [arxiv:2108.00946 tab. 2]. A user study is reported in the same paper
  [§5.1] [fig. 12]. These are author-reported.

Card: [StyleGAN-NADA](/papers/2108.00946). The card's Paper vs code section
lists where the released defaults differ from the paper's values, including
the layer-selection and mapper settings.

### Per-photo generator tuning (PTI)

The method inverts a real face into StyleGAN2's native W space with the
generator frozen. That code, the pivot, is then held fixed while the whole
generator is fine-tuned from its pretrained weights until the pivot reproduces
the photo [arxiv:2106.05744 §3.1] [§3.2]. The paper runs 450 inversion steps and
up to 350 tuning steps [arxiv:2106.05744 §4.5]. Edits then run on the tuned
copy.

- **Locality term:** each tuning step also samples a random code, moves the
  pivot a fixed distance towards it, and asks the tuned generator to match the
  original there. The weight is 0.1 and the distance is 30 [arxiv:2106.05744
  §3.3] [§4.5]. In the paper's sweep, a distance of 2 gives little
  regularisation and 120 gives unrealistic images [arxiv:2106.05744 app.
  Locality Regularization].
- **Pivot quality:** random or mean codes as the pivot fail to converge, so the
  inversion step is what the method depends on [arxiv:2106.05744 §3.2] [§4.4].
- **Reported results:** on CelebA-HQ reconstruction, LPIPS is 0.09 against 0.34
  for the best baseline, SG2 W+. ID similarity is 0.9 against 0.85
  [arxiv:2106.05744 tab. 1]. For single edits of smile, pose and age, ID
  similarity is 0.9 against 0.85 [arxiv:2106.05744 tab. 2]. These are
  author-reported.
- **Cost:** under 3 minutes per identity on one RTX 2080, with inversion about
  1 minute [arxiv:2106.05744 §4.5] [§5]. Each identity keeps its own full copy
  of the generator, so storage grows with the number of identities. The card's
  Pitfalls section gives the details.
- **Scope:** the method is shown only for StyleGAN2 on faces. Other generators
  are named as future work [arxiv:2106.05744 §5].

Card: [PTI](/papers/2106.05744). The card's Paper vs code section lists where
the released code differs from the paper, including the inversion learning rate
and whether the locality term is on by default.

### Noise-map embedding with masks (Image2StyleGAN++)

The method embeds a photo in two stages. First it optimises a W+ code, with the
noise maps fixed. Then it optimises the per-layer noise maps, with the code
fixed. Optimising both together gives high-frequency artifacts
[arxiv:1911.11544 §4.2] [fig. 2]. The paper gives the noise maps the
high-frequency detail, and the W+ code the semantic content [arxiv:1911.11544
§4]. Masks confine the loss to one region, and copying
intermediate activations from a second embedding blends the two
[arxiv:1911.11544 §5] [§7].

- **Reconstruction:** the authors report 39 to 45 dB PSNR with both stages. The
  W+ stage alone gives 19 to 22 dB, against about 20 dB for the previous best
  [arxiv:1911.11544 §1] [§7.1] [fig. 4]. The full schedule is 5000 W+ steps
  and 3000 noise steps [arxiv:1911.11544 §7.1].
- **Local edits:** the paper applies the same operations to inpainting, two-photo
  crossover, scribble-guided edits and local style transfer [arxiv:1911.11544
  §7.2] [§7.3] [§7.4] [§7.5]. Its inpainting table reports SSIM, MSE and PSNR
  at 256, 512 and 1024 px. At 1024 px the method scores 0.9176 SSIM, against
  0.8957 for partial convolution and 0.8693 for gated convolution
  [arxiv:1911.11544 tab. 1].
- **Cost:** one 32 GB V100 [arxiv:1911.11544 §6]. The paper gives no run time
  per image.
- **Evaluation scale:** inpainting uses 10 ground-truth images and 7 masks, and
  the edit-realism test uses one face-attribute classifier [arxiv:1911.11544
  §9.1] [§9.3]. The paper's own limitations include lighting, age and skin
  colour [arxiv:1911.11544 §9.1].

Card: [Image2StyleGAN++](/papers/1911.11544). The card's Pitfalls section notes
that the number of pixels that the loss divides by is not defined in the paper.

### Motion latent space (MotionCLIP)

This variant is not an image editor. The paper trains a transformer
auto-encoder on SMPL human motion. Its latent vector is pushed to match CLIP's
embedding of the motion's text label and of one rendered frame, using cosine
distance [arxiv:2203.08063 §3]. CLIP stays frozen, so a text prompt can be
encoded by CLIP and decoded straight into a motion with no text-motion training
[arxiv:2203.08063 §4.2].

- **Latent operations:** the paper shows linear interpolation between codes, and
  arithmetic in which the CLIP vector of a style word is added to a content
  code. It also reports action recognition by comparing the code with the
  embeddings of 60 class names [arxiv:2203.08063 §4.3]. These latent results
  are qualitative.
- **Text-to-motion:** in user studies, MotionCLIP is preferred over JL2P on
  76.7% of in-domain prompts and 75.3% of out-of-domain prompts
  [arxiv:2203.08063 tab. 1]. The paper does not state the number of
  participants or samples.
- **Style transfer:** against Aberman et al., which takes motion inputs, the
  average preference for MotionCLIP is 37.9% [arxiv:2203.08063 tab. 2].
- **Action recognition:** top-1 is 40.9% against 41.14% for 2s-AGCN, while top-5
  is 57.71% against 73.18% [arxiv:2203.08063 tab. 3]. Without the text loss,
  top-1 falls to 4.54% [arxiv:2203.08063 tab. 3].

Card: [MotionCLIP](/papers/2203.08063). The card's Paper vs code section lists
differences from the released code, including the loss weights. Its data
licences are not verified and need checking before use.

### Conditional adversarial autoencoder for face age (CAAE)

The method learns from faces without paired photos of one person at different
ages. An encoder maps a 128 x 128 face to a vector z, and a generator renders
the vector with an age label appended [arxiv:1702.08423 §4.1]. The age label is
one-hot over 10 bins [arxiv:1702.08423 §5.1]. A discriminator on z pushes the
codes toward a uniform prior, and an image discriminator judges realism
conditioned on the age label [§4.3] [§4.4]. At test time only the encoder and
generator run, and the input's true age is not needed [§5.2]. Keeping z fixed
and changing only the age label is the editing step [§3] [fig. 2].

- **Training data:** about 10,670 images from Morph, CACD and crawled sources,
  with ages and genders estimated from captions or an age estimator
  [arxiv:1702.08423 §5.1]. The paper sets lambda = 100 and gamma = 10 on the
  reconstruction and total-variation terms [§5.2]. The text does not state the
  latent size; the figure shows 50 [fig. 3].
- **Reported results:** a user study, not a recognition score. Against prior
  work, 52.77% of votes favour CAAE, 28.99% favour the prior work and 18.24%
  judge them equal [arxiv:1702.08423 §5.3]. Against ground truth, 48.38% judge
  the output the same person [§5.3]. The paper reports no measure of whether
  the requested age appears.
- **Vote counts:** the paper reports 3,208 votes from 63 volunteers, who
  answered 45 questions each. That allows at most 2,835 votes, and the paper
  does not explain the gap [arxiv:1702.08423 §5.3].
- **Cost:** not stated in the paper. The repository README gives about 2.5
  hours for 50 epochs on UTKFace on one TITAN X [code: README.md#L34]. That run
  uses a dataset other than the paper's.

Card: [CAAE](/papers/1702.08423). The card's Paper vs code section lists the
differences from the released code. The repository has no LICENSE file, so
its code cannot be reused on that basis.

### Text-driven StyleGAN editing with CLIP (StyleCLIP)

All three routes share one pretrained StyleGAN2 and a CLIP model. They push the
image toward the prompt with CLIP's cosine distance, and keep it near the source
with a penalty on the latent [arxiv:2103.17249 §4] [eq. 1] [eq. 2]. They differ
in what runs per image and what runs per prompt.

- **Route 1, per-image optimisation:** the W+ code is optimised for each image
  and prompt, with 200 to 300 iterations [arxiv:2103.17249 §4]. Inference takes
  98 s on a GTX 1080Ti [arxiv:2103.17249 tab. 1]. The paper calls the method
  sensitive to its parameters [§5] [§7].
- **Route 2, latent mapper:** one small network per group of W+ layers (coarse,
  medium and fine) predicts a residual, and the edited code is the input plus
  that residual [arxiv:2103.17249 §5] [eq. 3]. Each prompt needs its own
  training run of 10 to 12 h, after which an edit takes 75 ms
  [arxiv:2103.17249 tab. 1].
- **Route 3, global direction in style space S:** a text direction is built
  from prompt templates, and only the style channels that respond most to it
  are moved [arxiv:2103.17249 §6] [eq. 6] [eq. 7]. The step is G(s + α Δs), so
  α is a dial. Pre-processing takes 4 h, and each edit takes 72 ms, with no
  dependence on the input image [arxiv:2103.17249 tab. 1].
- **Reported results:** qualitative. The paper gives no image-quality, identity
  or user-study metric [arxiv:2103.17249 §7]. It says the global direction is
  similar to StyleSpace, which changes only the target attribute, and that
  TediGAN fails on all three of its comparison examples [§7] [app. B].

Card: [StyleCLIP](/papers/2103.17249). The card's Paper vs code section lists
where the released code differs from the paper, including the mapper's step
scale and the global channel selection. The code is MIT-licensed, but the
pretrained weights are downloaded separately and their licence is not covered
here.

### Point dragging with latent optimisation (DragGAN)

The user places handle points on a StyleGAN2 image and gives each a target.
Each iteration takes one optimiser step on the latent code. The loss pulls the
features after the sixth generator block, near each handle, one step toward
its target. Then each handle moves to the location whose feature best matches
its original feature [arxiv:2305.10973 §3.1] [§3.2] [§3.3]. The loop stops when
every handle is within a pixel tolerance of its target: 1 px for up to five
handles, 2 px for more [§3.4]. The paper reports 30 to 200 iterations in typical
use [§3.1]. The paper says no optical-flow or tracking network is used
[§1] [§3].

- **Latent space:** only the first six layers of a W+ code are optimised, and
  the rest stay fixed, which the paper says keeps the appearance unchanged
  [arxiv:2305.10973 §3.2]. W is available for a more constrained result
  [§3.2] [fig. 16].
- **Masks:** a user-drawn binary mask keeps unmasked features near their
  initial values, with weight λ = 20 [arxiv:2305.10973 §3.2] [§3.4].
- **Real images:** a real photo is first inverted with PTI, and the same loop
  then runs [arxiv:2305.10973 §4.1]. See the [PTI card](/papers/2106.05744) for
  that step.
- **Reported results:** on a face-landmark test of 1000 cases with one handle,
  the mean landmark distance is 2.44, against 11.64 for UserControllableLT. FID
  is 9.28, against 25.32 [arxiv:2305.10973 tab. 1]. The distance has no unit in
  the paper. Reconstruction error is also lower than the baseline's on all four
  datasets in the paper's table [arxiv:2305.10973 tab. 2].
- **Cost:** 2.0 s per edit for one point, against 0.03 s for UserControllableLT
  [arxiv:2305.10973 tab. 1]. The paper names one RTX 3090 "in most cases", but
  does not say that the timings were measured on it [arxiv:2305.10973 §1]
  [tab. 1].
- **Tracker choice matters:** on the same test, RAFT tracking gives 13.43 for
  one point, which is worse than the baseline's 11.64. PIPs tracking gives 2.98
  [arxiv:2305.10973 tab. 1].
- **Limits:** the authors report artifacts for out-of-distribution human poses,
  and handles in texture-less regions drift [arxiv:2305.10973 §4.3] [fig. 14].

Card: [DragGAN](/papers/2305.10973). The card's Paper vs code section records
that the authors' repository carries a research-only licence, so the card uses
the paper alone. Its Pitfalls section lists details the paper leaves open, such
as how the loss sums are normalised and the units of the stopping tolerance.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Real face photos with a fixed set of binary attributes (CelebA-style), and one cheap pass per edit instead of per-image optimisation | Trained encoder pair (IcGAN) | Two encoders learned after the cGAN give z and y in one forward pass [arxiv:1611.06355 §4.1]; the independent-encoder design had the lowest reconstruction loss [arxiv:1611.06355 §5.3] |
| A fixed set of named attributes (pose, light, expression, age, eyeglasses) on a StyleGAN-type face generator, and many edits per image | Attribute-conditioned flow on W | A trained map with attribute targets; inference of 0.21 to 0.61 s per call [arxiv:2008.02401 §5.3] |
| Chained edits on faces where identity must hold | Attribute-conditioned flow on W | Full method reports a same-person rate of 0.950 on sequential edits, against 0.550 for the best baseline [arxiv:2008.02401 tab. 2] |
| Free-form text instructions and no training budget | CLIP-guided latent optimisation | No training [arxiv:2204.08583 §1]; the price is minutes per image [arxiv:2204.08583 tab. 2] |
| Only a modest GPU is available | CLIP-guided latent optimisation | The paper reports a run under 11 GB of GPU memory [arxiv:2204.08583 §6.1] |
| No labelled data and no attribute classifier | Neither as written | The flow needs attribute labels for training and for source images [arxiv:2008.02401 §5.1]. IcGAN's E_y also needs labelled real images [arxiv:1611.06355 §4.1] |
| Code must be reusable commercially | CLIP-guided latent optimisation | The VQGAN-CLIP code is MIT-licensed. The StyleFlow code is marked non-commercial, so re-implementing from the paper is a decision for a human; see the [StyleFlow card](/papers/2008.02401). StyleGAN-NADA's code is MIT, but its StyleGAN2 checkpoint and datasets carry other licences [tab. 4]; see the [StyleGAN-NADA card](/papers/2108.00946). The IcGAN card lists its repository as BSD-3-Clause but flags that the licence text needs checking with the authors; see the [IcGAN card](/papers/1611.06355). StyleCLIP's code is MIT-licensed [code: LICENSE], but its pretrained weights are downloaded separately; see the [StyleCLIP card](/papers/2103.17249). The CAAE repository has no LICENSE file, so its code cannot be reused on that basis; see the [CAAE card](/papers/1702.08423). DragGAN's repository carries the NVIDIA StyleGAN3 licence, which allows research use only; see the [DragGAN card](/papers/2305.10973) |
| Target domain is far from the generator's training data (a new style, or a shape change such as dog to cat), and no target images exist | Text-only generator adaptation, then latent edits | The adapted generator keeps the source latent space, so the editing tools built for the source can be reused [arxiv:2108.00946 §5.2]. Training takes minutes for texture changes and hours for animal changes [arxiv:2108.00946 app. I] |
| A given image in the generator's own domain must be edited | Trained encoder pair, attribute-conditioned flow or CLIP-guided optimisation | Adaptation changes the generator and does not edit a given image by itself [arxiv:2108.00946 §4.1] |
| A real face photo must be reproduced closely and edited afterwards, and a few minutes of GPU time per identity is acceptable | Per-photo generator tuning (PTI) | Inversion into W, then tuning, gives LPIPS 0.09 against 0.34 for the best baseline [arxiv:2106.05744 tab. 1]; tuning takes under 3 minutes per identity [arxiv:2106.05744 §4.5]. Each identity keeps its own generator copy, as the card's Pitfalls section notes |
| Local edits, inpainting or blending two photos, with per-image optimisation | Noise-map embedding with masks (Image2StyleGAN++) | Masks restrict the embedding to a region, and activation copying blends two embeddings [arxiv:1911.11544 §5] [§7]; inpainting beats partial and gated convolution at three resolutions [arxiv:1911.11544 tab. 1] |
| Changing the apparent age of a face, with no paired photos of one person at different ages, and an age bin as the target | Conditional adversarial autoencoder (CAAE) | The encoder and age-conditioned generator run without a true input age [arxiv:1702.08423 §5.2]; the paper's evidence is a user study, not an age measure [§5.3] |
| Free text edits on a StyleGAN2 image, with a per-image budget of about 100 s and no training per prompt | StyleCLIP per-image optimisation | No prompt training; inference takes 98 s [arxiv:2103.17249 tab. 1]. Sensitive to its parameters [§5] |
| Free text edits on a StyleGAN2 image, many images per prompt, and a training budget of hours | StyleCLIP latent mapper | Each prompt is trained once for 10 to 12 h, then an edit takes 75 ms [arxiv:2103.17249 tab. 1] |
| Free text edits on a StyleGAN2 image, fastest per-image path, and only global attribute-like changes are needed | StyleCLIP global direction | 72 ms per edit with no per-image work, after a 4 h pre-processing step [arxiv:2103.17249 tab. 1]. Its result is not compared quantitatively [§7] |
| Precise, spatial edits on a StyleGAN2 image, such as moving a point by a set amount, where a few seconds per edit is acceptable | Point dragging (DragGAN) | Text guidance is imprecise for spatial edits [arxiv:2305.10973 §1]. Handles move by optimisation, with 2.0 s per edit on the face-landmark test [arxiv:2305.10973 tab. 1]; UserControllableLT is faster at 0.03 s but less accurate there [tab. 1] |
| The input is human motion, and a text prompt should produce a motion | Motion latent space (MotionCLIP) | A frozen CLIP encodes the prompt and the decoder runs on it directly [arxiv:2203.08063 §4.2]; the user-study preference is author-reported [arxiv:2203.08063 tab. 1] |

Attribute control on real photos adds an inversion step. The StyleFlow paper uses
an Image2StyleGAN-style projection into W+ [arxiv:2008.02401 §3] [§6.2]. Plan
for that step separately, because it sets the identity ceiling for every later
edit. IcGAN replaces the per-image projection with trained encoders
[arxiv:1611.06355 §4.1]. Its edit results are qualitative, so the paper gives no
measure of how well identity survives an edit [arxiv:1611.06355 §5.4]. PTI and
Image2StyleGAN++ keep per-photo optimisation instead. PTI adds a tuning step
after inversion, and Image2StyleGAN++ adds noise maps to the W+ embedding
[arxiv:2106.05744 §3.2] [arxiv:1911.11544 §4.2]. StyleCLIP's per-image route
also inverts first, with e4e, and its mapper and global routes work on that
inverted code [arxiv:2103.17249 §4] [§5] [§6]. CAAE has no per-image step: its
encoder runs once, as in IcGAN [arxiv:1702.08423 §5.2]. DragGAN also inverts a
real photo first, with PTI, and then optimises per edit [arxiv:2305.10973 §4.1].

## Implementation sketch

Common core of the editing routes. Variant-specific parts are marked in the
comments. The text-only adaptation route trains a generator and has no editing
step, so it is not in this sketch. The motion route is not an image editor
and is not in this sketch either.

```pseudocode
# Latent-space editing, common core  [arxiv:2008.02401 §6.2] [arxiv:2204.08583 §2]
# Input: optional source image, edit specification
# Output: edited image

def edit(image, edit_spec):
    # Start point: a latent that reproduces the input, or a random one
    # StyleFlow projects to W+; VQGAN-CLIP encodes into the latent grid
    # IcGAN instead runs trained encoders E_z and E_y [arxiv:1611.06355 §4.1]
    # PTI inverts to W, then tunes the generator weights [arxiv:2106.05744 §3.2]
    # Image2StyleGAN++ adds per-layer noise maps [arxiv:1911.11544 §4.2]
    # StyleCLIP route 1 inverts with e4e, then optimises W+ [arxiv:2103.17249 §4]
    # CAAE: one encoder pass gives z; the age label is then changed [§5.2]
    if image is not None:
        z = invert(image)
    else:
        z = sample_latent()
    # Move the latent toward the edit
    # StyleFlow re-decodes under the target attributes [§6.2]
    # VQGAN-CLIP takes gradient steps on a CLIP score [§2]
    # DragGAN steps W+ layers 0-5 toward handle targets, then re-tracks
    # the handles [arxiv:2305.10973 §3.2] [§3.3]
    # IcGAN keeps z and swaps the attribute vector y [arxiv:1611.06355 §5.4]
    z_edit = change_latent(z, edit_spec)
    # Decode; StyleFlow copies untouched W+ rows back from the source [§6.2.3]
    # IcGAN has no such copy step; it decodes G(z, y_new) directly
    # PTI decodes with its tuned generator, not the original [§3.2]
    return decode(z_edit, keep=z)
```

The inversion, the update and the output step differ most between the editing
routes. Keep those three in mind when you pick an implementation, and check the
cards' Paper vs code and Pitfalls sections before you copy any default.

## Contradictions

No contradictions between the papers' claims are known. Their numbers do not
measure the same thing and must not be compared. StyleFlow reports FID and face
identity on face data [arxiv:2008.02401 tab. 1] [tab. 2]. IcGAN reports
reconstruction loss and an attribute classifier's accuracy on CelebA
[arxiv:1611.06355 §5.3] [tab. 3]. VQGAN-CLIP reports human ratings on 12 text
prompts [arxiv:2204.08583 tab. 1]. StyleGAN-NADA reports FID on 10 and 100
target images and a user study on a different task [arxiv:2108.00946 tab. 2]
[§5.1]. PTI reports LPIPS, MSE and identity similarity on CelebA-HQ
[arxiv:2106.05744 tab. 1]. Image2StyleGAN++ reports PSNR on its own
reconstructions and inpainting metrics on its own test set [arxiv:1911.11544
§7.1] [tab. 1]. MotionCLIP reports user preference and BABEL-60 accuracy on
motion data [arxiv:2203.08063 tab. 1] [tab. 3]. CAAE reports vote shares from a
user study on identity [arxiv:1702.08423 §5.3]. StyleCLIP reports timings and
visual comparisons only [arxiv:2103.17249 tab. 1] [§7]. DragGAN reports its
landmark distance in a unit the paper does not state, and reconstruction MSE and
LPIPS on its own test set [arxiv:2305.10973 tab. 1] [tab. 2].

Inside one paper:

- PTI's reconstruction table marks MS-SSIM with a down arrow and gives its own
  method as best at 0.21. Standard MS-SSIM rises with similarity, so the arrow
  or the values may be inverted [arxiv:2106.05744 tab. 1]. Neither reading is
  resolved here.
- VQGAN-CLIP gives the GLIDE training cost as 19,200 V100-hours in the text, but
  20,200 in its break-even discussion and in the Table 2 caption
  [arxiv:2204.08583 §6.2] [tab. 2]. Neither figure is resolved here.
- VQGAN-CLIP gives a runtime for an RTX 2080 Ti in the text, but the runtime
  table has no such row [arxiv:2204.08583 §6] [tab. 2].
- StyleCLIP gives its own per-image time as "several minutes" in the text, but
  98 s in its timing table [arxiv:2103.17249 §4] [§5] [tab. 1]. Budget for the
  larger figure until you measure your own setup.
- CAAE's vote count, 3,208, is larger than its stated volunteers and questions
  allow [arxiv:1702.08423 §5.3]. Neither figure is resolved here.
- IcGAN's text gives an overall generator accuracy of about 86%, but the best
  generator value in its table is 89.83% [arxiv:1611.06355 §5.2] [tab. 3].
  Neither figure is resolved here.

## Papers

- [IcGAN](/papers/1611.06355): a conditional GAN with two trained encoders, so a real image can be inverted and its attributes changed.
- [StyleFlow](/papers/2008.02401): a conditional flow on StyleGAN's W space for attribute-targeted sampling and editing.
- [VQGAN-CLIP](/papers/2204.08583): text-guided generation and editing by optimising a VQGAN latent grid against CLIP, with no training.
- [StyleGAN-NADA](/papers/2108.00946): text-only adaptation of a StyleGAN2 generator to a new domain, so latent-space tools can run on the adapted generator.
- [PTI](/papers/2106.05744): inverts a face into StyleGAN2's W space, then tunes the generator so the fixed code reproduces the photo.
- [Image2StyleGAN++](/papers/1911.11544): embeds a photo into W+ and noise maps, with masks for local edits and inpainting.
- [MotionCLIP](/papers/2203.08063): aligns a human-motion latent space with CLIP, so text prompts decode to motion.
- [CAAE](/papers/1702.08423): a conditional adversarial autoencoder that changes a face's age by changing an age label, trained without paired ages.
- [StyleCLIP](/papers/2103.17249): CLIP-driven edits of StyleGAN2 images in three routes: per-image optimisation, a per-prompt latent mapper, and a global style-space direction.
- [DragGAN](/papers/2305.10973): drags chosen points of a StyleGAN2 image to targets by optimising its latent, with nearest-neighbour point tracking and no tracking network.