--- kind: concept slug: text-guided-image-editing title: Text-guided image editing summary: >- Changes an image from a text prompt. Covered: training-free Stable Diffusion editing that keeps layout; CLIP-driven optimisation (VQGAN, CLIPstyler); per-input layered editing (Text2LIVE); CLIP-steered StyleGAN2 in three routes; a trained outfit GAN; subject-driven and per-image Stable Diffusion. Trade-offs: compute per image, what each keeps, training. tags: [diffusion, gan, image-editing, training-free, stylegan, video-editing, style-transfer] papers: ["1710.07346", "2204.08583", "2204.02491", "2211.12572", "2305.14720", "2212.04489", "2310.07204", "2103.17249", "2112.00374"] related: [ddim-inversion, classifier-free-guidance, image-to-image-translation, diffusion-models, latent-diffusion, text-to-image-personalization, generative-adversarial-network, latent-space-editing, gan-inversion, vision-language-pretraining, internal-learning, neural-style-transfer] status: draft updated_at: 2026-11-01 --- ## Overview Given an input image and a target prompt, the goal is an edited image that follows the prompt while keeping some part of the input. The eight covered variants differ in what they keep. Plug-and-Play keeps the spatial layout of a real image [arxiv:2211.12572 §4]. VQGAN-CLIP starts from the input image, but its loss only scores the match to the text, plus an optional L2 penalty on the latent [arxiv:2204.08583 §2] [§2.4]; the paper reports no metric for how much of the input survives an edit [arxiv:2204.08583 §4.1]. FashionGAN changes only the clothing of a person in a photo, keeping pose and body shape, and is trained for that one task [arxiv:1710.07346 §1] [§3.1]. BLIP-Diffusion takes its subject from a reference photo rather than from the layout of the input image [arxiv:2305.14720 §3.2]. SINE trains a copy of Stable Diffusion on the input photo itself, so the source content is carried by the weights rather than by the layout alone [arxiv:2212.04489 §3.1] [§3.2]. StyleCLIP works on a pretrained StyleGAN2 rather than a diffusion model. A real photo is first inverted into the generator's latent space, and the edit moves that latent under CLIP's guidance [arxiv:2103.17249 §4] [§5]. Text2LIVE keeps the whole input under a learned edit layer, so where the layer's opacity is zero the input shows through unchanged [arxiv:2204.02491 eq. 1] [§3.1]. CLIPstyler needs no style image: it restyles a content photo to match a text description, training a small network per photo whose output is scored by CLIP [arxiv:2112.00374 §3.1] [§4.1]. Text alone gives little control over layout: the same prompt yields different poses and layouts from different initial noise [arxiv:2211.12572 §1]. The Plug-and-Play variant addresses this by running the model on the source image first and carrying part of its internal state into the edited run. The invariant of the Plug-and-Play variant: the edited run starts from the same initial noise as the source's reconstruction. With a different seed, injected features do not match the encoder skip features and the method fails [arxiv:2211.12572 app. B]. A 2023 survey groups editing into partial-noising methods (SDEdit-style), attention control, instruction-based fine-tuning, drag-style edits and inversion-based editing [arxiv:2310.07204 §3.4]. It gives no numbers for these, so this page does not rank them and covers only the cards listed below. ## Variants ### Plug-and-Play diffusion features (training-free) The source image is encoded and DDIM-inverted with an empty prompt to a noise latent. A guidance pass then denoises that latent back while recording a decoder ResBlock feature map and the decoder self-attention queries and keys. The edit pass starts from the same latent, samples with the target prompt, and overwrites the matching feature map and attention queries and keys with the recorded ones during the early sampling steps [arxiv:2211.12572 §4] [alg. 1]. The base model is a frozen Stable Diffusion v1.4 [arxiv:2211.12572 app. C]. The authors pick the feature map from the fourth decoder layer because, in their PCA, it encodes semantic parts (head, legs, torso) consistently across images with very different appearance [arxiv:2211.12572 §4]. Injecting only that feature map keeps too little of the structure; adding it at deeper layers keeps structure but leaks source colors into the result. Adding self-attention at higher-resolution layers on top of the feature map balances the two [arxiv:2211.12572 §4]. Reported defaults: 50 translation steps, feature injection in the first 40 and self-attention injection in the first 25, with classifier-free guidance scale 15 for real images and 7.5 for generated guidance images [arxiv:2211.12572 app. C]. Negative prompting, which replaces the unconditional term with a blend that includes a description of the source, mainly helps textureless inputs such as silhouettes [arxiv:2211.12572 §4] [arxiv:2211.12572 app. A.1]. Reported results are the authors' own, on benchmarks they built. Against SDEdit at three noise levels, P2P, DiffuseIT and VQGAN-CLIP, the authors report a structure distance comparable to SDEdit at a very low noise level and a text score comparable to SDEdit at a very high one [arxiv:2211.12572 §5.1] [fig. 9]. On the Wild-TI2I real-image set, the full method has a structure distance of 0.058, against 0.090 without the feature injection and 0.097 without the self-attention injection [arxiv:2211.12572 tab. 1]. The paper's P2P comparison covers only its two ImageNet-R benchmarks, because P2P needs word-aligned prompts [arxiv:2211.12572 §5.1]. Card: [Plug-and-Play Diffusion Features](/papers/2211.12572). ### VQGAN-CLIP latent optimisation (training-free, slow per image) Editing uses the same loop as generation. A pretrained VQGAN decoder turns a latent grid of codebook vectors into an image, and a pretrained CLIP model scores augmented crops of that image against the prompt. Each step averages the score over many crops, and Adam updates the latent grid [arxiv:2204.08583 §2] [§2.3]. For editing, the start latent is an encoded input image instead of random codes [arxiv:2204.08583 §2] [§2.1]. Nothing is trained [arxiv:2204.08583 §1]. The paper states a learning rate of 0.15 and 400 iterations [arxiv:2204.08583 §2]. Its runtime per image is 188 s on a V100 and 935 s on a K80 [arxiv:2204.08583 tab. 2]. A Colab run is reported under 3 minutes with under 11 GB of video memory [arxiv:2204.08583 §6.1]. The editing comparison with Open-Edit is visual only, with no metric [arxiv:2204.08583 §4.1]. The human study covers 12 prompts, rated 1 to 5. This method averages 4.6, and the best GLIDE baseline with CLIP guidance averages 3.3. The baselines were picked best-of-five and this method best-of-one [arxiv:2204.08583 tab. 1] [§3.2]. These are generation prompts, not edits. Card: [VQGAN-CLIP](/papers/2204.08583). ### Layered per-input editing with CLIP losses (trained per input, image and video) Text2LIVE trains a new generator for each input. The generator is a small U-Net that takes an augmented crop of the input and outputs an edit layer, a colour image plus an opacity map. The output is the opacity-weighted layer laid over the input [arxiv:2204.02491 eq. 1] [§3.1] [app. A.1]. Training uses CLIP only. It has a composition loss against the target prompt, a loss that asks CLIP to match the layer alone over a green background, and a structure loss that compares CLIP token self-similarity of input and output. Sparsity and bootstrap terms act on the opacity map [arxiv:2204.02491 eq. 2] [§3.1]. The method needs no task-specific pretraining and no mask [arxiv:2204.02491 §3.1]. For video, the frames are mapped to a pretrained, frozen neural layered atlas. The same image procedure runs on atlas crops, and its losses are applied to the reconstructed frames, so one edit carries across the video [arxiv:2204.02491 §3.2] [app. A.4]. The paper trains the foreground and background atlases as two separate generators [arxiv:2204.02491 app. A.4]. Cost is per input. The image setting uses 1000 iterations, and the authors report about nine minutes at 512 x 512 on an RTX 6000 [arxiv:2204.02491 app. A.3]. The video setting uses 3000 iterations on 70 frames and takes about an hour per layer on an RTX 8000 [arxiv:2204.02491 app. A.3]. Nothing carries over between inputs [arxiv:2204.02491 app. A.3]. Reported results are the authors' own, from a two-alternative human preference study on Mechanical Turk. For images, Text2LIVE received 0.82 to 0.86 of votes against three mask-free baselines: CLIPStyler, VQ-GAN+CLIP and Diffusion+CLIP [arxiv:2204.02491 tab. 1] [§4.3]. For video it received 0.73 and 0.74 against two video baselines, a single-image method run on the atlas and a per-frame generator [arxiv:2204.02491 tab. 1]. Blended-Diffusion and GLIDE need masks and were compared only qualitatively [arxiv:2204.02491 §4.2]. The paper reports that CLIP can pull new objects into an edit, for example candles on a "birthday cake", and calls the method unsuited to creating new objects [arxiv:2204.02491 §4.5] [fig. 9]. The shipped code differs from the paper in several settings, including the image sparsity weights, the structure-loss form and the bootstrap schedule. Check the card's Paper vs code section before you copy a value. Card: [Text2LIVE: Text-Driven Layered Image and Video Editing](/papers/2204.02491). ### Two-stage GAN for outfit change (trained, one domain) Given a photo of a person and a text description of an outfit, the model returns a 128x128 image of the same person, with the same pose and body shape, in the new outfit [arxiv:1710.07346 §1] [§3.5]. Stage 1 generates a 7-label segmentation map from a coarse 8x8 map of the wearer in which all clothing labels are merged into one class [arxiv:1710.07346 §3.2]. Stage 2 gives each label its own output channel and combines them pixel by pixel using the label map [arxiv:1710.07346 §3.3] [eq. 4]. Hair and face pixels are then copied back from the original photo [arxiv:1710.07346 §3.5]. Both conditional GANs are trained separately, with Adam [arxiv:1710.07346 §3.4]. Training needs one photo, one description and its segmentation for each person. No pairs of one person in two outfits are used [arxiv:1710.07346 §3.1]. The dataset has 78,979 images, split 70,000 for training and 8,979 for testing [arxiv:1710.07346 §3.5] [§4]. On attribute detection, the generated images score mAP 82.6, against 71.8 to 74.7 for the baselines and 86.8 for real photos [arxiv:1710.07346 tab. 1]. These are author-reported. The body does not state the learning rate, batch size or epoch count [arxiv:1710.07346 §3.4]. Card: [Be Your Own Prada: Fashion Synthesis with Structural Coherence](/papers/1710.07346). ### Subject-driven editing from a reference photo (pre-trained, zero-shot or short fine-tune) A frozen BLIP-2 image encoder and a trainable multimodal encoder turn a subject photo and its category word into 16 subject tokens. These are added to the prompt, and Stable Diffusion v1-5 denoises as usual [arxiv:2305.14720 §3.2] [§3.1] [§4.1]. For editing an existing image, the cross-attention maps of a chosen token give a mask. Unedited regions keep the original generation's latents, and the masked regions take the subject-driven result [arxiv:2305.14720 §3.3] [fig. 4]. The paper gives no numeric values for the editing thresholds [arxiv:2305.14720 §3.3]. No per-subject training is needed for the zero-shot setting. A short fine-tune of the U-Net on a few photos runs 40 to 120 steps, with a mean of 76 [arxiv:2305.14720 §3.3] [app. A.5] [tab. 3]. Pre-training used 500K steps on 16 A100 40GB GPUs over six days, and only the paper reports it; the released repository has fine-tuning configs, not that stage [arxiv:2305.14720 §4.1] [code: lavis/projects/blip_diffusion/finetune-db-dog.yaml#L45]. Reported results are the authors' own. On DreamBench, the zero-shot model scores DINO 0.594 and the fine-tuned model 0.670, against 0.668 for DreamBooth [arxiv:2305.14720 tab. 1]. That table measures subject fidelity in generated images, not edit quality, and the quantitative comparison covers no editing benchmark [arxiv:2305.14720 §4.2]. Card: [BLIP-Diffusion](/papers/2305.14720). ### Single-image fine-tune with guidance mixing (per-image training, qualitative results) SINE fine-tunes a pretrained Stable Diffusion on one image. Each training sample is a random patch of that image, resized to the model's native size, and the denoiser also gets a positional code for where the patch sits [arxiv:2212.04489 §3.3]. At sampling time the full-image code is used, and the noise prediction for early steps (timestep above K = 400) mixes the fine-tuned and pretrained models with weight v = 0.7. Later steps use the pretrained model alone [arxiv:2212.04489 §3.2] [§4] [§4.3]. The reported training setup is a base learning rate of 1e-6, batch size 1, 800 steps without patches and 10,000 with patches, on one RTX 8000 GPU [arxiv:2212.04489 §4]. The paper states no wall-clock time. Patch training is what the authors credit for edits at a larger size without duplicated objects, though very different aspect ratios still produce artifacts they show [arxiv:2212.04489 §4.3] [app. D]. Expect to choose K and v per image; the appendix says the best values differ by image [arxiv:2212.04489 app. C]. Reported results are qualitative. The paper has no numeric headline, its K and v sweeps are plots without values in the text [arxiv:2212.04489 fig. 8], and its comparisons keep the best of four samples per prompt [arxiv:2212.04489 §4.1]. The baselines are DreamBooth and Textual Inversion [arxiv:2212.04489 §4.2]. Card: [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](/papers/2212.04489). Implementation warning: the released CLI applies the fine-tuned term at the other end of the schedule from the paper. Its default window is 1 to 400, while the paper, README and notebook use the window above K = 400 [code: scripts/stable_txt2img_guidance.py#L190-L191]. Copy the notebook's values. ### Text-driven StyleGAN editing (pretrained GAN, three routes) StyleCLIP drives a pretrained StyleGAN2 generator with the CLIP cosine distance to the prompt [arxiv:2103.17249 §4] [eq. 1]. The paper offers three routes that trade cost per image against dependence on the input: - **Latent optimisation** (per image and prompt). Gradient descent on the latent minimises the CLIP term, a penalty on the distance from the source latent, and an optional ArcFace identity term [arxiv:2103.17249 §4] [eq. 1] [eq. 2]. The paper states 200 to 300 iterations [§4]. Inference takes 98 s on a single GTX 1080Ti [arxiv:2103.17249 tab. 1]. The paper calls this method sensitive to its parameters and leaves it out of its quantitative comparison [arxiv:2103.17249 §5] [§7]. - **Latent mapper** (trained per prompt). Three small networks, one each for coarse, medium and fine layer groups of the W+ latent, predict a residual for any input latent in one forward pass [arxiv:2103.17249 §5] [eq. 3]. Training takes 10 to 12 h and inference 75 ms, both on a GTX 1080Ti [tab. 1]. - **Global direction** (input-independent, in style space S). A text direction is computed from prompt embeddings, and per-channel CLIP relevance keeps only the style channels that move with it [arxiv:2103.17249 §6] [eq. 6] [eq. 7]. The strength is a user dial. Pre-processing takes 4 h and inference 72 ms, and the same direction applies to any input [tab. 1]. The paper's comparisons are visual. It reports no quantitative image-quality or identity metric [arxiv:2103.17249 §7] [app. B]. Card: [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](/papers/2103.17249). ### CLIP-trained text-only style transfer (trained per image, or once per text) CLIPstyler restyles a content photo from a text prompt alone. For each content image and text, it trains a small U-Net from scratch; the network's output is the stylised image [arxiv:2112.00374 §3.1] [§4.1]. The main term is a whole-image directional CLIP loss. It compares how the output moves in CLIP's embedding space from the content photo with the move from a source word to the target text [arxiv:2112.00374 eq. 2]. A patch term applies the same test to 64 random 128-pixel crops of the output, each with a random perspective change. Patches whose loss is at or below a threshold of 0.7 count as zero [arxiv:2112.00374 §3.2] [eq. 3] [eq. 4] [app. A]. A VGG-19 content term on two layers and a total-variation term complete the loss [arxiv:2112.00374 §4.1]. Cost is per image and per text. The paper trains 200 iterations with Adam, at a learning rate of 5 × 10⁻⁴ halved at iteration 100, and reports about 40 seconds per text on one RTX 2080 Ti at 512 × 512 [arxiv:2112.00374 §4.1]. The paper states that real-time per-input use needs its fast variant [arxiv:2112.00374 app. G]. The fast variant trains only the decoder of a pretrained encoder-decoder, on random crops of DIV2K images. It reports under 0.5 s of inference per content image [arxiv:2112.00374 §5.1] [app. A]. The authors say its quality is lower than the per-image method [arxiv:2112.00374 §5.1]. Reported results are the authors' own. In a 25-rater study, the method scores 3.78 for style, the highest of seven methods, against 3.32 for AdaIN with CLIP [arxiv:2112.00374 tab. 1]. Its content score is 4.18, below StyleCLIP's 4.35 [tab. 1]. Its CLIP score is 0.2515, against 0.2487 for AdaIN with CLIP [arxiv:2112.00374 tab. 2]; the paper does not name the CLIP model used for that score [app. B.3]. In the ablation, the full loss is preferred at 3.92 and removing the patch term drops it to 1.69 [arxiv:2112.00374 tab. 3]. Implementation warning: the paper's introduction and threshold paragraph say that high-scoring patches are rejected, but its equation 4 zeroes patches at or below the threshold [arxiv:2112.00374 §1] [§3.2] [eq. 4]. Follow the equation. Card: [CLIPstyler: Image Style Transfer with a Single Text Condition](/papers/2112.00374). ### Other editing methods (no card yet) The Plug-and-Play paper compares against SDEdit, Prompt-to-Prompt, DiffuseIT, VQGAN-CLIP, Text2LIVE, DiffusionCLIP and FlexIT [arxiv:2211.12572 §5.1]. Of these, only VQGAN-CLIP and Text2LIVE have cards here. The VQGAN-CLIP paper names Open-Edit, minDALL-E and GLIDE as its baselines [arxiv:2204.08583 §3.2] [§4.1]. The Text2LIVE paper names Blended-Diffusion, GLIDE, CLIPStyler, Diffusion+CLIP, VQ-GAN+CLIP, StyleCLIP and StyleGAN-NADA among its comparisons [arxiv:2204.02491 §4.2]. None of the methods without a card has one here, so this page says nothing about how they work or how they score. Prompt-to-Prompt is linked on arXiv for reference: [arXiv:2208.01626](https://arxiv.org/abs/2208.01626). SDEdit is linked on arXiv for reference: [arXiv:2108.01073](https://arxiv.org/abs/2108.01073). ## When to use which The rows below rest on the cards above. They do not rank the variants against each other, because the cards report different metrics on different tasks (see Contradictions). | Situation | Use | Why | |---|---|---| | Zero-shot edit of a real image, source layout must survive, frozen Stable Diffusion v1.4 available, source has clear semantic parts such as people, animals or objects | Plug-and-Play diffusion features | The method is built for this case and the authors' benchmarks report it [arxiv:2211.12572 §4] [arxiv:2211.12572 §5.1] | | Per-image compute is tight and the layout must survive | Plan for the cost of Plug-and-Play | Real images need about 1000 inversion and 1000 reconstruction U-Net calls before the edit, plus the edit pass itself [arxiv:2211.12572 app. C] | | Zero-shot open-domain edit, no training, minutes per image are acceptable, and no layout or fidelity metric is required | VQGAN-CLIP latent optimisation | Needs no training; the paper reports 188 s per image on a V100 [arxiv:2204.08583 tab. 2] and no edit metric [arxiv:2204.08583 §4.1] | | Source is textureless or its colors carry no meaning, e.g. segmentation masks with arbitrary colors | Not Plug-and-Play | The paper names this as a failure case [arxiv:2211.12572 §6] [fig. 12] | | Change the clothing on a person photo at 128x128, keep pose and body shape, and you can train on a comparable dataset | Two-stage GAN for outfit change | The model is built for this; the authors report higher attribute mAP than their baselines [arxiv:1710.07346 tab. 1]. The dataset terms and most training details are not stated, so check both before reuse [arxiv:1710.07346 §5] [§3.4] | | Place a specific subject from a reference photo into an image, Stable Diffusion v1-5 is acceptable, and 40 to 120 fine-tune steps per subject are affordable if you need more fidelity | BLIP-Diffusion | The zero-shot mode takes one photo at inference; the fine-tune is short [arxiv:2305.14720 §3.3]. The reported subject-fidelity numbers come from DreamBench, not from an edit benchmark [arxiv:2305.14720 tab. 1] | | One real photo must be edited by a new prompt, the content of that photo should survive, and per-image fine-tuning on one GPU is acceptable | SINE | Built for single-image editing, with 10,000 patch-training steps at batch 1 reported [arxiv:2212.04489 §4]. Its results are qualitative only, so this row carries no metric [arxiv:2212.04489 §4.2] [§4.3] | | Layout of a real photo must survive, and no reference photo of a subject is needed | Plug-and-Play diffusion features | Copying the source's internal features is the mechanism that keeps layout [arxiv:2211.12572 §4] | | Change the look of an existing object, or add an effect such as smoke, in one image without a mask, with no pretrained generator; about nine minutes per image on one GPU is acceptable | Text2LIVE, image setting | The layer keeps the input underneath [arxiv:2204.02491 eq. 1]. Training is 1000 iterations per input [arxiv:2204.02491 app. A.3]. The preference wins are against mask-free baselines only [arxiv:2204.02491 tab. 1] | | Apply the same kind of edit across a short video, and a pretrained neural layered atlas of that video exists | Text2LIVE, video setting | One edit to the atlas carries across frames [arxiv:2204.02491 §3.2]. A 70-frame layer takes about an hour [arxiv:2204.02491 app. A.3]. Artefacts in the atlas show up in the edit [arxiv:2204.02491 §4.5] | | Word-level control over a prompt that names parts of the source | Look at Prompt-to-Prompt (no card here) | The paper says P2P needs word-aligned source and target prompts and does not keep parts the prompt does not name [arxiv:2211.12572 §1] [arxiv:2211.12572 §2] | | Edit a StyleGAN2 image from text, a single image at a time, with no training, and about 100 s per image is acceptable | StyleCLIP latent optimisation | Needs no training; the paper reports 98 s on a GTX 1080Ti [arxiv:2103.17249 tab. 1]. The paper calls it parameter-sensitive, so expect to tune per edit [arxiv:2103.17249 §5] | | Restyle a photo from a text description with no style image, keep its content, and about 40 s of GPU training per photo and text is acceptable | CLIPstyler, per-image training | The method trains per content and text pair and reports the best style score among its seven compared methods [arxiv:2112.00374 §4.1] [tab. 1]. Its content score is below StyleCLIP's in the same study [tab. 1] | | Apply one text style to many photos quickly, with under 0.5 s per image after a single training run, and the fast variant's lower quality is acceptable | CLIPstyler fast variant | One decoder trained per text, then fast inference [arxiv:2112.00374 §5.1] [app. A]. The fast outputs sometimes spread texture into the background [§5.1] [fig. 9] | | Apply the same text edit to many inputs, each needing about 75 ms, and you can spend 10 to 12 h training a network per prompt | StyleCLIP latent mapper | One forward pass per input after training [arxiv:2103.17249 tab. 1] [§5]. Check the training data before reuse, since the paper does not state it [arxiv:2103.17249 §5] | | A single text-driven change for any input, with one strength dial, and 4 h of pre-processing per domain is acceptable | StyleCLIP global direction | The direction does not depend on the input and inference is 72 ms [arxiv:2103.17249 tab. 1] [§6]. Large shape changes can fail [arxiv:2103.17249 app. B] | ## Implementation sketch Own-words sketch of the Plug-and-Play variant. It omits classifier-free guidance and the negative prompt for brevity [arxiv:2211.12572 §4]. ```pseudocode # Structure-preserving edit by reusing source features [arxiv:2211.12572 alg. 1] # Input: source image I, target prompt P, steps=50, feat_steps=40, attn_steps=25 # Output: edited image def edit(I, P, steps=50, feat_steps=40, attn_steps=25): # Invert the source with an empty prompt to a shared start noise [app. C] x_T = ddim_invert(encode(I), prompt="", n=1000) # Guidance pass: record one decoder feature map and decoder q, k per step [§4] recorded = [] x = x_T for t in timesteps(steps): eps, feat, qk = unet_record(x, t, prompt="") recorded.append((feat, qk)) x = ddim_step(x, eps, t) # Edit pass: same start noise, target prompt, overwrite the early steps [§4] x = x_T for i, t in enumerate(timesteps(steps)): feat, qk = recorded[i] eps = unet_inject( x, t, P, feat=feat if i < feat_steps else None, qk=qk if i < attn_steps else None, ) x = ddim_step(x, eps, t) return decode(x) ``` Where variants differ: the other variants do not share this loop. VQGAN-CLIP optimises a latent grid against CLIP's score [arxiv:2204.08583 §2], Text2LIVE trains a per-input generator whose layer is composited over the input [arxiv:2204.02491 §3.1], FashionGAN trains two conditional GANs [arxiv:1710.07346 §3.1], BLIP-Diffusion conditions the U-Net on subject tokens instead of on a source feature map [arxiv:2305.14720 §3.2], and SINE fine-tunes the U-Net on the source and mixes its noise prediction into the pretrained one for early steps [arxiv:2212.04489 §3.2]. StyleCLIP has no diffusion trajectory at all: it optimises or maps a StyleGAN latent [arxiv:2103.17249 §4] [§5]. CLIPstyler trains a small U-Net per content image against CLIP's directional loss, with no latent to invert [arxiv:2112.00374 §3.1]. Their cards hold their own descriptions. ## Contradictions No card states a result that another card contradicts. The results are not comparable across cards, and the cards do not share a metric: - Plug-and-Play reports structure distance and CLIP text score on benchmarks built by its authors [arxiv:2211.12572 §5.1] [tab. 1]. - VQGAN-CLIP reports a 1-to-5 human rating on 12 generation prompts [arxiv:2204.08583 tab. 1]. - FashionGAN reports attribute-detection mAP on its own test set [arxiv:1710.07346 tab. 1]. - Text2LIVE reports the share of human votes in two-alternative comparisons, on 82 image-text pairs and 19 video-text pairs [arxiv:2204.02491 tab. 1] [§4.3]. Its image baselines include VQ-GAN+CLIP, but its vote share is not on the 1-to-5 scale that the VQGAN-CLIP card uses, so the two sets of numbers do not compare. - BLIP-Diffusion reports DINO, CLIP-I and CLIP-T on DreamBench, a subject-fidelity benchmark for generation [arxiv:2305.14720 tab. 1]. - SINE reports no numbers in its text. Its LPIPS and CLIP values are only in plots [arxiv:2212.04489 fig. 8]. - StyleCLIP reports timings and visual comparisons, with no quantitative image-quality metric [arxiv:2103.17249 tab. 1] [§7]. - CLIPstyler reports StyleCLIP scores of 1.47 for style and 4.35 for content, and a CLIP score of 0.1982, in its own study on face images for that baseline [arxiv:2112.00374 tab. 1] [tab. 2] [app. B.2]. These come from a setup that CLIPstyler chose, not from StyleCLIP's paper, so do not compare them with StyleCLIP's own visual results. - CLIPstyler's headline numbers are 1-to-5 user ratings from 25 raters and a CLIP cosine score, both from its own study [arxiv:2112.00374 tab. 1] [tab. 2]. They are not on the scale of the VQGAN-CLIP ratings or the Text2LIVE votes. - StyleCLIP's text says latent optimisation takes "several minutes" per edit [arxiv:2103.17249 §4] [§5], while its table gives 98 s on a GTX 1080Ti [arxiv:2103.17249 tab. 1]. The paper does not reconcile the two. The Plug-and-Play authors state that VQGAN-CLIP and DiffuseIT optimise the CLIP and DINO self-similarity metrics directly [arxiv:2211.12572 §5.1]. That is their reading of the method; the VQGAN-CLIP card does not measure structure. Guidance-weight convention differs between the survey and the editing cards. The survey writes classifier-free guidance as (1+w) times the conditional prediction minus w times the unconditional one [arxiv:2310.07204 eq. 11]. The Plug-and-Play and SINE cards write w times the conditional plus (1-w) times the unconditional [arxiv:2211.12572 eq. 5] [arxiv:2212.04489 eq. 2]. The two forms are the same with the weight shifted by one, so a survey w equals the editing cards' w minus 1. Check which form a code base uses before you copy a value. ## Papers - [Plug-and-Play Diffusion Features](/papers/2211.12572): training-free editing of a real image by copying one decoder feature map and decoder self-attention queries and keys from a source pass into a target-prompt pass on a frozen Stable Diffusion. - [VQGAN-CLIP](/papers/2204.08583): training-free text-driven generation and editing by optimising a pretrained VQGAN latent grid against CLIP's score. - [Be Your Own Prada: Fashion Synthesis with Structural Coherence](/papers/1710.07346): a two-stage conditional GAN that changes the outfit in a person's photo while keeping pose and body shape. - [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](/papers/2305.14720): a pre-trained subject encoder that conditions Stable Diffusion on a reference photo, with a mask-based editing mode. - [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](/papers/2212.04489): fine-tunes Stable Diffusion on one image with patch crops and a positional code, then mixes its prediction into the pretrained model for early steps. - [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](/papers/2103.17249): text-driven edits of a pretrained StyleGAN2 through per-image latent optimisation, a per-prompt latent mapper, or a global direction in style space. - [CLIPstyler: Image Style Transfer with a Single Text Condition](/papers/2112.00374): a per-image U-Net trained with patch-wise CLIP directional losses to restyle a photo from text alone, with a fast decoder-only variant. - [Text2LIVE: Text-Driven Layered Image and Video Editing](/papers/2204.02491): a per-input generator outputs a colour-and-opacity layer over the image, trained with CLIP losses, with a neural-atlas route for video. - [State of the Art on Diffusion Models for Visual Computing](/papers/2310.07204): a survey with no experiments of its own; its editing section is a narrative taxonomy, used here only as background.