---
kind: concept
slug: neural-style-transfer
title: Neural style transfer
summary: >-
  Restyles images and meshes. ArtFlow, a reversible flow against content leak (SSIM 0.45 against 0.29 for AdaIN); CLIPstyler, from a text prompt alone (3.78 against 3.32); STROTSS, per-image optimal-transport optimisation (95 s per 512 px image); Splice, a per-pair network on frozen DINO features (79.0% against STROTSS); Text2Mesh recolours a mesh (3.90 against 2.83).
tags: [style-transfer, invertible-networks, normalizing-flows, feature-transfer, self-supervised-features, diffusion-guidance, gan-inversion, 3d-mesh, clip, text-guided, patch-loss, optimal-transport, latency]
papers: ["1904.12785", "1911.11544", "2103.16877", "2112.00374", "2112.03221", "2112.05744", "2201.00424"]
related: [adaptive-instance-normalization, image-to-image-translation, diffusion-models, vision-transformer, self-supervised-learning, gan-inversion, latent-space-editing, differentiable-rendering, fourier-features, vision-language-pretraining, text-guided-image-editing, internal-learning, optimal-transport]
status: draft
updated_at: 2026-11-01
---

## Overview

The problem: take a content image and a style image, and produce the content
image rendered in the style's colours and textures [arxiv:2103.16877 §1].
Feed-forward methods do this with a fixed encoder, a feature transfer step
and a learned decoder [arxiv:2103.16877 §3.1].

The shared weakness is content leak. When an output is fed back in as the
next content image, the content fades over rounds, because the decoder does
not invert the encoder exactly [arxiv:2103.16877 §3.1] [arxiv:2103.16877 §3.2].

The invariant that the ArtFlow variants aim for is that the only step that
changes the picture is the style step. The image is mapped to latent features
by an exactly invertible network, the style is applied there, and the same
network maps the result back to pixels [arxiv:2103.16877 §4.1].

A second family does not transform a content image at all. It samples from
noise and steers a pretrained diffusion model with a score that compares
style statistics to a reference image [arxiv:2112.05744 §3.1]
[arxiv:2112.05744 §3.3].

A third approach keeps the layout of a structure image and changes the look
of its objects to match semantically related objects in a second, appearance
image. It judges the result with frozen self-supervised ViT features rather
than with global style statistics [arxiv:2201.00424 §1] [arxiv:2201.00424 §3].

Optimisation-based methods refine one output image by gradient descent,
against a fixed feature extractor, rather than running a trained network
forward. STROTSS is one of them. Each image is its own optimisation, which
takes 95 s at 512 px on one GTX 1080 [arxiv:1904.12785 tab. 1]
[arxiv:1904.12785 §4.4]. The masked StyleGAN step below is another.

A fourth works inside a fixed StyleGAN and edits a real photo. The photo is
first embedded into the generator's latent space, and a style term on VGG-16
features then restyles a masked part of it [arxiv:1911.11544 §4.1]
[arxiv:1911.11544 alg. 6].

A fifth changes the look of a 3D mesh rather than an image. It takes a
text prompt and trains a small network that gives each vertex a colour and a
bounded offset along its normal. The geometry is not replaced, and the
training signal is CLIP similarity between the prompt and rendered views
[arxiv:2112.03221 §1] [arxiv:2112.03221 §3.1] [arxiv:2112.03221 §3.2].

A sixth takes no style image at all. It takes a text prompt and trains a
small image-to-image network from scratch for each photo and prompt. The
network's output is scored by CLIP as a change in direction, from the word
"Photo" to the prompt, both for the whole image and for random patches of it
[arxiv:2112.00374 §1] [arxiv:2112.00374 §3.1] [arxiv:2112.00374 §4.1]
[arxiv:2112.00374 eq. 2] [arxiv:2112.00374 eq. 3].

## Variants

### Reversible-flow transfer with AdaIN or WCT (ArtFlow)

The encoder and decoder are replaced by one invertible network, a
Glow-style flow with two blocks of eight flows [arxiv:2103.16877 §5.1]
[arxiv:2103.16877 fig. 6]. Each flow is an actnorm, a learned 1×1
convolution and an additive coupling step [arxiv:2103.16877 §4.2]. The
latent transfer is AdaIN or WCT, so the same network serves both
[arxiv:2103.16877 §4.1] [arxiv:2103.16877 §4.3]. Card: [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](/papers/2103.16877).

Author-reported results against the same methods without the flow:
SSIM 0.45 for ArtFlow+AdaIN against 0.29 for AdaIN, and 0.47 for
ArtFlow+WCT against 0.27 for WCT [arxiv:2103.16877 tab. 1]. Over 20 rounds
of repeated transfer, the AdaIN and WCT variants show no visible content
leak, while the plain methods do [arxiv:2103.16877 §5.3]
[arxiv:2103.16877 fig. 1].

### Style guidance in diffusion sampling (semantic diffusion guidance)

The generator is an unconditional DDPM trained elsewhere, and it is not
retrained [arxiv:2112.05744 §3.1]. At each reverse step the mean is shifted by
a scaled gradient of a guidance score [arxiv:2112.05744 alg. 1]. For style,
the score is the negative sum, over CLIP feature layers, of squared differences
between the Gram matrices of the noisy sample and the noised reference
[arxiv:2112.05744 §3.3] [arxiv:2112.05744 eq. 9]. The scoring encoder is a
CLIP image encoder whose batch-normalisation layers are conditioned on the
timestep. It is finetuned on noised images without captions
[arxiv:2112.05744 §3.5]. Card: [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](/papers/2112.05744).

The paper reports no metric for style guidance. Its only table covers image
and text guidance on FFHQ [arxiv:2112.05744 tab. 1]. Style examples appear
among the qualitative figures, [arxiv:2112.05744 fig. 3] to
[arxiv:2112.05744 fig. 7], and are not measured. The released code implements
style guidance in a different form from the paper; see the card's Paper vs
code section before using it.

### Single-pair semantic appearance transfer (Splice)

A U-Net generator is trained from scratch on one structure and appearance
pair, with no adversarial term [arxiv:2201.00424 §3]. A frozen DINO ViT-B/8
provides three losses [arxiv:2201.00424 app. A.2]. The appearance loss matches
the deepest CLS token of the output to that of the appearance image. The
structure loss matches the deepest-layer key self-similarity of the output to
that of the structure image. The identity loss keeps the appearance image's
keys when the generator is fed that image [arxiv:2201.00424 eq. 4]
[arxiv:2201.00424 eq. 5] [arxiv:2201.00424 eq. 6] [arxiv:2201.00424 eq. 7].
Card: [Splicing ViT Features for Semantic Appearance Transfer](/papers/2201.00424).

Author-reported 2AFC preference for Splice, from a study on 65 images with
7000 judgments: against STROTSS, 79.0% on Wild-Pairs and 58.8% on Mountains
and 59.7% on AFHQ; against WCT², 83.1% on Wild-Pairs; against Swapping
Autoencoders, 56.3% on Mountains and 71.8% on AFHQ [arxiv:2201.00424 tab. 1]
[arxiv:2201.00424 §4.1.2]. Structure is kept by the authors' IoU measure at
0.88 on Wild-Pairs, against 0.89 for WCT² and 0.83 for STROTSS
[arxiv:2201.00424 tab. 2]. The paper states that WCT² scores highest on this
measure because it only changes colours [arxiv:2201.00424 §4.1.2].

The paper reports failures when the two objects are semantically unrelated or
one image is far from natural [arxiv:2201.00424 §4.3] [arxiv:2201.00424 fig. 10].

### Masked local style transfer in a fixed StyleGAN (Image2StyleGAN++)

This variant does not stylize an image in one forward pass. A real photo is
embedded into a pretrained StyleGAN, whose weights stay fixed
[arxiv:1911.11544 §5]. The embedding has two groups of variables: a W+ code,
which carries semantic content, and per-layer noise maps, which carry
high-frequency detail [arxiv:1911.11544 §4]. The W+ code is optimised first,
then the noise maps, each once [arxiv:1911.11544 §4.2].

For local style transfer, algorithm 6 combines two masked steps. A W+
optimisation runs on the content image under one blurred mask, and a masked
style step runs under the complementary mask. A noise step follows
[arxiv:1911.11544 alg. 6] [arxiv:1911.11544 §7.5]. The style step has 1000
iterations, then the noise step has 1000 [arxiv:1911.11544 §7.5]. The style
term is computed on VGG-16 conv3_3 features [arxiv:1911.11544 §4.1]. The
paper does not state its exact form beyond that. Card: [Image2StyleGAN++: How to Edit the Embedded Images?](/papers/1911.11544).

The paper reports no style-quality measure for this application. Its
measures are reconstruction PSNR [arxiv:1911.11544 §7.1], inpainting
[arxiv:1911.11544 tab. 1], edit realism [arxiv:1911.11544 tab. 2] and
perceptual length [arxiv:1911.11544 tab. 3]. None of them scores a style
transfer result.

### Text-driven mesh stylization (Text2Mesh)

The input is a triangle mesh with fixed faces, and the output is the same
mesh with a colour per vertex and a small displacement along each normal. A
Fourier-feature MLP, the neural style field, gives both from the vertex
position [arxiv:2112.03221 §3.1] [arxiv:2112.03221 eq. 1]. The displacement
is bounded to a small range [arxiv:2112.03221 §3.1]. No 3D generative model
or dataset of stylized meshes is used [arxiv:2112.03221 §1]. Training
maximises the cosine similarity between CLIP embeddings of augmented,
multi-view renders and the prompt [arxiv:2112.03221 eq. 5]. A
geometry-only render, which is grey and shows only the displacement, is one
of the three terms, so that geometry is trained as well as colour
[arxiv:2112.03221 §3.2]. Card: [Text2Mesh: Text-Driven Neural Stylization for Meshes](/papers/2112.03221).

Author-reported user study: 57 users rated 8 pairs each, on a 1 to 5 scale.
Text2Mesh scores 3.90 against 2.83 for a VQGAN-CLIP adaptation on natural
depiction, 4.04 against 3.60 on keeping the original content, and 3.91
against 2.59 on matching the target style [arxiv:2112.03221 tab. 1]. Pairs
with mismatched image and text scored 1.16 [arxiv:2112.03221 §4.3]. The
paper notes that a CLIP score can be high for a degenerate result, so its
CLIP ablation numbers are weak evidence on their own [arxiv:2112.03221 §4.2].

### Text-only stylization with CLIP directional losses (CLIPstyler)

The input is a content photo and a text prompt. No style image is used
[arxiv:2112.00374 §1]. For each photo and prompt, a small U-Net is trained
from scratch, and its output is the stylized photo [arxiv:2112.00374 §3.1]
[arxiv:2112.00374 §4.1]. Four weighted terms are summed: a whole-image
directional CLIP loss, a patch-wise directional CLIP loss, a VGG-19 content
loss and a total-variation term [arxiv:2112.00374 eq. 5]. The directional
loss compares how CLIP's embedding moves from the photo to the output with
how it moves from the source word to the prompt [arxiv:2112.00374 eq. 2].
Card: [CLIPstyler: Image Style Transfer with a Single Text Condition](/papers/2112.00374).

The patch term takes 64 random 128-pixel crops, each given a random
perspective change. A crop's loss is set to zero when it is at or below
0.7, and the term is averaged over all 64 crops [arxiv:2112.00374 §4.1]
[arxiv:2112.00374 app. A] [arxiv:2112.00374 eq. 3] [arxiv:2112.00374 eq. 4].
Training runs for 200 Adam steps at learning rate 5 × 10⁻⁴, with the rate
halved at step 100 [arxiv:2112.00374 §4.1]. The authors report about 40
seconds per photo and prompt on one RTX 2080 Ti [arxiv:2112.00374 §4.1].

A fast variant trains only the decoder of a pretrained VGG encoder-decoder,
on DIV2K crops, for one fixed prompt. The authors report under 0.5 seconds
per 512-pixel content image at inference [arxiv:2112.00374 §5.1], and about
4 seconds for a 3000 × 2000 output [arxiv:2112.00374 §5.2].

Author-reported results, from a 25-rater user study on 512 × 512 images. Style
scores 3.78 for CLIPstyler, the highest of seven methods, against 3.32 for
AdaIN with CLIP, the next highest [arxiv:2112.00374 tab. 1]. Content scores
4.18, below StyleCLIP at 4.35 [arxiv:2112.00374 tab. 1]. The CLIP score is
0.2515, against 0.2487 for AdaIN with CLIP [arxiv:2112.00374 tab. 2]. In the
ablation, removing threshold rejection lowers the overall preference from 3.92
to 3.02 [arxiv:2112.00374 tab. 3].

### Relaxed optimal-transport style loss, per-image optimisation (STROTSS)

The input is a content image and a style image, and no network is trained.
The output starts from the style image's mean colour plus the content's
high-frequency band, then is refined by RMSprop over a Laplacian pyramid at
four scales, from 64 to 512 px on the long side [arxiv:1904.12785 §2.5].

The style term is a relaxed earth mover's distance between sets of VGG16
hypercolumn features, with a cosine ground metric. Moment matching and a
colour palette term are added [arxiv:1904.12785 §2.2] [arxiv:1904.12785 eq. 7]
[arxiv:1904.12785 eq. 9] [arxiv:1904.12785 eq. 10]. The content term compares
the pattern of pairwise feature similarities, not pixels. It keeps layout while
pixel values change [arxiv:1904.12785 §2.3] [arxiv:1904.12785 eq. 11].
Card: [Style Transfer by Relaxed Optimal Transport and Self-Similarity](/papers/1904.12785).

Users can pin a region of the content to a region of the style. The paper
gives a cost weight of 5 for matched pairs and an infinite cost for
mismatched ones [arxiv:1904.12785 §2.4]. The released entry point supports
region masks only; see the card's Paper vs code section.

Author-reported, a 662-worker study covering three regimes (paired, unpaired
and texture) finds higher style scores than four prior methods at a given
content score. The text gives no numeric table [arxiv:1904.12785 §4.1]
[arxiv:1904.12785 fig. 8]. The paper's timing table puts it at 95 s at 512 px,
against 33 s for Gatys et al. 2016 in the same table [arxiv:1904.12785
tab. 1]. The relaxed transport distance is a lower bound on the exact one, at
about 0.60 of its value [arxiv:1904.12785 §4.3].

### Compared methods with no card here

The ArtFlow card compares against StyleSwap, AdaIN, WCT, LinearWCT,
OptimalWCT, Avatar-Net and Self-Contained [arxiv:2103.16877 tab. 1]. The
Splice card compares against Swapping Autoencoders, WCT² and SinCUT, and
against STROTSS, which has a card above [arxiv:2201.00424 §4.1]. Gatys et al.
2016 is the baseline in the STROTSS timing table and has no card either
[arxiv:1904.12785 tab. 1]. None of these has a card in this wiki, so they are
not described here beyond the numbers the two papers report. Their own papers
are not used as sources.

- The Text2Mesh card compares against a VQGAN-CLIP adaptation to meshes, which
  the paper uses as its only baseline [arxiv:2112.03221 §4.3]. It has no card
  here.
- AdaIN is the cheapest of these in the author-reported timing, 0.064 s at
  512×512 [arxiv:2103.16877 tab. 1]. See [adaptive-instance-normalization](/concepts/adaptive-instance-normalization).
- Avatar-Net is the slowest, 9.129 s at 512×512 [arxiv:2103.16877 tab. 1].
  ArtFlow's Avatar-Net variant is shown qualitatively only
  [arxiv:2103.16877 §5.3].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Output will be fed back in, or stylized many times | ArtFlow with AdaIN or WCT | Content leak is the failure the variant addresses; the authors report no visible leak after 20 rounds [arxiv:2103.16877 §5.3] |
| One-shot stylization, latency is the main constraint | Plain AdaIN | Author-reported 0.064 s against 0.408 s for ArtFlow+AdaIN at 512×512 [arxiv:2103.16877 tab. 1] |
| Stylization with WCT, speed matters | ArtFlow+WCT | Author-reported 0.428 s against 0.997 s for WCT, because it needs no multi-level stylization [arxiv:2103.16877 §5.2] |
| Style fidelity measured by Gram loss matters most | Not ArtFlow | ArtFlow is not best on Gram loss; OptimalWCT is best [arxiv:2103.16877 tab. 1] |
| Human preference is the target | ArtFlow | Most votes in the authors' user study, 314 of 799 effective votes [arxiv:2103.16877 tab. 2] |
| No budget to train a flow | Plain AdaIN | Training ArtFlow takes about 12 h on one RTX 2080Ti in the paper's setup [arxiv:2103.16877 §5.1] |
| Must keep the layout of an input image | ArtFlow with AdaIN or WCT | The flow maps the given content image back to pixels [arxiv:2103.16877 §4.1]; diffusion style guidance starts from noise, so it has no input layout to keep [arxiv:2112.05744 alg. 1] |
| Objects in one image should take on the look of related objects in another, and layout must stay | Splice | Object-level, semantic matching with frozen ViT features [arxiv:2201.00424 §3.2]; author IoU 0.88 on Wild-Pairs [arxiv:2201.00424 tab. 2] |
| Objects in the two images are semantically unrelated | Not Splice | The paper reports failure in this case [arxiv:2201.00424 §4.3] |
| Only colours need to change | Not Splice | The paper says WCT² keeps layout best because it only changes colours [arxiv:2201.00424 §4.1.2]; WCT² has no card here |
| Only a style reference is available, and a new image from noise is acceptable | Diffusion style guidance | Uses a pretrained diffusion model without retraining it [arxiv:2112.05744 §3.1]; the paper gives no style metric to compare against [arxiv:2112.05744 tab. 1] |
| Per-image cost is the constraint | Plain AdaIN | Each diffusion step runs a CLIP forward and backward pass, and the paper gives no runtime numbers [arxiv:2112.05744 §4]. Splice also trains a network per pair, about 20 min at 512×512 [arxiv:2201.00424 app. A.3] |
| One region of a real photo should take a new style, and the goal is a local edit | Masked local style transfer | The paper's stated aim is local edits of an embedded photo [arxiv:1911.11544 §1]; algorithm 6 splits the image into masked parts [arxiv:1911.11544 alg. 6] |
| A real photo must be embedded before any edit | Masked local style transfer | Embedding is the method's starting step [arxiv:1911.11544 §5]; reconstruction runs 5000 W+ and 3000 noise steps [arxiv:1911.11544 §7.1] |
| Input is a new image, or latency is the constraint | Not this variant | The method is iterative optimisation, and the paper gives no wall-clock time per image [arxiv:1911.11544 §6] |
| Stylization must be measured against a style metric | Not this variant | The paper reports no style-quality measure for this application [arxiv:1911.11544 tab. 1] |
| Layout must survive, pixel values may change a lot, and a user should pin regions of content to regions of style | STROTSS | Region pairs are a user input in the method, with a cost for mismatched pairs [arxiv:1904.12785 §2.4]; the released entry point supports region masks only, see the card |
| One image at a time, quality matters more than time, and no network may be trained | STROTSS | Each image is optimised with no training [arxiv:1904.12785 §2.5]; the paper's human study favours it over prior methods at a given content score [arxiv:1904.12785 §4.1] |
| Latency is the constraint | Not STROTSS | Author-reported 95 s per 512 px image on a GTX 1080 [arxiv:1904.12785 tab. 1]; Gatys et al. 2016 is faster at 33 s in the same table. The ArtFlow variant reports 0.408 s at 512 px, on its own hardware [arxiv:2103.16877 tab. 1] |
| A numeric style score is needed for a report | Not STROTSS | The human study has no numeric table in the text [arxiv:1904.12785 §4.1] [arxiv:1904.12785 fig. 8]; the paper's other numbers are timings and a REMD ratio |
| Input is a 3D triangle mesh, and the change is a text-driven colour and fine surface detail | Text2Mesh | It is built for this task, and the paper says no prior method does it, so its baseline is an adapted 2D method [arxiv:2112.03221 §4.3] |
| Input is an image, or the output must be an image | Not Text2Mesh | Text2Mesh outputs a recoloured and displaced mesh [arxiv:2112.03221 §3.1] |
| Interactive use, or a fixed time budget per asset | Not Text2Mesh | Per-mesh optimisation takes about 25 min on one RTX 2080 Ti for 1500 iterations [arxiv:2112.03221 app. D.2] |
| No CUDA GPU is available | Not Text2Mesh | The paper's timing runs on one RTX 2080 Ti [arxiv:2112.03221 app. D.2]; the released install needs CUDA, see the [Text2Mesh card](/papers/2112.03221) |
| Mesh has far more than the paper's size limit | Remesh first, or not Text2Mesh | The paper's limit is up to 180K triangles on 11 GB of GPU memory [arxiv:2112.03221 §3.1] |
| Prompt is unrelated to the object's category | Add the category to the prompt | For a mismatched prompt the shape can be erased; naming the category keeps content [arxiv:2112.03221 §4.6] |
| Only a text description is available, and about 40 s of GPU time per photo and prompt is acceptable | CLIPstyler, per-image | The method takes a text prompt and no style image [arxiv:2112.00374 §1]; training is about 40 s on one RTX 2080 Ti in the authors' setup [arxiv:2112.00374 §4.1] |
| Many photos for one fixed prompt, at interactive speed | CLIPstyler fast variant | Author-reported under 0.5 s per 512-pixel image [arxiv:2112.00374 §5.1]; the fast model is trained for one prompt [arxiv:2112.00374 §5.1]. Its losses differ from the paper's in the released code, see the card |
| Content must stay as close as possible to the input | Not CLIPstyler alone | Author-reported content score 4.18, below StyleCLIP's 4.35 [arxiv:2112.00374 tab. 1] |
| A reference style image exists and its look must be matched | Not CLIPstyler | The method takes no style image [arxiv:2112.00374 §1]; use a variant that takes one, such as ArtFlow [arxiv:2103.16877 §4.1] or Splice [arxiv:2201.00424 §3] |

Constraints behind the table:

- **Training data:** MS-COCO for content and WikiArt for style
  [arxiv:2103.16877 §5.1]. Splice trains on one pair per generator, and its
  evaluation uses AFHQ, Mountains and Wild-Pairs [arxiv:2201.00424 §4].
  Text2Mesh needs a mesh and a prompt, and no dataset [arxiv:2112.03221 §1].
- **Diffusion guidance needs:** a pretrained unconditional DDPM and a CLIP
  ResNet 50×16 image encoder, finetuned for noisy inputs
  [arxiv:2112.05744 §4.1].
- **STROTSS needs:** a VGG16 network for its features [arxiv:1904.12785 §2.1],
  and a GPU for the timings the paper reports, on a GTX 1080 [arxiv:1904.12785
  §4.4].
- **Splice needs:** a frozen DINO ViT-B/8 and a GPU for each pair. The paper
  reports training on a single RTX 6000 [arxiv:2201.00424 app. A.2]
  [arxiv:2201.00424 app. A.3].
- **Image2StyleGAN++ needs:** a pretrained StyleGAN, whose weights are not
  trained by the method [arxiv:1911.11544 §5], and VGG-16 features for the
  style and perceptual terms [arxiv:1911.11544 §4.1]. The paper's examples are
  mostly faces, with some LSUN cars and bedrooms [arxiv:1911.11544 §9.4]. It
  was run on one 32 GB V100 [arxiv:1911.11544 §6].
- **Released weights:** the ArtFlow card's Paper vs code section lists which
  checkpoints the repository provides. Check them before training from
  scratch.
- **CLIPstyler needs:** a pretrained CLIP model and VGG-19 weights for the
  content loss [arxiv:2112.00374 §4.1]. The paper does not name the CLIP model
  behind its CLIP score [arxiv:2112.00374 app. B.3]. The released script
  runs only on CUDA; see the [card](/papers/2112.00374) for that and for the
  fast variant's weights and data.
- **Licence:** the papers are CC BY 4.0. The released ArtFlow, diffusion,
  Splice and STROTSS code have no licence file, so their licence is unknown. Do not copy
  code from any of them without clearing that first. The Text2Mesh code is
  MIT, see its [card](/papers/2112.03221). The CLIPstyler card records MIT
  for its code, but the repository's LICENSE file names a different copyright
  holder, so check that before copying anything; see its
  [Paper vs code](/papers/2112.00374) section. The Image2StyleGAN++ card records
  no official code, see its [Paper vs code](/papers/1911.11544) section.

## Implementation sketch

Common core of the ArtFlow variants: forward pass on both images, latent
transfer, reverse pass [arxiv:2103.16877 §4.1]. The transfer step is the
only part that changes between AdaIN and WCT [arxiv:2103.16877 §4.3].

```pseudocode
# Forward flow on both images, lossless by design  [arxiv:2103.16877 §4.1]
def stylize(content, style, flow, transfer):
    z_c = flow.forward(content)
    z_s = flow.forward(style)
    # Transfer in latent space: AdaIN or WCT  [arxiv:2103.16877 §4.3]
    z_out = transfer(z_c, z_s)
    # Reverse pass maps the stylized latent back to pixels  [arxiv:2103.16877 §4.1]
    return flow.reverse(z_out)
```

```pseudocode
# One flow step, forward  [arxiv:2103.16877 §4.2]
def flow_step_forward(x):
    # Actnorm: per-channel scale and shift  [arxiv:2103.16877 eq. 4]
    x = scale * x + bias
    # Learned c×c mixing of channels  [arxiv:2103.16877 eq. 3]
    x = conv_1x1(x, W)
    # Additive coupling: keep half, shift the other half  [arxiv:2103.16877 §4.2]
    x_a, x_b = split(x)
    x_b = x_b + net(x_a)
    return concat(x_a, x_b)


# Exact inverse, given the same parameters  [arxiv:2103.16877 §4.2]
def flow_step_reverse(y):
    y_a, y_b = split(y)
    y_b = y_b - net(y_a)
    x = conv_1x1(concat(y_a, y_b), inverse(W))
    return (x - bias) / scale
```

The paper does not write the inverse out, and says it can be derived
[arxiv:2103.16877 §4.2]. The sketch above is that derivation applied to the
same step. Parameter values for `net`, `scale` and the initial state are
not stated in the paper; see Pitfalls.

The STROTSS style term is shown below. The content term and the coarse-to-fine
loop are in the [STROTSS card](/papers/1904.12785), under Method.

```pseudocode
# Relaxed EMD with cosine ground metric  [arxiv:1904.12785 eq. 7] [eq. 9]
# out_feats: (n, d) output features; style_feats: (m, d) style features
def strotss_remd(out_feats, style_feats):
    # Pairwise cosine distances  [arxiv:1904.12785 eq. 9]
    cost = 1 - normalise(out_feats) @ normalise(style_feats).T
    # Mean over outputs of cheapest style match  [arxiv:1904.12785 eq. 8]
    out_side = cost.min(axis=1).mean()
    # Mean over style features of cheapest output  [arxiv:1904.12785 eq. 8]
    style_side = cost.min(axis=0).mean()
    # Relaxed value is the larger one-sided cost  [arxiv:1904.12785 eq. 7]
    return max(out_side, style_side)
```

```pseudocode
# Splice loss for one structure and appearance pair  [arxiv:2201.00424 eq. 4]
# Training only: the frozen ViT computes the losses, the generator is updated
def splice_loss(G, dino, I_s, I_t):
    I_o = G(I_s)
    # Appearance: deepest CLS token of output vs appearance  [arxiv:2201.00424 eq. 5]
    loss_app = norm(cls(dino, I_t) - cls(dino, I_o))
    # Structure: deepest-layer key self-similarity  [arxiv:2201.00424 eq. 6]
    loss_struct = frob(selfsim(dino, I_s) - selfsim(dino, I_o))
    # Identity: keys of the appearance image kept by G  [arxiv:2201.00424 eq. 7]
    loss_id = frob(keys(dino, I_t) - keys(dino, G(I_t)))
    # Weights alpha = beta = 0.1  [arxiv:2201.00424 §3.3]
    return loss_app + 0.1 * loss_struct + 0.1 * loss_id
```

The masked style step of the local variant is shown below. The W+ and noise
steps around it follow the same loop structure [arxiv:1911.11544 alg. 6].

```pseudocode
# Masked style step, optimises w with n fixed  [arxiv:1911.11544 eq. 4]
# Input: W+ start code, noise maps, style mask, style image y
def masked_style_step(G, w_ini, n_ini, mask_s, y, steps):
    w = w_ini
    # 1000 iterations in the §7.5 application  [arxiv:1911.11544 §7.5]
    for _ in range(steps):
        # Style term, weight 5e-7 in this step  [arxiv:1911.11544 §6]
        loss = 5e-7 * style_loss(mask_s, G(w, n_ini), y)
        # Adam unless the step says otherwise  [arxiv:1911.11544 §6]
        w = adam_step(w, loss)
    return w
```

```pseudocode
# Style guidance during diffusion sampling  [arxiv:2112.05744 §3.1]
# Input: reference image, guidance scale; no content image
def style_guided_sample(model, reference_0, scale):
    # Start from standard Gaussian noise  [arxiv:2112.05744 alg. 1]
    x = sample_gaussian_noise()
    for t in reversed(range(T)):
        mu, sigma_sq = model(x, t)
        # Gradient of the Gram-based style score  [arxiv:2112.05744 eq. 9]
        grad = gradient_of(style_score, x, reference_0, t)
        # Shift the reverse-step mean  [arxiv:2112.05744 alg. 1]
        x = sample_gaussian(mu + scale * sigma_sq * grad, sigma_sq)
    return x
```

```pseudocode
# Text2Mesh loss for one step  [arxiv:2112.03221 §3.2]
# Input: mesh vertices and normals, CLIP embedding of the prompt
def text2mesh_loss(field, verts, normals, target, views):
    # Colour and bounded offset per vertex  [arxiv:2112.03221 §3.1]
    colors, offsets = field(verts)
    # Move each vertex along its normal  [arxiv:2112.03221 §3.1]
    displaced = verts + offsets * normals
    # Coloured render and grey render of the same mesh  [arxiv:2112.03221 §3.2]
    full = render(displaced, colors, views)
    grey = render(displaced, gray, views)
    # Global term on the coloured render  [arxiv:2112.03221 eq. 2]
    s_full = mean(clip_embed(global_aug(full)))
    # Local term on the coloured render  [arxiv:2112.03221 eq. 3]
    s_local = mean(clip_embed(local_aug(full)))
    # Geometry-only term on the grey render  [arxiv:2112.03221 eq. 4]
    s_displ = mean(clip_embed(local_aug(grey)))
    # Sum of cosines, minimised as its negative  [arxiv:2112.03221 eq. 5]
    return -sum(cosine(s, target) for s in [s_full, s_local, s_displ])
```

The sketch leaves out the anchor-view choice and the view sampling; see the
card's [Paper vs code](/papers/2112.03221) section for how the released code
differs on those and on the weighting of the three terms.

The CLIPstyler patch term is a different kind of sketch. It shows the
threshold rule and the averaging, which are the parts most often implemented
wrongly.

```pseudocode
# CLIPstyler patch term with threshold rejection  [arxiv:2112.00374 eq. 3]
# Input: 64 crops of the output, CLIP embedding of the content, text direction
def patch_term(crops, content_emb, text_dir, tau=0.7):
    losses = []
    for crop in crops:
        # Random perspective change, then CLIP embedding  [arxiv:2112.00374 app. A]
        d_img = normalise(normalise(clip_image(perspective(crop))) - content_emb)
        # One minus cosine similarity with the text direction  [arxiv:2112.00374 eq. 3]
        losses.append(1 - dot(d_img, text_dir))
    # Zero a patch whose loss is at or below tau  [arxiv:2112.00374 eq. 4]
    kept = [s if s > tau else 0 for s in losses]
    # Mean over all N patches, zeroed ones included  [arxiv:2112.00374 eq. 3]
    return sum(kept) / len(kept)
```

## Contradictions

The papers report no results on the same quantity, so no numbers
conflict. They do use the word "style" differently. In ArtFlow, style is
AdaIN or WCT applied to a content image's latent [arxiv:2103.16877 §4.3]. In
the diffusion paper, style is a Gram-matrix score that pulls a sample drawn
from noise toward a reference [arxiv:2112.05744 §3.3]. In Image2StyleGAN++,
style is a term on VGG-16 features that changes the W+ code of a fixed
generator under a mask [arxiv:1911.11544 §4.1] [arxiv:1911.11544 eq. 4]. In Splice, the
transferred quantity is the CLS token and key similarities of a frozen ViT,
applied to objects in the structure image [arxiv:2201.00424 §3.2]. In
Text2Mesh, style is a colour and a bounded normal offset on each vertex of a
mesh, scored by CLIP similarity to a text prompt [arxiv:2112.03221 §3.1]
[arxiv:2112.03221 §3.2]. In STROTSS, style is a relaxed optimal-transport
distance between sets of VGG16 features, plus moment and palette terms
[arxiv:1904.12785 §2.2]. Results from one do not transfer to the others.

CLIPstyler's threshold is described in opposite directions within the paper.
The introduction and the threshold paragraph say that patches with high
scores are rejected [arxiv:2112.00374 §1] [arxiv:2112.00374 §3.2]. Equation
4 zeroes a patch when its loss is at or below τ, so high-loss patches are
kept [arxiv:2112.00374 eq. 4]. This page states both and does not choose
between them. The released code follows the equation; see the
[CLIPstyler card](/papers/2112.00374).

## Pitfalls

- **Paper and code differ on training.** The paper uses batch size 2 and
  60,000 iterations [arxiv:2103.16877 §5.1]. The released code's defaults
  and scripts differ; the ArtFlow card's Paper vs code section lists each
  value. Use one source and state it.
- **Loss form.** The paper's two losses are written with L2 norms
  [arxiv:2103.16877 eq. 1] [arxiv:2103.16877 eq. 2]. The released code uses
  mean squared error instead, which changes the balance that the loss
  weights 0.1 and 1 set. The paper does not discuss this.
- **Content target in training.** The paper's content loss takes the AdaIN
  output as its target [arxiv:2103.16877 eq. 1]. The card records that the
  released code compares against the content image instead.
- **Operator order.** The text lists coupling first; the released code
  applies actnorm, then the 1×1 convolution, then coupling
  [arxiv:2103.16877 §4.2].
- **Metric details are missing.** The paper does not state the test set
  for its metrics or which VGG layers its SSIM and loss use
  [arxiv:2103.16877 tab. 1].
- **Gram loss is not a ranking.** Do not read Table 1 as an overall ranking.
  In that table OptimalWCT has the lowest Gram loss, and ArtFlow is not
  best on it [arxiv:2103.16877 tab. 1].
- **Style weight does not transfer.** The paper's style score is a sum of
  squared Gram differences [arxiv:2112.05744 eq. 9]. The released code uses a
  normalised, averaged form with a large fixed multiplier that is not in the
  paper. A scale tuned on one will not carry over to the other; see the
  diffusion card's Paper vs code section.
- **Gram statistics drop layout.** A Gram matrix holds channel-by-channel
  inner products and no spatial positions [arxiv:2112.05744 §3.3]. Style
  guidance therefore steers texture and colour, not where objects sit.
- **Style results are unmeasured.** The diffusion paper's only quantitative
  table is for image and text guidance on FFHQ [arxiv:2112.05744 tab. 1].
  Do not claim a style quality level from it.
- **Results are author-reported.** The ArtFlow comparison is on its own
  metrics and timing [arxiv:2103.16877 tab. 1]. The diffusion paper's Table 1
  is on FFHQ only, and StyleGAN+CLIP has the highest retrieval scores, which
  the paper says come at the cost of low diversity [arxiv:2112.05744 §4.2].
- **Splice loss scale.** The paper's weights give a total loss about a tenth
  of the released code's, with the same ratios. The paper does not say
  whether the absolute scale matters. See the Splice card's Paper vs code
  section for the other differences: norms, warm-up, training length and
  the augmentation probabilities [arxiv:2201.00424 §3.3].
- **STROTSS code differs from the paper.** The paper's content weight starts
  at 16 and is halved at each of four scales. That gives 2 at the last scale,
  not the stated 1 [arxiv:1904.12785 §2.5]. The released code also scales the
  content term by the number of sampled points, and differs on sampling,
  iterations and colour metric. See the card's Paper vs code section, and pick
  one source before tuning α.
- **STROTSS guidance.** The paper's point-to-point guidance is not reached by
  the released entry point, so use region masks only [arxiv:1904.12785 §2.4].
- **STROTSS results are random.** Sampling and a random grid offset make each
  run stochastic, and no seeds are stated [arxiv:1904.12785 §2.5].
- **STROTSS has no numeric human-study table.** The text gives no numbers for
  its 662-worker study, so do not quote a score for it [arxiv:1904.12785 §4.1].
- **Splice cost is per pair.** Each new pair needs its own training run,
  about 20 min at 512×512, and the paper gives no inference-time budget
  [arxiv:2201.00424 app. A.3].
- **Splice metrics are small.** The preference study has 65 images, and
  standard deviations reach 18 points [arxiv:2201.00424 §4.1.2]
  [arxiv:2201.00424 tab. 1]. The IoU table measures layout, not appearance
  quality [arxiv:2201.00424 tab. 2].
- **Local variant has no style metric.** Its tables cover inpainting, edit
  realism and perceptual length, and reconstruction PSNR is reported
  separately [arxiv:1911.11544 tab. 1] [arxiv:1911.11544 tab. 2]
  [arxiv:1911.11544 tab. 3] [arxiv:1911.11544 §7.1]. Do not read any of
  them as a style quality score.
- **Style weight is off in most steps.** The style weight is 5e-7 in the style
  step and 0 in the W+ step [arxiv:1911.11544 §6] [arxiv:1911.11544 eq. 2].
- **Summed outputs are unclear.** Algorithm 6 writes the W+ output as a sum of
  two results, and the text does not say how the two are combined
  [arxiv:1911.11544 alg. 6].
- **No official code.** The Image2StyleGAN++ card's [Paper vs code](/papers/1911.11544)
  section lists the search that found no repository.
- **Text2Mesh defaults are not the paper's setup.** Running the released main
  script with its defaults gives a different iteration count, no learning-rate
  decay and no geometry term. The paper's values, and the demo flags that
  match them, are on the [Text2Mesh card](/papers/2112.03221) in its Paper vs
  code section [arxiv:2112.03221 app. D.2].
- **Text2Mesh CLIP scores are weak evidence.** A degenerate result can score
  high [arxiv:2112.03221 §4.2]. Check outputs by eye, and keep the geometry
  term on, since the ablation shows the network can fake shape with shading
  [arxiv:2112.03221 §4.2].
- **Text2Mesh can erase the shape.** A prompt unrelated to the object can
  remove its content. Naming the object category in the prompt helps
  [arxiv:2112.03221 §4.6].
- **Text2Mesh metrics are small.** The user study has 57 users rating 8 pairs
  each. Standard deviations on the three questions run from 0.37 to 0.68
  [arxiv:2112.03221 tab. 1].
- **CLIPstyler threshold direction.** Equation 4 zeroes patches at or below
  τ = 0.7, while the prose says high-scoring patches are rejected
  [arxiv:2112.00374 eq. 4] [arxiv:2112.00374 §3.2]. An implementation that
  follows the prose inverts the mechanism. Check the equation and the card's
  Paper vs code section before writing it.
- **CLIPstyler patch mean.** The patch term divides by all 64 patches,
  zeroed ones included [arxiv:2112.00374 eq. 3]. Averaging only the kept
  patches gives a different scale, and the loss weights were tuned with the
  paper's form.
- **CLIPstyler CLIP score is close.** The margin over AdaIN with CLIP is 0.003,
  0.2515 against 0.2487 [arxiv:2112.00374 tab. 2]. The score is not a style
  measure, and the paper does not name the CLIP model used for it
  [arxiv:2112.00374 app. B.3].
- **CLIPstyler baselines on faces.** The StyleCLIP and StyleGAN-NADA
  comparisons use face images as content, since those generators were trained
  on faces [arxiv:2112.00374 app. B.2] [arxiv:2112.00374 §4.3]. Do not read
  them as results on general photos.
- **CLIPstyler statistics.** The user studies have 25 and 20 raters, and the
  paper gives means with no confidence intervals or significance tests
  [arxiv:2112.00374 app. B.2] [arxiv:2112.00374 app. C.2].
- **CLIPstyler fast variant differs from the paper.** The paper says the fast
  model uses the same losses and decay as the per-image method
  [arxiv:2112.00374 app. A]. The released code uses a different content loss,
  learning-rate decay and output range; see the card's Paper vs code section.

## Papers

- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](/papers/1904.12785): per-image optimisation with a relaxed optimal-transport style loss over VGG16 features and a self-similarity content loss, with optional region guidance.
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](/papers/2103.16877): reversible flow in place of the encoder-decoder, with AdaIN or WCT in latent space, to stop content leak under repeated stylization.
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](/papers/2112.05744): Gram-matrix style score as one of several guidance types that steer a pretrained diffusion sampler at sampling time, with no retraining.
- [Splicing ViT Features for Semantic Appearance Transfer](/papers/2201.00424): single-pair U-Net trained with losses from frozen DINO ViT features, so that objects in a structure image take on the look of semantically related objects in an appearance image.
- [Image2StyleGAN++: How to Edit the Embedded Images?](/papers/1911.11544): embeds a photo into a fixed StyleGAN in two stages, then applies masked edits, including a masked style step, that stay local.
- [CLIPstyler: Image Style Transfer with a Single Text Condition](/papers/2112.00374): a per-photo U-Net trained with patch-wise CLIP directional losses, restyling a photo from a text prompt with no style image.
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](/papers/2112.03221): a small network that colours a fixed mesh and offsets its vertices along the normal, trained with CLIP similarity on augmented multi-view renders.