Neural style transfer

Overview

The problem: take a content image and a style image, and produce the content image rendered in the style's colours and textures 2103.16877 §1. Feed-forward methods do this with a fixed encoder, a feature transfer step and a learned decoder 2103.16877 §3.1.

The shared weakness is content leak. When an output is fed back in as the next content image, the content fades over rounds, because the decoder does not invert the encoder exactly 2103.16877 §3.1 2103.16877 §3.2.

The invariant that the ArtFlow variants aim for is that the only step that changes the picture is the style step. The image is mapped to latent features by an exactly invertible network, the style is applied there, and the same network maps the result back to pixels 2103.16877 §4.1.

A second family does not transform a content image at all. It samples from noise and steers a pretrained diffusion model with a score that compares style statistics to a reference image 2112.05744 §3.1 2112.05744 §3.3.

A third approach keeps the layout of a structure image and changes the look of its objects to match semantically related objects in a second, appearance image. It judges the result with frozen self-supervised ViT features rather than with global style statistics 2201.00424 §1 2201.00424 §3.

Optimisation-based methods refine one output image by gradient descent, against a fixed feature extractor, rather than running a trained network forward. STROTSS is one of them. Each image is its own optimisation, which takes 95 s at 512 px on one GTX 1080 1904.12785 tab. 1 1904.12785 §4.4. The masked StyleGAN step below is another.

A fourth works inside a fixed StyleGAN and edits a real photo. The photo is first embedded into the generator's latent space, and a style term on VGG-16 features then restyles a masked part of it 1911.11544 §4.1 1911.11544 alg. 6.

A fifth changes the look of a 3D mesh rather than an image. It takes a text prompt and trains a small network that gives each vertex a colour and a bounded offset along its normal. The geometry is not replaced, and the training signal is CLIP similarity between the prompt and rendered views 2112.03221 §1 2112.03221 §3.1 2112.03221 §3.2.

A sixth takes no style image at all. It takes a text prompt and trains a small image-to-image network from scratch for each photo and prompt. The network's output is scored by CLIP as a change in direction, from the word "Photo" to the prompt, both for the whole image and for random patches of it 2112.00374 §1 2112.00374 §3.1 2112.00374 §4.1 2112.00374 eq. 2 2112.00374 eq. 3.

Variants

Reversible-flow transfer with AdaIN or WCT (ArtFlow)

The encoder and decoder are replaced by one invertible network, a Glow-style flow with two blocks of eight flows 2103.16877 §5.1 2103.16877 fig. 6. Each flow is an actnorm, a learned 1×1 convolution and an additive coupling step 2103.16877 §4.2. The latent transfer is AdaIN or WCT, so the same network serves both 2103.16877 §4.1 2103.16877 §4.3. Card: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows.

Author-reported results against the same methods without the flow: SSIM 0.45 for ArtFlow+AdaIN against 0.29 for AdaIN, and 0.47 for ArtFlow+WCT against 0.27 for WCT 2103.16877 tab. 1. Over 20 rounds of repeated transfer, the AdaIN and WCT variants show no visible content leak, while the plain methods do 2103.16877 §5.3 2103.16877 fig. 1.

Style guidance in diffusion sampling (semantic diffusion guidance)

The generator is an unconditional DDPM trained elsewhere, and it is not retrained 2112.05744 §3.1. At each reverse step the mean is shifted by a scaled gradient of a guidance score 2112.05744 alg. 1. For style, the score is the negative sum, over CLIP feature layers, of squared differences between the Gram matrices of the noisy sample and the noised reference 2112.05744 §3.3 2112.05744 eq. 9. The scoring encoder is a CLIP image encoder whose batch-normalisation layers are conditioned on the timestep. It is finetuned on noised images without captions 2112.05744 §3.5. Card: More Control for Free! Image Synthesis with Semantic Diffusion Guidance.

The paper reports no metric for style guidance. Its only table covers image and text guidance on FFHQ 2112.05744 tab. 1. Style examples appear among the qualitative figures, 2112.05744 fig. 3 to 2112.05744 fig. 7, and are not measured. The released code implements style guidance in a different form from the paper; see the card's Paper vs code section before using it.

Single-pair semantic appearance transfer (Splice)

A U-Net generator is trained from scratch on one structure and appearance pair, with no adversarial term 2201.00424 §3. A frozen DINO ViT-B/8 provides three losses 2201.00424 app. A.2. The appearance loss matches the deepest CLS token of the output to that of the appearance image. The structure loss matches the deepest-layer key self-similarity of the output to that of the structure image. The identity loss keeps the appearance image's keys when the generator is fed that image 2201.00424 eq. 4 2201.00424 eq. 5 2201.00424 eq. 6 2201.00424 eq. 7. Card: Splicing ViT Features for Semantic Appearance Transfer.

Author-reported 2AFC preference for Splice, from a study on 65 images with 7000 judgments: against STROTSS, 79.0% on Wild-Pairs and 58.8% on Mountains and 59.7% on AFHQ; against WCT², 83.1% on Wild-Pairs; against Swapping Autoencoders, 56.3% on Mountains and 71.8% on AFHQ 2201.00424 tab. 1 2201.00424 §4.1.2. Structure is kept by the authors' IoU measure at 0.88 on Wild-Pairs, against 0.89 for WCT² and 0.83 for STROTSS 2201.00424 tab. 2. The paper states that WCT² scores highest on this measure because it only changes colours 2201.00424 §4.1.2.

The paper reports failures when the two objects are semantically unrelated or one image is far from natural 2201.00424 §4.3 2201.00424 fig. 10.

Masked local style transfer in a fixed StyleGAN (Image2StyleGAN++)

This variant does not stylize an image in one forward pass. A real photo is embedded into a pretrained StyleGAN, whose weights stay fixed 1911.11544 §5. The embedding has two groups of variables: a W+ code, which carries semantic content, and per-layer noise maps, which carry high-frequency detail 1911.11544 §4. The W+ code is optimised first, then the noise maps, each once 1911.11544 §4.2.

For local style transfer, algorithm 6 combines two masked steps. A W+ optimisation runs on the content image under one blurred mask, and a masked style step runs under the complementary mask. A noise step follows 1911.11544 alg. 6 1911.11544 §7.5. The style step has 1000 iterations, then the noise step has 1000 1911.11544 §7.5. The style term is computed on VGG-16 conv3_3 features 1911.11544 §4.1. The paper does not state its exact form beyond that. Card: Image2StyleGAN++: How to Edit the Embedded Images?.

The paper reports no style-quality measure for this application. Its measures are reconstruction PSNR 1911.11544 §7.1, inpainting 1911.11544 tab. 1, edit realism 1911.11544 tab. 2 and perceptual length 1911.11544 tab. 3. None of them scores a style transfer result.

Text-driven mesh stylization (Text2Mesh)

The input is a triangle mesh with fixed faces, and the output is the same mesh with a colour per vertex and a small displacement along each normal. A Fourier-feature MLP, the neural style field, gives both from the vertex position 2112.03221 §3.1 2112.03221 eq. 1. The displacement is bounded to a small range 2112.03221 §3.1. No 3D generative model or dataset of stylized meshes is used 2112.03221 §1. Training maximises the cosine similarity between CLIP embeddings of augmented, multi-view renders and the prompt 2112.03221 eq. 5. A geometry-only render, which is grey and shows only the displacement, is one of the three terms, so that geometry is trained as well as colour 2112.03221 §3.2. Card: Text2Mesh: Text-Driven Neural Stylization for Meshes.

Author-reported user study: 57 users rated 8 pairs each, on a 1 to 5 scale. Text2Mesh scores 3.90 against 2.83 for a VQGAN-CLIP adaptation on natural depiction, 4.04 against 3.60 on keeping the original content, and 3.91 against 2.59 on matching the target style 2112.03221 tab. 1. Pairs with mismatched image and text scored 1.16 2112.03221 §4.3. The paper notes that a CLIP score can be high for a degenerate result, so its CLIP ablation numbers are weak evidence on their own 2112.03221 §4.2.

Text-only stylization with CLIP directional losses (CLIPstyler)

The input is a content photo and a text prompt. No style image is used 2112.00374 §1. For each photo and prompt, a small U-Net is trained from scratch, and its output is the stylized photo 2112.00374 §3.1 2112.00374 §4.1. Four weighted terms are summed: a whole-image directional CLIP loss, a patch-wise directional CLIP loss, a VGG-19 content loss and a total-variation term 2112.00374 eq. 5. The directional loss compares how CLIP's embedding moves from the photo to the output with how it moves from the source word to the prompt 2112.00374 eq. 2. Card: CLIPstyler: Image Style Transfer with a Single Text Condition.

The patch term takes 64 random 128-pixel crops, each given a random perspective change. A crop's loss is set to zero when it is at or below 0.7, and the term is averaged over all 64 crops 2112.00374 §4.1 2112.00374 app. A 2112.00374 eq. 3 2112.00374 eq. 4. Training runs for 200 Adam steps at learning rate 5 × 10⁻⁴, with the rate halved at step 100 2112.00374 §4.1. The authors report about 40 seconds per photo and prompt on one RTX 2080 Ti 2112.00374 §4.1.

A fast variant trains only the decoder of a pretrained VGG encoder-decoder, on DIV2K crops, for one fixed prompt. The authors report under 0.5 seconds per 512-pixel content image at inference 2112.00374 §5.1, and about 4 seconds for a 3000 × 2000 output 2112.00374 §5.2.

Author-reported results, from a 25-rater user study on 512 × 512 images. Style scores 3.78 for CLIPstyler, the highest of seven methods, against 3.32 for AdaIN with CLIP, the next highest 2112.00374 tab. 1. Content scores 4.18, below StyleCLIP at 4.35 2112.00374 tab. 1. The CLIP score is 0.2515, against 0.2487 for AdaIN with CLIP 2112.00374 tab. 2. In the ablation, removing threshold rejection lowers the overall preference from 3.92 to 3.02 2112.00374 tab. 3.

Relaxed optimal-transport style loss, per-image optimisation (STROTSS)

The input is a content image and a style image, and no network is trained. The output starts from the style image's mean colour plus the content's high-frequency band, then is refined by RMSprop over a Laplacian pyramid at four scales, from 64 to 512 px on the long side 1904.12785 §2.5.

The style term is a relaxed earth mover's distance between sets of VGG16 hypercolumn features, with a cosine ground metric. Moment matching and a colour palette term are added 1904.12785 §2.2 1904.12785 eq. 7 1904.12785 eq. 9 1904.12785 eq. 10. The content term compares the pattern of pairwise feature similarities, not pixels. It keeps layout while pixel values change 1904.12785 §2.3 1904.12785 eq. 11. Card: Style Transfer by Relaxed Optimal Transport and Self-Similarity.

Users can pin a region of the content to a region of the style. The paper gives a cost weight of 5 for matched pairs and an infinite cost for mismatched ones 1904.12785 §2.4. The released entry point supports region masks only; see the card's Paper vs code section.

Author-reported, a 662-worker study covering three regimes (paired, unpaired and texture) finds higher style scores than four prior methods at a given content score. The text gives no numeric table 1904.12785 §4.1 1904.12785 fig. 8. The paper's timing table puts it at 95 s at 512 px, against 33 s for Gatys et al. 2016 in the same table [arxiv:1904.12785 tab. 1]. The relaxed transport distance is a lower bound on the exact one, at about 0.60 of its value 1904.12785 §4.3.

Compared methods with no card here

The ArtFlow card compares against StyleSwap, AdaIN, WCT, LinearWCT, OptimalWCT, Avatar-Net and Self-Contained 2103.16877 tab. 1. The Splice card compares against Swapping Autoencoders, WCT² and SinCUT, and against STROTSS, which has a card above 2201.00424 §4.1. Gatys et al. 2016 is the baseline in the STROTSS timing table and has no card either 1904.12785 tab. 1. None of these has a card in this wiki, so they are not described here beyond the numbers the two papers report. Their own papers are not used as sources.

• The Text2Mesh card compares against a VQGAN-CLIP adaptation to meshes, which the paper uses as its only baseline 2112.03221 §4.3. It has no card here.
• AdaIN is the cheapest of these in the author-reported timing, 0.064 s at 512×512 2103.16877 tab. 1. See adaptive-instance-normalization.
• Avatar-Net is the slowest, 9.129 s at 512×512 2103.16877 tab. 1. ArtFlow's Avatar-Net variant is shown qualitatively only 2103.16877 §5.3.

When to use which

Constraints behind the table:

• Training data: MS-COCO for content and WikiArt for style 2103.16877 §5.1. Splice trains on one pair per generator, and its evaluation uses AFHQ, Mountains and Wild-Pairs 2201.00424 §4. Text2Mesh needs a mesh and a prompt, and no dataset 2112.03221 §1.
• Diffusion guidance needs: a pretrained unconditional DDPM and a CLIP ResNet 50×16 image encoder, finetuned for noisy inputs 2112.05744 §4.1.
• STROTSS needs: a VGG16 network for its features 1904.12785 §2.1, and a GPU for the timings the paper reports, on a GTX 1080 [arxiv:1904.12785 §4.4].
• Splice needs: a frozen DINO ViT-B/8 and a GPU for each pair. The paper reports training on a single RTX 6000 2201.00424 app. A.2 2201.00424 app. A.3.
• Image2StyleGAN++ needs: a pretrained StyleGAN, whose weights are not trained by the method 1911.11544 §5, and VGG-16 features for the style and perceptual terms 1911.11544 §4.1. The paper's examples are mostly faces, with some LSUN cars and bedrooms 1911.11544 §9.4. It was run on one 32 GB V100 1911.11544 §6.
• Released weights: the ArtFlow card's Paper vs code section lists which checkpoints the repository provides. Check them before training from scratch.
• CLIPstyler needs: a pretrained CLIP model and VGG-19 weights for the content loss 2112.00374 §4.1. The paper does not name the CLIP model behind its CLIP score 2112.00374 app. B.3. The released script runs only on CUDA; see the card for that and for the fast variant's weights and data.
• Licence: the papers are CC BY 4.0. The released ArtFlow, diffusion, Splice and STROTSS code have no licence file, so their licence is unknown. Do not copy code from any of them without clearing that first. The Text2Mesh code is MIT, see its card. The CLIPstyler card records MIT for its code, but the repository's LICENSE file names a different copyright holder, so check that before copying anything; see its Paper vs code section. The Image2StyleGAN++ card records no official code, see its Paper vs code section.

Implementation sketch

Common core of the ArtFlow variants: forward pass on both images, latent transfer, reverse pass 2103.16877 §4.1. The transfer step is the only part that changes between AdaIN and WCT 2103.16877 §4.3.

# Forward flow on both images, lossless by design  [arxiv:2103.16877 §4.1]
def stylize(content, style, flow, transfer):
    z_c = flow.forward(content)
    z_s = flow.forward(style)
    # Transfer in latent space: AdaIN or WCT  [arxiv:2103.16877 §4.3]
    z_out = transfer(z_c, z_s)
    # Reverse pass maps the stylized latent back to pixels  [arxiv:2103.16877 §4.1]
    return flow.reverse(z_out)

# One flow step, forward  [arxiv:2103.16877 §4.2]
def flow_step_forward(x):
    # Actnorm: per-channel scale and shift  [arxiv:2103.16877 eq. 4]
    x = scale * x + bias
    # Learned c×c mixing of channels  [arxiv:2103.16877 eq. 3]
    x = conv_1x1(x, W)
    # Additive coupling: keep half, shift the other half  [arxiv:2103.16877 §4.2]
    x_a, x_b = split(x)
    x_b = x_b + net(x_a)
    return concat(x_a, x_b)


# Exact inverse, given the same parameters  [arxiv:2103.16877 §4.2]
def flow_step_reverse(y):
    y_a, y_b = split(y)
    y_b = y_b - net(y_a)
    x = conv_1x1(concat(y_a, y_b), inverse(W))
    return (x - bias) / scale

The paper does not write the inverse out, and says it can be derived 2103.16877 §4.2. The sketch above is that derivation applied to the same step. Parameter values for net, scale and the initial state are not stated in the paper; see Pitfalls.

The STROTSS style term is shown below. The content term and the coarse-to-fine loop are in the STROTSS card, under Method.

# Relaxed EMD with cosine ground metric  [arxiv:1904.12785 eq. 7] [eq. 9]
# out_feats: (n, d) output features; style_feats: (m, d) style features
def strotss_remd(out_feats, style_feats):
    # Pairwise cosine distances  [arxiv:1904.12785 eq. 9]
    cost = 1 - normalise(out_feats) @ normalise(style_feats).T
    # Mean over outputs of cheapest style match  [arxiv:1904.12785 eq. 8]
    out_side = cost.min(axis=1).mean()
    # Mean over style features of cheapest output  [arxiv:1904.12785 eq. 8]
    style_side = cost.min(axis=0).mean()
    # Relaxed value is the larger one-sided cost  [arxiv:1904.12785 eq. 7]
    return max(out_side, style_side)

# Splice loss for one structure and appearance pair  [arxiv:2201.00424 eq. 4]
# Training only: the frozen ViT computes the losses, the generator is updated
def splice_loss(G, dino, I_s, I_t):
    I_o = G(I_s)
    # Appearance: deepest CLS token of output vs appearance  [arxiv:2201.00424 eq. 5]
    loss_app = norm(cls(dino, I_t) - cls(dino, I_o))
    # Structure: deepest-layer key self-similarity  [arxiv:2201.00424 eq. 6]
    loss_struct = frob(selfsim(dino, I_s) - selfsim(dino, I_o))
    # Identity: keys of the appearance image kept by G  [arxiv:2201.00424 eq. 7]
    loss_id = frob(keys(dino, I_t) - keys(dino, G(I_t)))
    # Weights alpha = beta = 0.1  [arxiv:2201.00424 §3.3]
    return loss_app + 0.1 * loss_struct + 0.1 * loss_id

The masked style step of the local variant is shown below. The W+ and noise steps around it follow the same loop structure 1911.11544 alg. 6.

# Masked style step, optimises w with n fixed  [arxiv:1911.11544 eq. 4]
# Input: W+ start code, noise maps, style mask, style image y
def masked_style_step(G, w_ini, n_ini, mask_s, y, steps):
    w = w_ini
    # 1000 iterations in the §7.5 application  [arxiv:1911.11544 §7.5]
    for _ in range(steps):
        # Style term, weight 5e-7 in this step  [arxiv:1911.11544 §6]
        loss = 5e-7 * style_loss(mask_s, G(w, n_ini), y)
        # Adam unless the step says otherwise  [arxiv:1911.11544 §6]
        w = adam_step(w, loss)
    return w

# Style guidance during diffusion sampling  [arxiv:2112.05744 §3.1]
# Input: reference image, guidance scale; no content image
def style_guided_sample(model, reference_0, scale):
    # Start from standard Gaussian noise  [arxiv:2112.05744 alg. 1]
    x = sample_gaussian_noise()
    for t in reversed(range(T)):
        mu, sigma_sq = model(x, t)
        # Gradient of the Gram-based style score  [arxiv:2112.05744 eq. 9]
        grad = gradient_of(style_score, x, reference_0, t)
        # Shift the reverse-step mean  [arxiv:2112.05744 alg. 1]
        x = sample_gaussian(mu + scale * sigma_sq * grad, sigma_sq)
    return x

# Text2Mesh loss for one step  [arxiv:2112.03221 §3.2]
# Input: mesh vertices and normals, CLIP embedding of the prompt
def text2mesh_loss(field, verts, normals, target, views):
    # Colour and bounded offset per vertex  [arxiv:2112.03221 §3.1]
    colors, offsets = field(verts)
    # Move each vertex along its normal  [arxiv:2112.03221 §3.1]
    displaced = verts + offsets * normals
    # Coloured render and grey render of the same mesh  [arxiv:2112.03221 §3.2]
    full = render(displaced, colors, views)
    grey = render(displaced, gray, views)
    # Global term on the coloured render  [arxiv:2112.03221 eq. 2]
    s_full = mean(clip_embed(global_aug(full)))
    # Local term on the coloured render  [arxiv:2112.03221 eq. 3]
    s_local = mean(clip_embed(local_aug(full)))
    # Geometry-only term on the grey render  [arxiv:2112.03221 eq. 4]
    s_displ = mean(clip_embed(local_aug(grey)))
    # Sum of cosines, minimised as its negative  [arxiv:2112.03221 eq. 5]
    return -sum(cosine(s, target) for s in [s_full, s_local, s_displ])

The sketch leaves out the anchor-view choice and the view sampling; see the card's Paper vs code section for how the released code differs on those and on the weighting of the three terms.

The CLIPstyler patch term is a different kind of sketch. It shows the threshold rule and the averaging, which are the parts most often implemented wrongly.

# CLIPstyler patch term with threshold rejection  [arxiv:2112.00374 eq. 3]
# Input: 64 crops of the output, CLIP embedding of the content, text direction
def patch_term(crops, content_emb, text_dir, tau=0.7):
    losses = []
    for crop in crops:
        # Random perspective change, then CLIP embedding  [arxiv:2112.00374 app. A]
        d_img = normalise(normalise(clip_image(perspective(crop))) - content_emb)
        # One minus cosine similarity with the text direction  [arxiv:2112.00374 eq. 3]
        losses.append(1 - dot(d_img, text_dir))
    # Zero a patch whose loss is at or below tau  [arxiv:2112.00374 eq. 4]
    kept = [s if s > tau else 0 for s in losses]
    # Mean over all N patches, zeroed ones included  [arxiv:2112.00374 eq. 3]
    return sum(kept) / len(kept)

Contradictions

The papers report no results on the same quantity, so no numbers conflict. They do use the word "style" differently. In ArtFlow, style is AdaIN or WCT applied to a content image's latent 2103.16877 §4.3. In the diffusion paper, style is a Gram-matrix score that pulls a sample drawn from noise toward a reference 2112.05744 §3.3. In Image2StyleGAN++, style is a term on VGG-16 features that changes the W+ code of a fixed generator under a mask 1911.11544 §4.1 1911.11544 eq. 4. In Splice, the transferred quantity is the CLS token and key similarities of a frozen ViT, applied to objects in the structure image 2201.00424 §3.2. In Text2Mesh, style is a colour and a bounded normal offset on each vertex of a mesh, scored by CLIP similarity to a text prompt 2112.03221 §3.1 2112.03221 §3.2. In STROTSS, style is a relaxed optimal-transport distance between sets of VGG16 features, plus moment and palette terms 1904.12785 §2.2. Results from one do not transfer to the others.

CLIPstyler's threshold is described in opposite directions within the paper. The introduction and the threshold paragraph say that patches with high scores are rejected 2112.00374 §1 2112.00374 §3.2. Equation 4 zeroes a patch when its loss is at or below τ, so high-loss patches are kept 2112.00374 eq. 4. This page states both and does not choose between them. The released code follows the equation; see the CLIPstyler card.

Pitfalls

• Paper and code differ on training. The paper uses batch size 2 and 60,000 iterations 2103.16877 §5.1. The released code's defaults and scripts differ; the ArtFlow card's Paper vs code section lists each value. Use one source and state it.
• Loss form. The paper's two losses are written with L2 norms 2103.16877 eq. 1 2103.16877 eq. 2. The released code uses mean squared error instead, which changes the balance that the loss weights 0.1 and 1 set. The paper does not discuss this.
• Content target in training. The paper's content loss takes the AdaIN output as its target 2103.16877 eq. 1. The card records that the released code compares against the content image instead.
• Operator order. The text lists coupling first; the released code applies actnorm, then the 1×1 convolution, then coupling 2103.16877 §4.2.
• Metric details are missing. The paper does not state the test set for its metrics or which VGG layers its SSIM and loss use 2103.16877 tab. 1.
• Gram loss is not a ranking. Do not read Table 1 as an overall ranking. In that table OptimalWCT has the lowest Gram loss, and ArtFlow is not best on it 2103.16877 tab. 1.
• Style weight does not transfer. The paper's style score is a sum of squared Gram differences 2112.05744 eq. 9. The released code uses a normalised, averaged form with a large fixed multiplier that is not in the paper. A scale tuned on one will not carry over to the other; see the diffusion card's Paper vs code section.
• Gram statistics drop layout. A Gram matrix holds channel-by-channel inner products and no spatial positions 2112.05744 §3.3. Style guidance therefore steers texture and colour, not where objects sit.
• Style results are unmeasured. The diffusion paper's only quantitative table is for image and text guidance on FFHQ 2112.05744 tab. 1. Do not claim a style quality level from it.
• Results are author-reported. The ArtFlow comparison is on its own metrics and timing 2103.16877 tab. 1. The diffusion paper's Table 1 is on FFHQ only, and StyleGAN+CLIP has the highest retrieval scores, which the paper says come at the cost of low diversity 2112.05744 §4.2.
• Splice loss scale. The paper's weights give a total loss about a tenth of the released code's, with the same ratios. The paper does not say whether the absolute scale matters. See the Splice card's Paper vs code section for the other differences: norms, warm-up, training length and the augmentation probabilities 2201.00424 §3.3.
• STROTSS code differs from the paper. The paper's content weight starts at 16 and is halved at each of four scales. That gives 2 at the last scale, not the stated 1 1904.12785 §2.5. The released code also scales the content term by the number of sampled points, and differs on sampling, iterations and colour metric. See the card's Paper vs code section, and pick one source before tuning α.
• STROTSS guidance. The paper's point-to-point guidance is not reached by the released entry point, so use region masks only 1904.12785 §2.4.
• STROTSS results are random. Sampling and a random grid offset make each run stochastic, and no seeds are stated 1904.12785 §2.5.
• STROTSS has no numeric human-study table. The text gives no numbers for its 662-worker study, so do not quote a score for it 1904.12785 §4.1.
• Splice cost is per pair. Each new pair needs its own training run, about 20 min at 512×512, and the paper gives no inference-time budget 2201.00424 app. A.3.
• Splice metrics are small. The preference study has 65 images, and standard deviations reach 18 points 2201.00424 §4.1.2 2201.00424 tab. 1. The IoU table measures layout, not appearance quality 2201.00424 tab. 2.
• Local variant has no style metric. Its tables cover inpainting, edit realism and perceptual length, and reconstruction PSNR is reported separately 1911.11544 tab. 1 1911.11544 tab. 2 1911.11544 tab. 3 1911.11544 §7.1. Do not read any of them as a style quality score.
• Style weight is off in most steps. The style weight is 5e-7 in the style step and 0 in the W+ step 1911.11544 §6 1911.11544 eq. 2.
• Summed outputs are unclear. Algorithm 6 writes the W+ output as a sum of two results, and the text does not say how the two are combined 1911.11544 alg. 6.
• No official code. The Image2StyleGAN++ card's Paper vs code section lists the search that found no repository.
• Text2Mesh defaults are not the paper's setup. Running the released main script with its defaults gives a different iteration count, no learning-rate decay and no geometry term. The paper's values, and the demo flags that match them, are on the Text2Mesh card in its Paper vs code section 2112.03221 app. D.2.
• Text2Mesh CLIP scores are weak evidence. A degenerate result can score high 2112.03221 §4.2. Check outputs by eye, and keep the geometry term on, since the ablation shows the network can fake shape with shading 2112.03221 §4.2.
• Text2Mesh can erase the shape. A prompt unrelated to the object can remove its content. Naming the object category in the prompt helps 2112.03221 §4.6.
• Text2Mesh metrics are small. The user study has 57 users rating 8 pairs each. Standard deviations on the three questions run from 0.37 to 0.68 2112.03221 tab. 1.
• CLIPstyler threshold direction. Equation 4 zeroes patches at or below τ = 0.7, while the prose says high-scoring patches are rejected 2112.00374 eq. 4 2112.00374 §3.2. An implementation that follows the prose inverts the mechanism. Check the equation and the card's Paper vs code section before writing it.
• CLIPstyler patch mean. The patch term divides by all 64 patches, zeroed ones included 2112.00374 eq. 3. Averaging only the kept patches gives a different scale, and the loss weights were tuned with the paper's form.
• CLIPstyler CLIP score is close. The margin over AdaIN with CLIP is 0.003, 0.2515 against 0.2487 2112.00374 tab. 2. The score is not a style measure, and the paper does not name the CLIP model used for it 2112.00374 app. B.3.
• CLIPstyler baselines on faces. The StyleCLIP and StyleGAN-NADA comparisons use face images as content, since those generators were trained on faces 2112.00374 app. B.2 2112.00374 §4.3. Do not read them as results on general photos.
• CLIPstyler statistics. The user studies have 25 and 20 raters, and the paper gives means with no confidence intervals or significance tests 2112.00374 app. B.2 2112.00374 app. C.2.
• CLIPstyler fast variant differs from the paper. The paper says the fast model uses the same losses and decay as the per-image method 2112.00374 app. A. The released code uses a different content loss, learning-rate decay and output range; see the card's Paper vs code section.

Papers

• Style Transfer by Relaxed Optimal Transport and Self-Similarity: per-image optimisation with a relaxed optimal-transport style loss over VGG16 features and a self-similarity content loss, with optional region guidance.
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows: reversible flow in place of the encoder-decoder, with AdaIN or WCT in latent space, to stop content leak under repeated stylization.
• More Control for Free! Image Synthesis with Semantic Diffusion Guidance: Gram-matrix style score as one of several guidance types that steer a pretrained diffusion sampler at sampling time, with no retraining.
• Splicing ViT Features for Semantic Appearance Transfer: single-pair U-Net trained with losses from frozen DINO ViT features, so that objects in a structure image take on the look of semantically related objects in an appearance image.
• Image2StyleGAN++: How to Edit the Embedded Images?: embeds a photo into a fixed StyleGAN in two stages, then applies masked edits, including a masked style step, that stay local.
• CLIPstyler: Image Style Transfer with a Single Text Condition: a per-photo U-Net trained with patch-wise CLIP directional losses, restyling a photo from a text prompt with no style image.
• Text2Mesh: Text-Driven Neural Stylization for Meshes: a small network that colours a fixed mesh and offsets its vertices along the normal, trained with CLIP similarity on augmented multi-view renders.