StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows

In plain words

StyleGAN can draw very realistic faces from random numbers, but it has no dial for "older" or "smiling". The usual fix trains a new GAN that takes labels, which means a retrain and often worse images. StyleFlow keeps the pretrained GAN and learns a separate map on its internal code. That map is a normalizing flow: a model that moves points exactly and reversibly between a simple random distribution and the distribution of real codes. Its shape depends on the requested attributes, such as pose, lighting, expression, gender, age and facial hair.

To edit a photo, the method first finds the code that reproduces it. It asks the flow where that code sits in the simple distribution, given the photo's current attributes. Then it changes the attribute values and runs the flow forward again, which gives a new code that stays where the GAN can draw convincing images. Each edit touches only some of the GAN's input layers, so the rest of the picture tends to survive, and edits can be chained one after another.

TL;DR

StyleFlow fits one conditional normalizing flow between StyleGAN's W latent and a standard normal, conditioned on 17 face attributes. Sampling runs the flow in reverse with target attributes. Editing encodes with the source attributes, decodes with the target ones, and swaps only the edit's W+ rows back to the source. Author-reported gains over latent-arithmetic and linear baselines on identity and FID. The released code is non-commercial.

Problem

• GAN images are photorealistic, but attribute control normally needs labels at training time. Conditional GANs give that control at a cost in quality and require retraining §1 §2.
• Vector arithmetic in the GAN latent space can change attributes but also moves codes out of the training distribution and alters identity §1 fig. 2.
• Sequential edits make the problem worse: small entanglement errors accumulate §8.7.
• The method needs an attribute classifier to label training data and source images §3 §5.1.

Method

The paper sets two tasks. Attribute-conditioned sampling draws realistic images with target attributes. Attribute-controlled editing changes a given image toward target attributes while keeping its identity §3.

StyleGAN maps a 512-d Gaussian sample z to a 512-d intermediate vector w in W space. For editing, the paper uses the extended space W+, which stacks 18 copies of 512-d vectors, one per generator layer §3. The flow is trained on W, and W+ is used for edits and for real images §3.

Normalizing-flow background used by the method, as the paper states it §4:

p_w(\mathbf{w}) = p_z(\mathbf{z}) \left| \det \frac{\partial \phi}{\partial \mathbf{z}} \right|^{-1} eq. 3

\log p_w(\mathbf{w}) = \log p_z(\mathbf{z}_0) - \sum_{n=1}^{K} \log \det \left| \frac{\partial \phi_n}{\partial \mathbf{z}_n} \right| eq. 4

For planar flows the update is \mathbf{z}_{n+1} = \mathbf{z}_n + \mathbf{u}_n h(\mathbf{w}_n^\top \mathbf{z}_n + b) eq. 5, with log-density term \log p_w(\mathbf{w}) = \log p_z(\mathbf{z}_0) - \sum_{n} \log |1 + \mathbf{u}_n^\top \xi(\mathbf{z}_{n-1})| eq. 6 and \xi(\mathbf{z}) = h'(\mathbf{w}^\top \mathbf{z} + b)\,\mathbf{w} §4.1.

For continuous flows the state evolves as an ODE and the log-density change is an integral of a trace §4.2:

\mathbf{z}(t_1) = \mathbf{z}(t_0) + \int_{t_0}^{t_1} \phi(\mathbf{z}(t), t; \theta)\, dt eq. 7

\log p(\mathbf{z}(t_1)) = \log p(\mathbf{z}(t_0)) - \int_{t_0}^{t_1} \operatorname{Tr}\left(\frac{\partial \phi}{\partial \mathbf{z}(t)}\right) dt eq. 8

The paper states that its flow is conditional, with the attribute vector entering the velocity field §4.2 §5.2.

Steps

1. Dataset. Sample 10k z from the StyleGAN Gaussian. Map each to W, using truncation factor 0.7 as suggested for StyleGAN §5.1. Render the images. Label gender, pitch, yaw, eyeglasses, age, facial hair, expression and baldness with a face-attribute service, and lighting with DPR's nine spherical-harmonic coefficients. The attribute vector has 17 entries for faces §5.1.
2. Attribute-translation model. A stack of gate-bias blocks ("ConcatSquash") with 512-d layers: each layer's output is a linear map of the input, gated by a sigmoid of a context-dependent term, plus a context-dependent bias. The time variable is broadcast and concatenated to the attributes to form the context. Moving batch-norm layers sit before and after the CNF stack. The paper says the gate-bias output passes through tanh before the next stage §5.2 fig. 5; the code places tanh differently (see Paper vs code).
3. Joint attribute training. All 17 attributes enter one network together. The paper reports that training one attribute at a time learns entangled vector fields that change other attributes §5.2 fig. 12.
4. Training. Maximise the likelihood of W codes given attributes, using the continuous log-density change eq. 8. The trace is estimated with a Hutchinson estimator; the ODE is solved with dopri5 and the adjoint method §5.2 §5.3. Algorithm 1 gives the loop alg. 1.
5. Sampling. Draw z ~ N(0, I) and set w = Phi(z, a) eq. 1 §6.1.
6. Editing. Project the image to W with an Image2StyleGAN-style procedure §3 §6.2. Measure its attributes a_0. Recover the base latent z_0 = Psi(w_0, a_0) by reverse inference. Decode with the target attributes a_t. Apply the edit-specific W+ subset §6.2.1 §6.2.2 §6.2.3.
7. Subset selection. Each edit replaces only certain W+ rows with the edited vector and keeps the rest from the source §6.2.3. The indices are listed in the Hyperparameters table.
8. Sequential edits. Two modes: a fast one that does not re-project after each edit, and an accurate one that re-projects all 18 vectors after every edit §6.2. The paper says the fast mode can produce sudden jumps.
9. Interpolation. At inference, linear interpolation runs in attribute space rather than in latent space §5.2 §7.5.

The paper's equation 2 writes I_t = Phi(z_0, a_t), but Phi returns a latent, so the image is I(Phi(z_0, a_t)). We keep the paper's form below:

\mathbf{z}_0 = \Psi(\mathbf{w}_0, \mathbf{a}_0) = \Phi^{-1}(\mathbf{w}_0, \mathbf{a}_0), \qquad I_t = \Phi(\mathbf{z}_0, \mathbf{a}_t) eq. 2

# Flow training  [alg. 1] [§5.3] [code: train_flow.py#L84-L101]
# Input: W latents (B, 512), attributes (B, 17)
# Output: scalar loss

def training_step(latents, attributes):
    # Integrate latents through the flow, tracking the log-density change [§5.3]
    base_points, delta_log_p = flow(latents, attributes)
    # Log-density of each base point under N(0, I), summed over dims [§5.3]
    base_log_density = standard_normal_logprob(base_points)
    # Exact log-likelihood of each latent under the model [eq. 8] [§5.3]
    log_likelihood = base_log_density - delta_log_p
    # Maximise the mean log-likelihood by minimising its negative [§5.3]
    return -log_likelihood.mean()

# Attribute-conditioned sampling  [§6.1] [eq. 1] [fig. 6]
# Input: target attributes (17,), count N
# Output: N images

def sample_images(target_attributes, count):
    # Draw base noise from the standard normal [§6.1] [eq. 1]
    noise = standard_normal((count, 512))
    # Map noise to W under the target attributes [§6.1] [eq. 1]
    latents = flow_reverse(noise, target_attributes)
    # Decode latents with the StyleGAN generator [§6.1]
    return generator(latents)

# Attribute-controlled editing  [§3] [eq. 2] [§6.2]
# Input: image, target attributes (17,)
# Output: edited image

def edit_image(image, target_attributes):
    # Project the image to W; the projection code is not released [§3] [§6.2]
    w_source = project_to_w(image)
    # Measure the source attributes with the attribute models [§3] [§5.1]
    source_attributes = attribute_models(generator(w_source))
    # Reverse-encode to base noise under the source attributes [eq. 2] [§6.2.1]
    base_noise = flow_forward(w_source, source_attributes)
    # Decode the base noise under the target attributes [eq. 2] [§6.2.2]
    w_edited = flow_reverse(base_noise, target_attributes)
    # Replace only this edit's W+ rows; copy the rest from the source [§6.2.3]
    w_plus = copy_rows(w_source, w_edited, edit_rows)
    return generator(w_plus)

Hyperparameters

W+ edit subsets (replaced rows out of 0 to 17)

Results

Author-reported. Face setup: StyleGAN2 pretrained on FFHQ, 1024 x 1024, 70,000 images §7.1. Car setup: StyleGAN2 on LSUN-Car, 512 x 384, 16,185 images §7.1 fig. 17. Baselines: Image2StyleGAN (retrained on StyleGAN2 with the authors' re-implementation), InterFaceGAN (retrained on the same data), GANSpace (official code), StyleRig (numbers supplied by StyleRig's authors) §7.3. Identity is scored with a face-recognition model §7.2.2 [Fr]. Edit consistency uses an attribute classifier §7.2.3.

Sequential-edit realism, FID, lower is better (pose, light, expression edits; generated images) §7.5 tab. 1

Identity preservation on generated images §7.5 tab. 2. Cs = cosine similarity (higher is better). Ed = Euclidean distance (lower is better). V1 is without subset selection; V2 is the full method.

I2S = Image2StyleGAN, IG = InterFaceGAN, GS = GANSpace. Acc is the fraction of pairs the face classifier judges to be the same person.

Edit consistency under permuted edit orders, mean absolute error (lower is better) §7.5 tab. 3

The paper notes that for expression and gender the attribute classifier returns binary values, so every method scores zero on those §7.5.

Real-image identity preservation (StyleRig numbers are from its authors) §7.7 tab. 5. Columns: InterFaceGAN (IG), GANSpace (GS), StyleRig (SR), StyleFlow.

On real images, the paper says StyleRig scores better on the projection itself, and StyleFlow scores better on all edit types except expression §7.7.

Ablation over the number of CNF blocks §5.3 tab. 4

The paper says 2 and 3 blocks overfit, and that 6 blocks trade a slight loss of disentanglement for speed §5.3.

Qualitative results, described in words

• Sampling: females with glasses in a target pose; 50-year-old males with facial hair; smiling 5-year-old children in a target pose fig. 6 §6.1.
• Sequential edits on generated and real faces, including pose, lighting, expression, gender, age, eyeglasses and baldness; the edit order is shown not to change quality fig. 7 fig. 8 fig. 9 §7.4.
• Attribute transfer: pose, expression and lighting from one target, and eyeglasses, facial hair, age and gender from another fig. 10.
• Non-linear versus linear paths: the StyleFlow path keeps hair, clothing and head coverings longer than a linear path in W fig. 11 §7.5.
• Shared versus per-attribute training: joint training keeps identity and unedited attributes better fig. 12 §7.6.
• Subset selection: the background is better preserved with the block fig. 13 §7.6.
• Real-image comparisons against the baselines fig. 14 fig. 15.
• User study: 25 participants compared methods pairwise on visual quality and identity preservation; the paper reports StyleFlow preferred across the comparisons fig. 16 §7.7.
• Cars: sequential SUV or hatchback conversion, rotation and colour change fig. 17 §7.8. The paper reports accuracy 95% for the car recognizer, 80% for the hatchback or SUV conversion and 100% for colour. Rotation is shown only visually §7.8.
• Interface: sliders and checkpoints for sequential edits fig. 18 §7.10.

Paper vs code

Code: RameenAbdal/StyleFlow at commit 53a8c54c7f93fc97e1a8dc5893947e4c9df77d58 (cloned at depth 1). The owner matches the first author, and the project page links it [abstract]. Checked against main.py, main_attribute.py, train_flow.py, module/flow.py, module/cnf.py, module/odefunc.py, module/diffeq_layers.py, module/normalization.py, and README.md.

• Epochs. The paper gives 10 epochs §5.3. train_flow.py defaults to 3 train_flow.py.
• Hutchinson estimator. The paper says 10 evaluations of the trace estimator §5.2. The code samples one Gaussian probe per ODE solve and reuses it across that solve's function evaluations, drawing a fresh one each forward pass odefunc.py odefunc.py.
• Model depth. The paper calls the selected model "4 stacked CNF functions" §5.2 tab. 4. The code's training default and the pretrained model used by main.py use five hidden widths of 512 with num_blocks=1 train_flow.py main.py flow.py. How these counts map to each other is not stated, and the parameter counts in the paper were not recomputed.
• Tanh. The paper says the output of the gate-bias stack passes through tanh before the next stage §5.2. The code applies tanh between hidden layers only; the final velocity layer has no activation odefunc.py flow.py.
• Time direction. The paper's Algorithm 1 writes z(t1) = w and integrates from t1 to t0 alg. 1. The code feeds w with integration times [0, T] into the forward flow, which produces the base latent train_flow.py cnf.py. The reverse direction gives sampling and editing main.py main.py.
• End time. The paper does not give the integration times alg. 1. The code makes the end time a learned parameter initialised at 1.0 cnf.py flow.py.
• Edit subsets. Glasses and facial hair differ from the paper, as the table above shows §6.2.3 main.py main.py.
• Truncation. The paper uses factor 0.7 for dataset preparation §5.1. The dataset-preparation code is not in the repository. The included generators default to 0.5 main.py run_generator.py.
• Real-image projection. The paper uses a re-implementation of Image2StyleGAN in W+ §6.2 §8.9. The README says the projection method is under review and not included [README.md]. Only the editing side is in the repository.
• Training order and seed. The paper does not say whether data are shuffled. The code sets shuffle=False and seeds with 0 train_flow.py train_flow.py.
• Batch-norm constants. The paper does not state them. The code uses eps 1e-4 and decay 0.1, with weight and bias initialised to zero normalization.py normalization.py.
• Adam defaults. The paper's "default" Adam parameters match PyTorch's defaults train_flow.py.
• Attribute names. The UI order is Gender, Glasses, Yaw, Pitch, Baldness, Beard, Age, Expression, then six lighting names main.py. The paper's "facial hair" is the code's "Beard".

Pitfalls

• Licensing conflict. The paper is CC BY 4.0. The code is marked CC BY-NC-SA 4.0 (non-commercial) in its README, with no LICENSE file. The README calls the code "for academic research use only" [README.md]. This card has no code excerpts; the pseudocode is our own, and the code citations point to the repository. The card still names a non-commercial code licence, and a human should decide whether code-derived pseudocode and the comparison are acceptable on a commercial site.
• Equation 2 notation. I_t = Phi(z_0, a_t) writes an image as the output of Phi, which returns a latent w. Decode with the generator after Phi.
• Attribute classifier dependence. Training labels come from the Microsoft Face API and DPR §5.1. The paper does not say how well this transfers to other classifiers, and it does not state the API version. Reproduction needs equivalent labels.
• Hutchinson variance. A single probe per solve makes the loss stochastic. Reported numbers depend on that noise and on the seed; the paper gives no variance.
• Hidden numerical sensitivity. Both solves use tolerance 1e-5 with dopri5 and adjoint gradients. The paper reports inference times of 0.61 s and 0.21 s without saying what one timed call covers, and a training rate of 1.07 to 2.5 iter/s on a Titan XP §5.3.
• Data shape ambiguity. train_flow.py applies log-det per row with zeros(batch, x[0].shape[2], 1) and view(batch, x[0].shape[2], 1).sum(1) train_flow.py. That suggests a row axis, which would match W+ rather than the single 512-d W vector in the paper §5.1. The data file was not parsed, so this was not resolved; check the shapes before implementing.
• Log-likelihood scale. The code sums log-density over dims per sample train_flow.py, so the reported values in the millions are sums over the latent dimensions, not per-dimension averages tab. 4.
• Reprojection modes. The fast mode skips re-projection after each edit; the paper says it can cause sudden jumps. The accurate mode re-projects all 18 vectors §6.2. Pick one deliberately.
• Edit-dependent identity. Edits depend on the source latent by design §8.2. Two images with the same target attributes get different edit paths, so there is no single edit vector to cache.
• Cars. Car results are qualitative except the accuracy numbers for a recognizer; the rotation edit is not quantified §7.8.
• Numbering. Equation, figure and table numbers were counted from the LaTeX source in order; the rendered PDF was not checked. The appendix is counted as §10 because the source uses \section{Appendix} without \appendix. Cite by section if a number is off.
• Attribution. Results are author-reported and include baselines the authors re-implemented or retrained §7.3. The user study had 25 participants and 18 real images §7.7.

Lineage

• Builds on: StyleGAN and StyleGAN2 for the generator and W space §3 (karras2019analyzing in the paper's bibliography); continuous normalizing flows, FFJORD and neural ODEs §4.2; gate-bias layers from FFJORD and PointFlow, with moving batch norm from PointFlow §5.2; Image2StyleGAN for projection §2 §6.2.
• Compared against: Image2StyleGAN, InterFaceGAN, GANSpace and StyleRig §7.3.
• Superseded by: not stated in this wiki.