Fréchet Inception Distance (FID)

Overview

FID compares two sets of images with no pairing between them. Features from a pretrained Inception-v3 network are computed for the real set and the generated set. Each set is summarised by a Gaussian fit, and the score is the Fréchet distance between the two Gaussians, which the survey describes as a Wasserstein-2 distance 2103.09396 §1.1. Because the Gaussian fits use the full feature covariance, the score reflects both the centre and the spread of the two sets.

The score has known limits. It is biased, it needs many samples (the survey gives 50K as a usual size, not a rule, and says small samples over-estimate it) 2103.09396 §1.1. It combines quality and diversity into one number, so it does not say which of the two has failed 2103.09396 §1.1. It can be raised by memorising training images 2103.09396 §4.1. Its result also depends on the feature network and on image preprocessing, as the variants below show.

The survey is the only source on this page. It is a review that runs no experiments, so the numbers below are the ones it quotes from cited works and have not been checked here. Each variant is linked to the same card.

Variants

Standard FID

The base score as described above: pooled Inception-v3 features, one Gaussian per set, and the Fréchet distance between them 2103.09396 §1.1. The survey gives neither the formula in full, nor the feature layer, nor the image preprocessing. Card: Pros and Cons of GAN Evaluation Measures: New Developments.

Spatial FID (sFID)

Adds intermediate features to the pooled ones. The survey lists the features as the pooled pool3 layer plus the first seven channels of the mixed 6/conv layer 2103.09396 §2.1.1. The survey does not give the reason for the choice of layers. Card.

Class-aware FID (CAFD)

Computes a Fréchet distance for each class and averages them over K classes 2103.09396 §2.1.2. It needs class labels for the generated and real images. The value of K depends on the dataset and is not stated in the survey. Card.

FastFID

An optimisation of FID that the survey describes as cheap enough to use as a training loss 2103.09396 §2.1.3. The cited work reports the computation as 25 to 500 times faster than standard FID, depending on the number of generated samples 2103.09396 §2.1.3. Card.

Memorization-informed FID (MiFID)

Multiplies FID by a penalty that is large when generated images are close to training images 2103.09396 §2.1.4. The closeness measure is a cosine-based memorization distance, and the penalty switches on below a threshold τ. The survey does not state τ, the small constant ε, or the feature space used for the cosine similarity 2103.09396 §2.1.4. Card.

Bias-corrected FID and IS

Removes the finite-sample bias of FID and Inception Score by extrapolation. The survey notes that the bias depends on both the number of images and the generator 2103.09396 §2.1.5. Use this when the sample count is fixed and below the usual size. Card.

Clean FID

A standard preprocessing protocol. The survey reports that the choice of resize library, interpolation kernel and image encoding can change FID substantially 2103.09396 §2.1.6. Using the same protocol removes one source of difference between two scores. Card.

Fréchet Video Distance (FVD)

The video counterpart, with features from a video network. The survey gives the feature network as I3D, pretrained on Kinetics-400 and Kinetics-600 2103.09396 §2.1.7. For the video setting and its human study, see generative-model-evaluation. Card.

Self-supervised features

Replaces the ImageNet classifier features with self-supervised features in FID and related scores 2103.09396 §2.2. The survey lists this as an option for the bias that Inception-based scores inherit from ImageNet. It gives no head-to-head comparison between this option and the others [arxiv:2103.09396 §1.1] §2.2. Card.

When to use which

Whatever the variant, report the sample count, the feature network and the preprocessing. The survey treats all three as changing the score 2103.09396 §1.1 §2.1.6.

Implementation sketch

The core is the same for every variant: fit two Gaussians to two feature sets and take the distance. The variants change which features are used, how many there are, or what is multiplied or averaged afterwards.

# Fréchet distance between two feature sets  [arxiv:2103.09396 §1.1]
# Input:  real features x_real (n x d), generated features x_gen (m x d)
# Output: scalar; the survey describes it as W2 between Gaussians

def frechet_distance(x_real, x_gen):
    # Fit one Gaussian to each feature set  [arxiv:2103.09396 §1.1]
    mu_r, cov_r = mean(x_real), covariance(x_real)
    mu_g, cov_g = mean(x_gen), covariance(x_gen)
    # Squared distance between the means  [arxiv:2103.09396 §1.1]
    mean_term = sum((mu_r - mu_g) ** 2)
    # Covariance term; the survey does not say how to compute the
    # matrix square root  [arxiv:2103.09396 §1.1]
    cov_term = trace(cov_r + cov_g - 2 * sqrtm(cov_r @ cov_g))
    return mean_term + cov_term

Where the variants differ: sFID uses intermediate features as well as the pooled ones 2103.09396 §2.1.1; CAFD calls the function once per class and averages 2103.09396 §2.1.2; MiFID multiplies the result by a memorization penalty 2103.09396 §2.1.4. The feature network and the image preprocessing are not part of this sketch and are not stated in the survey 2103.09396 §1.1 §2.1.6.

Contradictions

None known between sources; the page has one. Within the survey, the class-aware formula prints the mean term as an unsquared norm, and the survey does not say whether that is intended 2103.09396 §2.1.2. The sketch above uses the squared form implied by the Wasserstein-2 description. Check the original work before implementing CAFD.

Papers

• Pros and Cons of GAN Evaluation Measures: New Developments: a survey of FID variants and other GAN evaluation measures, with no new experiments.