Are GAN generated images easy to detect? A critical analysis of the state-of-the-art

In plain words

Image generators built from GANs can now make pictures that look real. Tools that try to tell generated pictures from camera photos often look very good on the kind of generator they were trained on. This paper asks how well they hold up when the generator is new, or when the picture has been compressed or resized, as happens on social networks.

The authors take seven published detectors, train them on one set of generated images, and test them on images from other generators and in harder conditions. Several detectors lose a lot of accuracy once images are compressed or resized. The best variant they found is a standard image network that skips early downsampling, and it holds up well across the generators they tested.

The paper is a benchmark rather than a new product, and its results are the authors' own. The code they released is for inference only.

TL;DR

• Benchmarks seven published GAN-image detectors, all trained on ProGAN images and tested on generators absent from training §3 §4.
• Several detectors degrade under JPEG compression and resizing, and 2x downsampling is catastrophic §4 fig. 3.
• The best variant is the Wang2020 baseline with no down-sampling in the first layer. Its accuracy is above 90% on every test set in tab. 3 (author-reported).
• The paper proposes no new architecture beyond that variant. Its training details (optimizer, schedule, epochs) are not stated §4.

Problem

Detectors seem to find GAN images easy to spot, but the authors argue that this is overly optimistic. Performance drops sharply when training data is not aligned with the test data and when images are distorted §1. Routine compression and resizing on the web destroy the traces these detectors rely on §1. Screening also needs low false-alarm rates, and accuracy at a fixed threshold is not enough for that purpose §4.

Method

This is an empirical study. It defines a common training set, a set of unseen test generators, and a set of robustness tests, then evaluates existing detectors and a few variants of one of them.

1. Training and validation data §3

• Real images: 362K from LSUN.
• Fake images: 362K from 20 ProGAN models, one per LSUN object category.
• All training images are 256x256. A 4K-image subset is used for validation.
• The training set is the one provided by Wang et al. (2020) §3.

2. Test data §3 tab. 1

• The test generators were never seen in training. Low-resolution ProGAN images are excluded, because ProGAN is in training. Their high-resolution versions are used instead.
• High-resolution real images from CelebA-HQ are not used, because the authors consider them GAN-upsampled versions of low-resolution real images §3.
• The paper states about 39K synthetic images in total §3. The row sums of tab. 1 give 42.0K (see Pitfalls).

Source: tab. 1.

3. Detectors compared §4 tab. 2

Source: tab. 2.

4. Metrics §4

• AUC.
• Accuracy at a fixed threshold of 0.5.
• Probability of detection at 5% false-alarm rate (Pd@5%) and at 1% (Pd@1%).

5. Robustness tests §4 fig. 3

• Each detector is evaluated on images with JPEG compression at several levels, and on images resized by several factors. Pd@5% is the reported metric.
• The levels and factors are not given in the text. They appear on the axes of fig. 3.

6. Ablation on the Wang2020 baseline §4 fig. 4 fig. 5 tab. 3

The baseline is the Wang2020 model. The authors vary it as follows.

• No ImageNet pre-training ("no-pretrain").
• An initial layer for residual extraction ("residual").
• No down-sampling in the first layer, as suggested by Boroumand et al. ("no-down").
• Stronger augmentation ("strong-aug"): Gaussian noise, geometric transforms, cut-out, and brightness and contrast changes.
• For the no-down variant only, two other backbones: Xception ("Xception no-down") and EfficientNet-B4 ("Efficient no-down").

The text gives no numbers for the first three variants or for strong-aug. Their results appear only in fig. 4 and fig. 5. The no-down variant is the one reported in tab. 3, in two versions: trained on ProGAN, and trained on 23 StyleGAN2 models §4.

7. Pd at a fixed false-alarm rate §4

The paper does not say how the decision threshold for Pd is set. The sketch below uses real images, which is the usual choice.

# Pd at a fixed false-alarm rate  [§4]
# Input:  scores on real and fake images, false-alarm rate (e.g. 0.05)
# Output: share of fake images detected at that rate
# The paper does not say which real images set the threshold; this
# sketch uses real images, the usual choice.

def pd_at_far(real_scores, fake_scores, far):
    # Threshold such that the given share of real images scores above it
    threshold = quantile(real_scores, 1 - far)
    # Share of fake images above the threshold
    return mean(fake_scores > threshold)

8. The no-down detector §4 resnet50nodown.py

The detector is a ResNet-50 with the first layer changed. The released code is inference only (see Paper vs code). The pseudocode follows the released implementation.

# No-down ResNet-50 as released (inference only)
# Input:  images (B, 3, H, W), values in [0, 1]
# Output: one logit per image (B, 1)

def detect(images):
    # Normalise with the ImageNet mean and std
    # [code: resnet50nodown.py#L108-L110]
    x = (images - mean) / std                    # (B, 3, H, W)

    # Stem: 7x7 conv and 3x3 max-pool, both with stride 1
    # (paper: "first layer" only) [§4] [code: resnet50nodown.py#L87-L90]
    x = conv7x7(x, stride=1)                     # (B, 64, H, W)
    x = max_pool(x, kernel=3, stride=1)          # (B, 64, H, W)

    # Four residual bottleneck stages; strides 1, 2, 2, 2
    # [code: resnet50nodown.py#L91-L94]
    x = residual_stages(x, blocks=[3, 4, 6, 3])  # (B, 2048, H/8, W/8)

    # Global average pooling, then one linear unit
    # [code: resnet50nodown.py#L96-L98]
    x = global_average_pool(x)                   # (B, 2048, 1, 1)
    return linear(x)                             # (B, 1)

Hyperparameters

Results

Author-reported. Setup: the models are trained on ProGAN, or on StyleGAN2 for the last column, and tested on the generators in tab. 1. The baseline is Wang2020 §4. The table lists accuracy at threshold 0.5 and Pd@5% for each test set, written as accuracy / Pd@5%. The underlines in the source table are omitted.

Source: tab. 3.

Other findings, in the authors' words and figures:

• On uncompressed images at original size, AUC is generally very good for low-resolution images, and several methods exceed 0.9. Accuracy at threshold 0.5 is much lower. Pd at low false-alarm rates is poor for most methods, and only some keep a good detection ability §4 fig. 2.
• High-resolution results are somewhat better, with the same general pattern §4 fig. 2.
• Several methods suffer dramatic impairments once images are compressed or resized. Methods trained with augmentation are more stable. A 2x downsampling has catastrophic effects §4 fig. 3.
• Against the Wang2020 baseline, the authors report that avoiding down-sampling in the first block gives an average gain of about 15% in accuracy and about 14% in Pd@5% §4. See Pitfalls for a recomputation from tab. 3.
• The authors describe these experiments as very limited and preliminary §4.

Paper vs code

The official repository is grip-unina/GANimageDetection, linked from the paper's footnote §4. It was cloned at commit 543943cdf281df7417751e794109431d0975df88 and the license is recorded above. The repository contains only the ResNet-50 no-down model and a script that scores a folder of images. The training code, the data pipeline and the augmentation code are not in it README.md main.py.

• Default weights are the StyleGAN2-trained model. The script's default weights file is gandetection_resnet50nodown_stylegan2.pth main.py. The README says that weights trained on ProGAN also exist README.md. Running the default therefore reproduces the StyleGAN2 column of tab. 3, not the ProGAN column. The ProGAN weights file name is not given in the repository.
• Weights are not in the repository. They are downloaded from an external host, and no checksum is given README.md.
• Stem stride. The paper says the first layer does not down-sample §4. The code sets the stride to 1 for both the 7x7 convolution and the max-pool, so the first four stages all run at a higher resolution than a standard ResNet-50 resnet50nodown.py resnet50nodown.py.
• Training-set size. The paper says the best variant is trained on 23 StyleGAN2 models §4. The README describes a StyleGAN2 training set of 720K fake images and 11 pristine datasets, and does not give a model count README.md. The two are not reconciled.
• Output. The code returns a raw logit per image and never applies a sigmoid or threshold resnet50nodown.py main.py. The paper reports accuracy at a fixed threshold of 0.5 §4, and the code does not say how that threshold maps onto the logit.
• Input normalisation. The code applies the ImageNet mean and standard deviation resnet50nodown.py. The paper does not state a normalisation.
• Large images. The code tiles images larger than 1536 px on both sides into 1024 px tiles and averages the logits, weighted by tile area resnet50nodown.py resnet50nodown.py. The paper does not mention this. The 1024x1024 test images are below the threshold, so the tiling does not change the paper's high-resolution results.
• Not checkable. Optimizer, learning rate, schedule, batch size, epochs, augmentation parameters and the Pd threshold source are not in the paper or the repository.

Pitfalls

• The threshold of 0.5 is ambiguous. The paper gives accuracy at 0.5 §4, but the score it applies to is not defined. The code returns a logit with no stated sign convention resnet50nodown.py. Accuracy at 0.5 can't be reproduced from the released code without an assumption about the scale.
• Pd is defined by a false-alarm rate, and the threshold source is not stated §4. Compute it on real images, as the pseudocode does, and say so in any reproduction.
• Never rescale inputs. A 2x downsampling is catastrophic §4 fig. 3. Preprocessing that resizes would destroy the signal the detectors use.
• Compute and memory cost of the no-down stem. Compared with a standard ResNet-50, each residual stage in the code runs at four times the spatial resolution per side, so the first stage has sixteen times as many pixels resnet50nodown.py. The paper reports no cost figures §4.
• Averages in the text do not match tab. 3. The paper says the gain is about 15% in accuracy and 14% in Pd@5% §4. Recomputing from tab. 3: accuracy rises by 16.5 points on average across the eight low-resolution rows and by 13.7 points across all twelve rows. Pd@5% rises by 16.6 and 13.3 points. The paper does not say which rows it averaged.
• Fake-image count does not match. The text gives about 39K synthetic images §3. The row values in tab. 1 sum to 42.0K (30.0K low-resolution and 12.0K high-resolution).
• One test row is not unseen. The high-resolution ProGAN row is from the same generator family as training §3 tab. 3. The "unseen generator" claim does not cover it.
• Real test images differ from training real images. Training uses LSUN real images, and the test real images are ImageNet, COCO, the unpaired real dataset and RAISE §3. The authors call the training and test data "not aligned" §4, so the measured gap mixes generator and content effects.
• Some parameters are missing. The strong-augmentation settings §4 and the compression and resize levels fig. 3 are not given in the text. Replicating the robustness results requires the figure axes or the authors' scripts.
• Test strategies differ per method tab. 2. Spec uses a central crop, M-Gb and PatchForensics resize, and the others use the full image. A fair comparison needs each method's own strategy.
• The "above 90%" claim is limited. It is about the no-down variant trained on ProGAN tab. 3 §4, not about the other methods or the other variants.
• The results are the authors' own and they describe them as very limited and preliminary §4. There is no independent replication in the sources.
• Licence of the code. The repository's licence allows use only for informational and nonprofit purposes, and prohibits profit-oriented use LICENSE.md. This card therefore uses no code verbatim. Whether the terms also cover the downloadable weights is not stated.

Lineage

• Builds on: the Wang et al. (2020) baseline, its training set and its augmentation §3 §4. The idea of removing down-sampling in the first layer comes from SRNet, Boroumand et al. (2019) §4 tab. 2.
• Compared against: Xception (Marra et al., 2018), SRNet (Boroumand et al., 2019), Spec (Zhang et al., 2019), M-Gb (Xuan et al., 2019), Co-Net (Nataraj et al., 2019), Wang2020 (Wang et al., 2020) and PatchForensics (Chai et al., 2020) tab. 2. No cards for these works exist in this wiki yet, so they are named without links.
• Superseded by: not stated in the paper or the repository.