wikillm

Paper · 2022-02-07 · cs.CV, cs.LG, eess.IV

FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations

Detects GAN-made images with a ResNet-50 that sees each input plus a learned frequency-domain perturbation. FrePGAN makes the perturbation from the FFT with an adversarial loss and a compression loss, trained alongside the classifier. On ProGAN with four categories it reports 79.4% mean accuracy and 87.2 mean AP over eight unseen test sets. Author-reported.

Image manipulation detectionGenerative adversarial network (GAN)Frequency-domain learningDomain generalizationdeepfake-detectiongan-detectionfrequency-domainadversarial-perturbation

arXivPDFMarkdown14 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Fake pictures made by image generators often carry a faint pattern in their frequency content, the fine-grained texture that a spectrum analysis reveals. A detector that learns this pattern does well on the generator it was trained on and poorly on others, because each generator leaves a different pattern.

The idea here is to make the training pictures less revealing before the detector sees them. A small network learns a faint, frequency-based pattern that it adds to every picture, real or fake. The added pattern hides the generator's fingerprint, so the detector has to find other clues. The network that adds the pattern is trained against a discriminator, the same way a generative adversarial network is trained, while a second rule keeps the added pattern small. The detector and the pattern network are trained at the same time, so the detector keeps learning new clues as the pattern changes.

The point is a detector that still works on image generators it has not seen.

TL;DR

Deepfake detectors trained on one generator often fail on others. FrePGAN trains a frequency-based perturbation generator, adversarially and with a compression loss, together with a ResNet-50 classifier that sees each image plus the generated perturbation. The authors report better generalisation than four baselines on unseen GANs, manipulations, resolutions and categories. All numbers are author-reported.

Problem

Detectors fit the frequency artifacts of the generator used in training, which vary with the GAN architecture and the object category. Because of this they overfit and fail to generalise § Introduction. The paper says that the key to generalisation is to reduce the effect of these artifacts during training rather than to exploit them § Introduction. The paper's test scenarios vary the GAN model, colour and geometric manipulations, object category and image resolution § Introduction § Experiments (Deepfake Detection Performance).

Method

The paper's sections are unnumbered (\setcounter{secnumdepth}{0}), so this card cites them by title. [§ Method] means the paper's "Deepfake Detection Framework" section and [§ Experiments] means "Experimental Results". The subsection name follows in parentheses.

Components. FrePGAN is a generator G and a discriminator D § Method (Frequency Perturbation GAN). The deepfake classifier C is a ResNet-50 § Method (Deepfake Classifier). Real and fake images are both perturbed by G before C sees them § Method (Framework).

Training. G, D and C are trained in one loop, not in sequence. Early on, C learns the easy frequency artifacts; later, as G learns to hide them, C must find more general features § Method (Training of Deepfake Detection Framework).

Perturbation generator. G takes the image x of shape w×h×c § Method (Frequency Perturbation GAN).

  1. The image is transformed by a 2-D FFT. The real and imaginary parts of each channel are stored as separate channels, so the frequency map has 2c channels § Method (Perturbation Map Generator).
  2. A frequency-level generator H, which the paper describes as an encoder-decoder like image-to-image GANs, maps the frequency map to a map of the same size § Method (Perturbation Map Generator).
  3. An inverse FFT returns a pixel-level perturbation of size w×h×c eq. 1 § Method (Perturbation Map Generator).

Discriminator. D follows the DCGAN discriminator: strided convolutions and a final binary classifier § Method (Perturbation Discriminator).

Losses. G minimises L_G, a weighted sum of an adversarial term and a compression term, with weight λ = 0.5 eq. 3 § Method (Training of FrePGAN). The adversarial term L_adv is applied to G(x) eq. 4. The compression term L_com is the squared L2 norm of G(x) eq. 5. D is trained on real images and on x + G(x) eq. 6. The overall objective is written as eq. 2, but training updates G and D separately eq. 2 alg. 1.

Classifier. The classifier input is A_G(x) = x + G(x) eq. 7 § Method (Input of Deepfake Classifier). C is trained with binary cross-entropy on A_G(x) eq. 8 § Method (Training of Deepfake Classifier).

Algorithm. Each mini-batch computes L_adv, L_com, L_D and L_C, then updates G, D and C. The loop runs until 20 epochs have passed alg. 1. The pseudocode below follows that order. Details the paper does not state are marked in the code comments and in Pitfalls.

Inference. For a test image x', the paper computes x' + Ĝ(x'), the trained generator's perturbed image, and passes it to Ĉ; the output is real or fake § Method (Deepfake Image Prediction). The decision threshold is not stated.

Key equations, reproduced in full:

G(x) = \mathcal{F}^{-1}\big( H( \mathcal{F}(x) ) \big) eq. 1

\widehat{G}, \widehat{D} = \arg_{G, D} \min_G \max_D \mathcal{L}_{G} + \mathcal{L}_{D} eq. 2

\mathcal{L}_G = \lambda\,\mathcal{L}_{adv} + (1 - \lambda)\,\mathcal{L}_{com} eq. 3

\mathcal{L}_{adv} = \mathbb{E}_{x \sim \mathbf{X}}\big[ \log\big( 1 - D(G(x)) \big) \big] eq. 4

\mathcal{L}_{com} = \mathbb{E}_{x \sim \mathbf{X}}\big[ \| G(x) \|_2^2 \big] eq. 5

\mathcal{L}_D = \mathbb{E}_{x_r \sim \mathbf{X}_r}\big[ \log D(x_r) \big] + \mathbb{E}_{x \sim \mathbf{X}}\big[ \log\big( 1 - D(x + G(x)) \big) \big] eq. 6

A_G(x) = x + G(x) eq. 7

\mathcal{L}_C = \mathbb{E}_{(x,y) \sim (\mathbf{X},\mathbf{Y})}\big[ y \log C(A_G(x)) + (1-y) \log(1 - C(A_G(x))) \big] eq. 8

Pseudocode, in the form of the paper's Algorithm 1 alg. 1. The generator forward pass is first:

Pseudocode
# Perturbation map G(x) = F^-1(H(F(x)))  [eq. 1]
# [§ Method] (Perturbation Map Generator)
def perturbation_map(images):
    # FFT per channel; real and imaginary parts stacked
    # [§ Method] (Perturbation Map Generator)
    spectrum = fft_real_imag(images)           # (B, 2c, h, w)
    # Frequency-level generator H, same size in and out
    # [§ Method] (Perturbation Map Generator)
    spectrum_out = H(spectrum)                 # (B, 2c, h, w)
    # Inverse FFT to pixel space; 2c -> c step not stated
    # [§ Method] (Perturbation Map Generator)
    return inverse_fft(spectrum_out)           # (B, c, h, w)

The training loop follows the algorithm:

Pseudocode
# Training loop  [alg. 1]
# Input: labelled batches; G, D random; C from ImageNet weights  [alg. 1]
def train(dataset):
    # Repeat until 20 epochs  [alg. 1] [§ Method] (Implementation Details)
    for epoch in range(20):
        # Mini-batches of size 16  [§ Method] (Implementation Details)
        for images, labels in dataset:
            train_step(images, labels)
Pseudocode
# One mini-batch step  [alg. 1]
def train_step(images, labels):
    # Forward pass: all losses from the same batch  [alg. 1]
    perturbation = G(images)                    # (B, c, h, w)
    # Adversarial term on the perturbation  [eq. 4]
    loss_adv = mean(log(1 - D(perturbation)))
    # Compression term, squared L2 norm  [eq. 5]
    loss_com = mean(squared_norm(perturbation))
    # Generator loss, lambda = 0.5  [eq. 3] [§ Method] (Training of FrePGAN)
    loss_g = 0.5 * loss_adv + 0.5 * loss_com
    # Discriminator loss on real images and perturbed fakes  [eq. 6]
    loss_d = (
        mean(log(D(real(images))))
        + mean(log(1 - D(images + perturbation)))
    )
    # Classifier loss on A_G(x) = x + G(x)  [eq. 7] [eq. 8]
    loss_c = bce(C(images + perturbation), labels)
    # Update G (Adam, lr 1e-4), then D (maximise L_D), then C (Adam, lr 1e-4)
    update(G, minimise(loss_g))
    update(D, maximise(loss_d))
    update(C, minimise(loss_c))

The values 0.5, 16, 20, 1e-4 and 1e-1 come from § Method (Training of FrePGAN) and (Implementation Details); see Hyperparameters.

Hyperparameters

NameValueWhereNotes
Optimiser (G, D, C)Adam§ Method (Implementation Details)betas, epsilon, weight decay not stated
Learning rate, G1e-4§ Method (Implementation Details)
Learning rate, D1e-1§ Method (Implementation Details)Printed as 10^{-1}; 1000 times G's rate. Possibly a typo; not stated
Learning rate, C1e-4§ Method (Implementation Details)schedule not stated
λ (weight of L_adv in L_G)0.5§ Method (Training of FrePGAN) eq. 3L_com weight is 1 − λ
Batch size16§ Method (Implementation Details)
Epochs20§ Method (Implementation Details) alg. 1
Input size256×256 when image sizes vary§ Method (Implementation Details)Size for the main training run not stated
Generator GVGG-style architecture§ Method (Implementation Details)Layer count, widths, kernels not stated
Discriminator DDCGAN discriminator§ Method (Implementation Details)
Classifier CResNet-50, ImageNet pretrained§ Method (Deepfake Classifier) § Method (Implementation Details)
Decision thresholdnot stated
Number of GAN training images per categorynot stated
Hardware, training timenot stated

Results

Author-reported. The training set is 20 object categories of ProGAN, following Wang et al. § Experiments (Dataset). The paper does not name the category used in the 1-, 2- and 4-class settings of Table 3 tab. 3. Test sets: ProGAN faces at 1024×1024 and horses at 256×256 § Experiments (Dataset); StyleGAN, StyleGAN2, BigGAN, CycleGAN, StarGAN, GauGAN and a Deepfake (FaceForensics++) set § Experiments (Dataset). Metrics: accuracy (Acc.) and average precision (A.P.) § Experiments (Deepfake Detection Performance). Baselines: Wang et al. (CNN, image-based), Frank et al., Durall et al., and Jeong et al. (frequency-based) § Experiments (Deepfake Detection Performance). The Jeong baseline is cited as wacv; its bibliography entry is titled "BiHPF". Whether it is the authors' own prior work is not stated.

Table 1: manipulated face images, 1024×1024 tab. 1. Acc. / A.P. per manipulation.

ModelOriginalHueBrightnessSaturationGammaContrastBlurRotation
Wang99.9 / 10073.9 / 81.361.8 / 74.774.3 / 84.470.2 / 83.266.6 / 79.745.3 / 49.171.3 / 80.4
Frank99.6 / 99.485.5 / 97.284.2 / 97.291.2 / 98.085.4 / 97.484.3 / 96.753.8 / 90.399.6 / 99.4
Durall99.7 / 99.398.6 / 97.993.6 / 88.898.6 / 97.997.2 / 95.394.8 / 91.050.0 / 53.298.4 / 97.6
Jeong99.8 / 10085.0 / 92.689.9 / 90.896.9 / 99.699.7 / 10090.8 / 91.667.1 / 99.299.0 / 100
Ours100 / 10095.0 / 99.799.5 / 100100 / 10085.5 / 98.698.8 / 10098.5 / 98.5100 / 100

Table 2: resizing, trained and tested at a range of resolutions tab. 2. Acc. / A.P.

Model102451225612864
Wang99.9 / 10097.6 / 97.366.1 / 74.462.6 / 69.450.4 / 54.9
Frank99.6 / 99.492.2 / 90.290.5 / 86.091.3 / 86.989.7 / 85.1
Durall99.7 / 99.385.1 / 79.080.0 / 73.777.2 / 70.977.9 / 71.7
Jeong99.8 / 10097.9 / 99.997.8 / 99.889.4 / 96.959.7 / 62.2
Ours100 / 100100 / 100100 / 10098.0 / 99.995.9 / 99.4

Table 3: cross-model performance, by number of training categories tab. 3. Acc. / A.P. per test set; Mean is the average over the eight test sets.

TrainingModelInputProGANStyleGANStyleGAN2BigGANCycleGANStarGANGauGANDeepfakeMean
1 classWangImage50.4 / 63.850.4 / 79.368.2 / 94.750.2 / 61.350.0 / 52.950.0 / 48.250.3 / 67.650.1 / 51.552.5 / 64.9
1 classFrankFreq78.9 / 77.969.4 / 64.867.4 / 64.062.3 / 58.667.4 / 65.460.5 / 59.567.5 / 69.152.4 / 47.365.7 / 63.3
1 classDurallFreq85.1 / 79.559.2 / 55.270.4 / 63.857.0 / 53.966.7 / 61.499.8 / 99.658.7 / 54.853.0 / 51.968.7 / 65.0
1 classJeongFreq82.5 / 81.468.0 / 62.868.8 / 63.667.0 / 62.575.5 / 74.290.1 / 90.173.6 / 92.151.6 / 49.972.1 / 72.1
1 classOursImage95.5 / 99.480.6 / 90.677.4 / 93.063.5 / 60.559.4 / 59.999.6 / 10053.0 / 49.170.4 / 81.574.9 / 79.3
2 classesWangImage64.6 / 92.752.8 / 82.875.7 / 96.651.6 / 70.558.6 / 81.551.2 / 74.353.6 / 86.650.6 / 51.557.3 / 79.6
2 classesFrankFreq85.7 / 81.373.1 / 68.575.0 / 70.976.9 / 70.886.5 / 80.885.0 / 77.067.3 / 65.350.1 / 55.375.0 / 71.2
2 classesDurallFreq79.0 / 73.963.6 / 58.867.3 / 62.169.5 / 62.965.4 / 60.899.4 / 99.467.0 / 63.050.5 / 50.270.2 / 66.4
2 classesJeongFreq87.4 / 87.471.6 / 74.177.0 / 81.182.6 / 80.686.0 / 86.693.8 / 80.875.3 / 88.253.7 / 54.078.4 / 79.1
2 classesOursImage99.0 / 99.980.8 / 92.072.2 / 94.066.0 / 61.869.1 / 70.398.5 / 10053.1 / 51.062.2 / 80.675.1 / 81.2
4 classesWangImage91.4 / 99.463.8 / 91.476.4 / 97.552.9 / 73.372.7 / 88.663.8 / 90.863.9 / 92.251.7 / 62.367.1 / 86.9
4 classesFrankFreq90.3 / 85.274.5 / 72.073.1 / 71.488.7 / 86.075.5 / 71.299.5 / 99.569.2 / 77.460.7 / 49.178.9 / 76.5
4 classesDurallFreq81.1 / 74.454.4 / 52.666.8 / 62.060.1 / 56.369.0 / 64.098.1 / 98.161.9 / 57.450.2 / 50.067.7 / 64.4
4 classesJeongFreq90.7 / 86.276.9 / 75.176.2 / 74.784.9 / 81.781.9 / 78.994.4 / 94.469.5 / 78.154.4 / 54.678.6 / 77.9
4 classesOursImage99.0 / 99.980.7 / 89.684.1 / 98.669.2 / 71.171.1 / 74.499.9 / 10060.3 / 71.770.9 / 91.979.4 / 87.2

Table 4: ablation, trained on ProGAN horses tab. 4. Acc. / A.P.; "Self" is the training-set test, "Category" and "Model" are the unseen-category and unseen-GAN tests, "Manipulation" is the manipulated set.

GeneratorL_comL_advSelfCategory (mean)Model (mean)Manipulation (mean)Mean
Freqnoyes99.0 / 10090.9 / 98.269.4 / 76.190.9 / 98.487.6 / 93.2
Freqyesno100 / 10091.0 / 98.567.3 / 74.491.3 / 99.187.4 / 93.0
Img (pixel-level)yesyes99.8 / 10092.9 / 99.268.7 / 77.492.4 / 99.688.5 / 94.1
Freqyesyes100 / 10095.5 / 99.474.9 / 79.396.8 / 99.591.8 / 94.6

Figure results without a table. The unknown-category comparison is in fig. 4 and shows values only in the figure; the text says the model is best in all categories. The perturbation-map effects are in fig. 5 and fig. 6: they show the averaged and per-image power spectra, with and without the added map.

Headline numbers. Ours, 4 classes: mean Acc. 79.4, mean A.P. 87.2 over eight test sets tab. 3. Ours, 1 class: mean Acc. 74.9, A.P. 79.3 tab. 3. Ablation best: mean Acc. 91.8, A.P. 94.6 with the frequency generator and both losses tab. 4.

Paper vs code

No official code.

What I searched: the arXiv abstract page (no code link, no comment, no project page); the paper's LaTeX source (no URL); GitHub repository search for "FrePGAN" (0 results), "frepgan deepfake" (0), "frequency perturbation deepfake GAN" (0), the GitHub web search for "FrePGAN" (0), and a user search for the first author's email handle (0). The only repositories that matched a general query are deepfake awesome-lists and unrelated projects, which are not implementations of this method.

Pitfalls

  • Which input D sees. L_adv eq. 4 applies D to the perturbation G(x). L_D eq. 6 applies D to the perturbed image x + G(x). The text describes D as separating G(x) from real images, but the loss does not match that description. Not stated which the implementation uses.
  • Label convention. The text says D(x_r) = 0 and D(G(x)) = 1 § Method (Perturbation Discriminator). L_D eq. 6 maximises log D(x_r), which treats real images as 1. Follow the loss, but note the mismatch.
  • Objective form. Eq. 2 writes one min-max over L_G + L_D. Training updates G and D separately, with the order given in alg. 1. The eq. 2 form is a summary.
  • Discriminator learning rate. D is printed at 10^{-1}, which is far above G and C (10^{-4}) for Adam. This may be a typo. Not stated.
  • Scale of L_com. The norm is written without a normalisation. Sum or mean over pixels changes its size, and λ = 0.5 is fixed, so the balance with L_adv depends on that choice. Not stated.
  • Real images. The expectation in L_adv and L_com is over X, a batch of images. The text says real images should get near-zero maps, but no term forces that directly. Whether real images are in X for L_adv and L_com is not stated.
  • Gradient path into G from C. Eq. 8 depends on G through A_G(x). Whether L_C back-propagates into G is not stated. The algorithm updates C alone.
  • Update order. Algorithm 1 computes all losses from one forward pass, then updates G, D and C. The text says G is updated first, then D. Both orderings give different gradients. Not stated which the implementation uses.
  • Output of H and inverse FFT. H maps 2c channels to 2c, and F^{-1} must return c channels. The step that combines real and imaginary parts, and any normalisation of the FFT, are not stated.
  • Pixel range. x + G(x) is not clipped or rescaled in the text. Not stated.
  • Decision threshold and inference cost. The threshold for real or fake is not stated. Inference runs G and C on each image; the cost is not stated.
  • GauGAN claim. The text says the model achieves the highest Acc. and A.P. on GauGAN § Experiments (Deepfake Detection of Unknown GAN Models). In Table 3, the Ours row is below Jeong at 1 class (53.0 / 49.1 vs 73.6 / 92.1), 2 classes (53.1 / 51.0 vs 75.3 / 88.2) and 4 classes (60.3 / 71.7 vs 69.5 / 78.1). Treat the text claim as contradicted by the table.
  • Gamma manipulation. The text says the model is the most robust across manipulations. In Table 1, Ours scores 85.5 / 98.6 on Gamma, below Jeong (99.7 / 100) and Durall (97.2 / 95.3).
  • Resizing claim. The text says the model keeps "at least 97.8%" below 256 pixels § Experiments (Deepfake Detection of Resized Face Images). Table 2 gives 98.0 at 128 and 95.9 at 64 (accuracy). 97.8 is not the minimum; it is Jeong's 256 value. The table is the more precise source.
  • Category count. The text says the training set holds "20 objects" of ProGAN § Experiments (Dataset). The category used in the 1, 2 and 4-class settings is not named, and the number of training images is not stated.
  • Few-class claim. The text says 1-class performance is "similar" to 4-class. Table 3 gives 74.9 / 79.3 (1 class) against 79.4 / 87.2 (4 classes): a gap of 4.5 Acc. and 7.9 A.P. points.
  • Two-class mean. Jeong has a higher mean Acc. than Ours at 2 classes (78.4 vs 75.1). Ours is higher on mean A.P. (81.2 vs 79.1). Check both columns.
  • Figure numbering. Figure and table numbers in this card follow LaTeX source order, computed by hand. The compiled PDF was not read; check against it.
  • Compute. Batch size 16, 20 epochs, two adversarial networks and a ResNet-50 are the only stated costs. Hardware, wall-clock time and memory are not stated.

Lineage

  • Builds on: the DCGAN discriminator § Method (Perturbation Discriminator); the VGG and ResNet architectures § Method (Implementation Details); ImageNet pretraining.
  • Compared against: Wang et al. (adobe, "CNN-generated images are surprisingly easy to spot...for now"), Frank et al. (frank, "Leveraging Frequency Analysis for Deep Fake Image Recognition"), Durall et al. (watch_cvpr20, "Watch your Up-Convolution"), and Jeong et al. (wacv, "BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detection") § Experiments (Deepfake Detection Performance). Titles are from the paper's bibliography; no arXiv ids are linked here.
  • Superseded by: not known from the content of this wiki.

Source and license

Adapted from “FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations” by Yonghyun Jeong, Doyeon Kim, Youngmin Ro, Jongwon Choi (arXiv:2202.03347), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.