Deepfake Detection that Generalizes Across Benchmarks

In plain words

Detectors that spot face-swap fakes often work well on the kind of fake they were trained on and poorly on a new way of making fakes. GenD starts from a large general-purpose image model that already sees images well. Three such models are tried: CLIP, Meta's Perception Encoder and DINOv3. Only a small part of the model is retrained: the layer-normalisation settings, plus a simple linear classifier on top.

Before classification, each image's feature vector is scaled to unit length, so every feature sits on the surface of a sphere. Two extra loss terms shape that sphere. One pulls features of the same class together. The other spreads all features evenly over the surface, so they do not bunch up.

The authors report that this beats more complex recent detectors on 14 public benchmarks from 2019 to 2025, after training on one older dataset. They also find that training fakes should be paired with the real video each was made from, and that training only on the newest fakes does not guarantee good generalisation.

TL;DR

GenD adapts a frozen ViT-L image encoder by training only its LayerNorm parameters and a 1024-to-2 linear classifier. The feature is the L2-normalised classification token, trained with cross-entropy plus an alignment loss and a uniformity loss. Frames are averaged per video. Trained on FaceForensics++ c23 and tested on 14 cross-dataset benchmarks, the authors report a mean video AUROC of 91.2 to 91.6 across the three backbones, against 88.4 and 88.5 for ForAda and Effort. All results are author-reported; the competitors were rerun by the authors with their official weights.

Problem

A deepfake detector trained on one manipulation method often fails on a new method §1. Large pretrained encoders are a plausible starting point. Many recent detectors add architectural complexity or temporal and audio modelling §1 §4.2, and fine-tuning the whole backbone is reported to overfit and generalise poorly §4.3. The paper assumes labelled real and fake face videos for training, a foundation image encoder, and benchmarks with video-level labels §3.2 §4.1.

Method

The model is one image encoder, a feature step and a linear head fig. 1. The encoder is one of three ViT-L models: CLIP ViT-L/14 (OpenAI), PE core L (Meta) or DINOv3 ViT-L/16 (Meta). The paper keeps the 1024-dimensional classification token and discards the rest §3.1.

The trainable parameters are the weights and bias of the linear layer (1024 by 2) and all LayerNorm blocks of the encoder. The rest of the encoder is frozen. The paper says these trainable parameters are about 0.03 percent of the total §3.1. The number is stated, not checked here.

The feature is L2-normalised, and the linear classifier reads the normalised feature. The uniformity and alignment losses also act on the normalised feature §3.1 fig. 1.

The loss is cross-entropy on the real and fake logits, plus alpha times an alignment term and beta times a uniformity term eq. 1. The alignment term is the mean squared distance between normalised features of same-label pairs eq. 2. The uniformity term is the log of the mean of exp(-2 times squared distance) over pairs of features drawn from the data eq. 3. For CLIP the paper sets alpha to 0.1 and beta to 0.5, tuned on the validation set §3.1.

Training uses Adam with betas 0.9 and 0.999, no weight decay, bfloat16 weights, batch size 128 and a cyclic cosine learning-rate schedule with one-epoch warm-up per 10-epoch cycle §3.1. The precise schedule and the code's values differ; see Paper vs code.

Video-level prediction takes 32 frames per video, computes the softmax per frame and averages the probabilities §3.1.

# GenD training step  [§3.1] [eq. 1]
# Input:  images (B, 3, 224, 224); labels (B,) with 0 real, 1 fake
# Output: scalar loss

def training_step(images, labels):
    # Frozen ViT-L encoder; only its LayerNorm parameters train  [§3.1]
    # The 1024-d classification token is the feature  [§3.1]
    tokens = encoder(images)                     # (B, 1024)

    # Scale each feature to unit length: it lies on a hypersphere  [§3.1]
    embeddings = tokens / norm(tokens, dim=1)    # (B, 1024)

    # Linear classifier on the normalised feature  [§3.1]
    logits = linear(embeddings)                  # (B, 2)

    # Cross-entropy on real and fake logits  [§3.1] [eq. 1]
    loss = cross_entropy(logits, labels)

    # Alignment term, weight alpha = 0.1 for CLIP  [eq. 2]
    loss = loss + 0.1 * alignment(embeddings, labels)

    # Uniformity term, weight beta = 0.5 for CLIP  [eq. 3]
    loss = loss + 0.5 * uniformity(embeddings)
    return loss

# Alignment over same-label pairs in the batch  [eq. 2]
# The paper leaves the pair set P+ undefined. The code uses all
# equal-label pairs inside the mini-batch
# [code: src/losses/unifalign.py#L4-L43]

def alignment(embeddings, labels):
    # Positive pairs: two different samples with the same label
    pairs = same_label_pairs(labels)
    # Mean squared Euclidean distance over those pairs
    return mean(squared_distance(embeddings, pairs))

# Uniformity over sample pairs in the batch  [eq. 3]
# The code uses unordered pairs i < j and clamps the mean at 1e-6
# [code: src/losses/unifalign.py#L46-L63]

def uniformity(embeddings, t=2.0):
    # Squared distance for each unordered pair in the batch
    sq_dists = pairwise_squared_distances(embeddings)
    # Mean of exp(-t * d^2), then log
    kernel = mean(exp(-t * sq_dists))
    return log(max(kernel, 1e-6))

# Video-level fake score  [§3.1] [code: src/model/base.py#L61-L83]
# Input: 32 aligned face crops of one video  [§3.2]

def video_fake_score(frames):
    # One logit pair per frame  [§3.1]
    logits = model(frames)                       # (32, 2)
    # Softmax per frame, then the mean over frames  [§3.1]
    probs = softmax(logits, dim=1)               # (32, 2)
    video_probs = mean(probs, dim=0)             # (2,)
    return video_probs[1]

Hyperparameters

Results

Author-reported. Video-level AUROC (%). Setup: trained on FaceForensics++ c23 (cross-dataset protocol), 14 test benchmarks from 2019 to 2025, five training seeds for GenD rows. Competitor numbers in tab. 2 come from their original papers. Those in tab. 4 were rerun by the authors with the official weights and preprocessing §4.2.

Table 2: cross-dataset comparison, from tab. 2

The paper says GenD is best on CDFv2 (CLIP) and FFIW (DINO) in this table §4.2.

Table 4: reproduced cross-dataset AUROC, 14 test sets, from tab. 4

Standard deviations over five seeds for GenD are in [supp. tab. S3]. The paper says the higher average across all 14 sets shows better generalisation §4.2.

Table 3: in-domain AUROC on FF++ test, from tab. 3

Table 5: reported versus reproduced AUROC, from tab. 5 (reported = the original paper, reproduced = the authors' rerun)

Table 6: ablation, 14 test sets, from tab. 6 (averages over five seeds; the last row of each block is the full GenD method)

Setups 1 to 3 use CLIP, 4 to 6 PE, 7 to 9 DINOv3. LP is a frozen backbone with a linear probe; LN adds LayerNorm tuning; UA adds L2 normalisation and the uniformity and alignment losses.

Table S1: paired versus unpaired training data, from [supp. tab. S1] (20 random splits each, mean +- std; the backbone is not stated in the caption)

Table S2: PEFT comparison, CLIP ViT-L/14, mean AUROC only, from [supp. tab. S2]

Other results (author-reported, described from the figures, not tabulated):

• Paired versus unpaired §4.4 fig. 3: with PE, validation AUROC is higher and overfitting slower for paired training sets, averaged over 20 splits each. The supplement reports the same behaviour for CLIP on FF++ [supp. fig. S1], and faster overfitting, with the same pattern, for the FAVC [supp. fig. S2] and CDFv2 [supp. fig. S3] training sets.
• Detection difficulty over years §4.5 fig. 4: a model trained on FF++ (2019) generalises across more test sets than one trained on DSv2 (2025), which the authors say does poorly on older benchmarks. FFIW (2021) is a third training set.
• Robustness §4.6 fig. 5: under Gaussian blur, JPEG and downscaling, GenD (PE) is compared with ForAda and Effort on all 14 test sets. The text gives no numbers, and the authors report GenD's error bars over five seeds.
• Reported speed: 120 frames per second on an A100 at batch size 1, about 2 GB of VRAM §3.1. Not checked against the code.

Paper vs code

Official code: https://github.com/yermandy/GenD at 387a42266dd385fbe8f7626c5d3aa03eea3bfaab, MIT license (LICENSE). The abstract and the README link this repository, and the owner handle yermandy resembles the first author's name; the LICENSE copyright line reads "Andy". The code is a training and evaluation framework (PyTorch Lightning, src/), with configs in src/exp/. The training data is on Hugging Face per the README README.md. The data lists are not in the clone: only eight .txt lists under config/, all for FF++ test and CDFv2 test.

Searched for the code: the abstract link and the README agree. No other repositories were considered.

Differences:

• Learning-rate schedule. Paper: a one-epoch warm-up from 1e-5 to 3e-4 at the start of each cycle, then cosine decay to 1e-5 over nine epochs §3.1. Code: cosine warm restarts with a 10-epoch cycle, minimum 1e-6, and no warm-up in the WACV configs (warmup_epochs = 0). The warm-up branch exists but only runs once, at the start, when enabled config.py wacv_rebuttal.py GenD.py.
• Batch size. Paper: 128 §3.1. Code: 512 by default, and the WACV configs do not change it; gradient accumulation is 512 divided by 512, so one step per batch config.py run.py.
• Precision. Paper: "weight precision bfloat16" §3.1. Code: bf16-mixed config.py run.py. Lightning's mixed mode keeps fp32 weights with bf16 autocast (library behaviour, not checked here).
• Stopping. Paper: "for most runs" training did not improve after two cycles (20 epochs) §3.1, which describes what was observed rather than a rule. Code: 30 epochs maximum and 30 epochs of patience [code: src/exp/wacv_rebuttal.py#L212-L213]. With patience equal to the maximum run length, early stopping cannot fire; every run goes to 30 epochs.
• Model selection. Paper: the text uses "validation AUROC" §3.2, and does not say which checkpoint is tested. Code: monitors val/mAP_video (macro average precision over both classes, not AUROC) and tests the best-mAP checkpoint config.py [code: src/model/base.py#L204-L208] run.py.
• Frame sampling. Paper: "sample 32 frames evenly" §3.2. Code: the preprocessing default is at_least, which orders all frames by a maximum-spread permutation starting at the middle of the video, then keeps the first 32 with a detectable face detector.py [code: detector.py#L105-L106]. The README command uses -m at_least -n 32, with --det_thres 0.1 README.md. Training lists are per-seed random 32-frame subsets wacv_rebuttal.py.
• Face size. Paper: crop and resize to 224 x 224 after a 1.3 margin §3.2. Code: the landmark-based alignment sets the crop size from landmark geometry and scale 1.3 detector.py; the README command passes --target_size none README.md. The 224 resize is applied by the model's processor clip_encoder.py. Whether the released Hugging Face data was made this way is not stated.
• Alignment and uniformity. Paper: the alignment pair set P+ is not defined eq. 2; the uniformity term is an expectation over independent draws eq. 3. Code: alignment uses all equal-label pairs inside the mini-batch; uniformity averages over unordered pairs i < j in the batch, with the mean clamped at 1e-6 before the log unifalign.py [code: src/losses/unifalign.py#L63].
• Loss weights for PE and DINOv3. Paper states alpha and beta for CLIP only §3.1. Code uses 0.1 and 0.5 for all three encoders [code: src/exp/wacv_rebuttal.py#L46] wacv_rebuttal.py [code: src/exp/wacv_rebuttal.py#L184].
• Augmentation values. Paper lists the augmentation types only §3.2. The code's defaults give exact ranges and probabilities config.py augmentations.py. The WACV configs do not override them, so the defaults apply.
• Robustness settings. Paper: resize to 224, 114 and 64 pixels §4.6. Code: 224, 112 and 64 wacv_rebuttal_aug_robustness.py wacv.sh. The interpolation index order in wacv.sh is mislabelled; the comment in src/config.py matches Pillow's constants config.py wacv.sh.
• LoRA and PEFT ablation. Paper: LoRA on every transformer weight matrix, rank one §4.3. Code: the default LoRA targets out_proj only, rank 1, alpha 32 config.py GenD.py. The released experiment files contain no LoRA, BitFit or full fine-tuning configuration, so the PEFT table cannot be reproduced from the release as shipped.
• Paper arithmetic. The paper gives 56.8 percent for full fine-tuning in the text [supp §S2 text] and 57.9 in its table [supp. tab. S2]; the table's mean (57.9) matches its 13 values.
• Paper-internal naming. The paper calls the DeepSpeak sets "DSv1" in places; the code's DeepSpeak_v1_1 is the set reported as DSv1.1 §4.1 wacv_rebuttal.py.
• Sweep labels. In the alpha/beta sweep, the code assigns the swept alpha to uniformity and beta to alignment_labels, while the run names use the opposite letters. Named runs are consistent [code: src/exp/wacv_rebuttal.py#L288-L291] wacv_rebuttal.py.
• Test sets. The code's test dictionary has more sets than the 14 in the paper (for example FaceFusion, KoDF-adversarial and several FakeAVCeleb subsets) wacv_rebuttal.py. The 14 cross-dataset sets in the paper are a subset of these.
• Encoder pooling. The CLIP feature is HF pooler_output, which for CLIP applies the post-LayerNorm to the CLS token (library behaviour, not checked here) clip_encoder.py. DINOv3 uses last_hidden_state[:, 0] dino_encoder.py. For PE, the code removes the timm head and uses the backbone output; the pooling inside timm is not checked here perception_encoder.py.

Pitfalls

• The unfreezing rule is by substring. unfreeze_layers matches any parameter whose name contains one of the strings GenD.py. For PE and DINOv3, the string norm may match more than LayerNorm weights and biases. Check the trainable parameter list, and confirm the 0.03 percent figure, before a long run.
• Uniformity and alignment need trainable parameters upstream. With a frozen encoder and a linear head, the two extra losses are constant with respect to the trainable parameters, so they do nothing. The paper reports that adding them without LayerNorm tuning gives identical numbers [supp. tab. S8] [supp. §S4]. Keep LayerNorm tuning on when using these losses.
• The losses depend on the batch. The alignment and uniformity estimates use whatever pairs are in the mini-batch unifalign.py. The paper uses 128, the code 512. A different batch size changes the loss scale and the set of positive pairs.
• Silent NaN handling. If the total loss is NaN, the code logs a warning and replaces the loss with zero times the logits loss.py. A run can keep training with no gradient on that step.
• Uniformity is clamped. The mean kernel is clamped at 1e-6 before the log unifalign.py. A collapsed batch hits the clamp and the gradient stops there.
• Warm restarts jump back to the peak. Without a warm-up, each 10-epoch cycle restarts the learning rate at 3e-4 from 1e-6 GenD.py. Paper and code differ here, so use one schedule explicitly.
• Checkpoint choice changes the reported number. The code tests the best-mAP checkpoint on the custom validation set run.py. The paper does not say how it picked the test checkpoint. Results can move if you select by AUROC.
• Frame sampling matters. The released preprocessing spreads 32 frames by maximum spacing and skips frames without a face. The paper describes even sampling. Video-level scores average over whatever frames the test list contains base.py.
• Training pairs must be built. The paired and unpaired split lists are not in the clone, and no script that generates them was found; the experiment file only names them wacv_rebuttal_paired_unpaired.py.
• Compute for the reproduction. Training is 30 epochs at batch 512 by default, on lists that are not in the clone. The data must be downloaded from Hugging Face (see the README) before the configs will run.

Lineage

• Builds on: UniFD (a frozen-backbone linear probe, used as the baseline setting §4.3); LN-tuning [arXiv:2211.08682]; uniformity and alignment losses [arXiv:2005.10242] (the code links this arXiv id).
• Compared against: ForAda §4.2 (CLIP ViT-L/14 adapter, CVPR 2025), Effort §4.2 (CLIP ViT-L/14, ICML 2025), and others in tab. 2. The authors reran ForAda and Effort with their official weights.
• Superseded by: not known from content on this wiki.