---
kind: concept
slug: noise-residual-features
title: Noise residual features
summary: >-
  High-pass residuals (fixed SRM filters) that remove colour and keep fine noise, fed as a second stream to spot manipulated faces. Only one paper card so far: a two-stream detector whose ablation shows the noise stream alone helps on unseen forgery methods but not on the training one, and that fusion and multi-scale features add to it. All numbers are author-reported.
tags: [high-pass-filters, forgery-detection, two-stream, residuals]
papers: ["2103.12376"]
related: [image-manipulation-detection, channel-attention, attention-mechanism]
status: draft
updated_at: 2026-10-18
---

## Overview

A noise residual is what remains of an image after a high-pass filter removes
its smooth, colour-dominated content. The authors' claim is that a manipulated
region can leave statistical traces in this residual that the RGB image does
not show, so a detector that reads only colour can latch onto the wrong cues
[arxiv:2103.12376 §3.1] [arxiv:2103.12376 §3.3]. The filters are SRM kernels,
borrowed from steganalysis, applied to the input image and again to
intermediate feature maps [arxiv:2103.12376 §4.1]. The kernel set is not
stated [arxiv:2103.12376 §4.1].

The shared idea across the design options is the same: keep a colour stream
for low-frequency texture and add a residual stream for high-frequency traces,
then decide how the two interact [arxiv:2103.12376 §4]. The residual can also
steer the colour stream with attention [arxiv:2103.12376 §4.2]. For the
neighbouring task of locating tampered regions, see
[Image manipulation detection](/concepts/image-manipulation-detection).

## Variants

The one paper card here ablates its design one module at a time, so the
variants below are its ablation rows, trained on FF++ F2F and tested on other
FF++ methods [arxiv:2103.12376 tab. 3]. Their paper does not name them as
variants.

### SRM noise stream alone

Only the high-pass branch is used. On the F2F-trained ablation it scores AUC
0.758 on DF, 0.913 on FS and 0.858 on NT, against 0.803, 0.762 and 0.696 for
the RGB Xception baseline [arxiv:2103.12376 tab. 3]. Card: [Generalizing Face
Forgery Detection with High-frequency Features](/papers/2103.12376).

### Two-stream fusion

An RGB stream and a noise stream are combined before classification. The
ablation gives AUC 0.810 on DF, 0.922 on FS and 0.894 on NT [arxiv:2103.12376
tab. 3]. Card: [Generalizing Face Forgery Detection with High-frequency
Features](/papers/2103.12376).

### Fusion with residual-guided spatial attention

A spatial attention map computed from the residual gates the RGB features
before fusion [arxiv:2103.12376 §4.2] [arxiv:2103.12376 eq. 2]. The ablation
gives AUC 0.819 on DF, 0.966 on FS and 0.927 on NT [arxiv:2103.12376 tab. 3].
Card: [Generalizing Face Forgery Detection with High-frequency
Features](/papers/2103.12376).

### Fusion with dual cross-modality attention

The two streams exchange information through attention over all positions. The
correlation matrix between them is HW × HW [arxiv:2103.12376 §4.3]. The
ablation gives AUC 0.801 on DF, 0.975 on FS and 0.957 on NT [arxiv:2103.12376
tab. 3]. Card: [Generalizing Face Forgery Detection with High-frequency
Features](/papers/2103.12376).

### Full model: multi-scale noise features

The noise features are taken at several scales inside the network, not only
from the input residual [arxiv:2103.12376 §4.1] [arxiv:2103.12376 eq. 1]. With
RSA and DCMA added, the ablation gives AUC 0.837 on DF, 0.987 on FS and 0.984
on NT [arxiv:2103.12376 tab. 3]. Card: [Generalizing Face Forgery Detection
with High-frequency Features](/papers/2103.12376).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Trained on one forgery method, must hold up on others | Full model (multi-scale noise, RSA, DCMA) | Trained on F2F, its FS and NT AUC are 0.987 and 0.984, against 0.762 and 0.696 for RGB Xception [arxiv:2103.12376 tab. 3] |
| Test forgeries close to the DF method | Include the RGB stream; do not use noise alone | Noise only reaches 0.758 on DF against 0.803 for RGB [arxiv:2103.12376 tab. 3] |
| Only one stream affordable | Not settled by this card | Noise alone is weaker on DF but stronger on FS and NT than RGB [arxiv:2103.12376 tab. 3] |
| Tight compute or memory budget | Skip DCMA, or measure first | The card reports no runtime or memory figures, and DCMA's correlation matrix grows with the square of the positions [arxiv:2103.12376 §4.3] |
| Compressed (LQ) video | Treat the cross-paper comparison as indicative | The LQ comparison is video-level and copies competitor numbers from their papers [arxiv:2103.12376 tab. 6] |

## Implementation sketch

Common core of the two-stream design, in our own words. Steps 1 and 2 are the
noise variant; step 3 is the RSA variant; step 4 is the fusion. The paper's
DCMA would replace the plain concatenation in step 4 with cross-stream
attention, applied twice [arxiv:2103.12376 §4.3].

```pseudocode
# Forward pass of a two-stream noise-residual detector  [§4]
# Input:  face crop rgb (B, 3, 256, 256)  [§5.1]
# Output: manipulation score  [§4.3]

def detect(rgb):
    # 1. High-pass residual of the input via fixed SRM kernels  [§4.1]
    residual = srm_filter(rgb)
    # 2. Colour and noise features; the paper takes these at several scales  [§4.1]
    rgb_feat = rgb_encoder(rgb)
    noise_feat = noise_encoder(residual)
    # 3. Residual-guided attention gates the colour features  [§4.2]
    rgb_feat = spatial_attention(residual) * rgb_feat
    # 4. Fuse both streams with channel attention, then score  [§4.3]
    fused = channel_attention(concat(rgb_feat, noise_feat))
    return classifier(fused)  # trained with AM-Softmax [§4.3]
```

Not stated and not to be filled from elsewhere: the SRM kernel set, the number
of scales, whether the filters are fixed or learned [§4.1], the attention
reduction ratio [§4.3], and the margin and scale of AM-Softmax [§4.3].

## Contradictions

Only one paper card lists this concept, so there is no cross-paper
disagreement to report. Within the paper:

- The ablation text says performance "gradually improves" with each module
  [arxiv:2103.12376 §5.2]. On DF, adding DCMA lowers AUC from 0.819 to 0.801
  [arxiv:2103.12376 tab. 3].
- The Xception baseline differs between two tables: the DF-trained FS cell is
  0.485 in [arxiv:2103.12376 tab. 1] and 0.490 in [arxiv:2103.12376 tab. 5],
  and the NT-trained DF cell is 0.800 and 0.799 respectively. The text does not
  explain the difference.

## Papers

- [Generalizing Face Forgery Detection with High-frequency Features](/papers/2103.12376):
  a two-stream detector with multi-scale SRM noise features, residual-guided
  attention and cross-stream attention; author-reported cross-database AUC
  gains over Xception and Face X-ray.