Paper · 2022-03-27 · cs.CV
PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition
Treats face anti-spoofing as fine-grained recognition of patch types (capture device times presenting material). Non-distorted 160-px patches train an asymmetric angular-margin softmax (live margin 0.4, spoof 0.1) plus a patch-similarity loss; tests average live probability over nine patches. Author-reported: 0.0% ACER on OULU-NPU protocol 1; 11.82% HTER on one domain-generalization protocol.
Angular margin losses (ArcFace and relatives)Domain generalizationSelf-supervised learningface-anti-spoofingpatch-based-recognitionangular-marginself-supervised-learningdomain-generalization
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
A face unlock system can be fooled by a fake face, such as a printed photo or a screen. Face anti-spoofing is the job of telling a real live face from those fakes. Many earlier methods learned from small datasets, so they often picked up the quirks of those datasets instead of anything general.
PatchNet takes a different route. It cuts small square pieces out of the face image, keeps their original pixels, and asks the network to name the kind of capture each piece came from: which camera and which material it was shown on. The pieces show how the image was captured and what it was shown on. Learning to tell these kinds of capture apart pushes the network to pick up fine local detail rather than overall appearance.
Two extra training rules shape the feature space. One pushes live pieces into a tighter cluster than fake pieces, because fakes vary much more. The other asks pieces cut from the same face to produce similar features. At test time, many pieces from one face are scored and their live scores are averaged. The authors report strong results on several standard benchmarks, which were measured in their own setup.
TL;DR
Face anti-spoofing is treated as fine-grained recognition of patch types, where a type is a capture device combined with a presenting material. Training uses non-distorted 160-px patches and an asymmetric angular-margin softmax, with a similarity term that ties together two views of the same face §3.1 §4.2. The authors report state-of-the-art results on most intra-dataset, cross-dataset and domain-generalization protocols, all author-reported §4.3-§4.5. No official code was found.
Problem
Binary live-versus-spoof training on small, low-diversity FAS datasets tends to overfit to collection biases, and the learned features fail on unseen environments, spoof types and devices §1. Typical presentation attacks include printed photos and screens fig. 4 §4.6. Earlier fixes used pixel-level auxiliary supervision (depth or reflection maps) or domain-generalization training, and the authors argue that the capture characteristics of a device and its material are ignored by those methods §1 [abstract]. Resizing the whole face to a fixed size also distorts the fine cues that separate capture types §1.
Method
Overview. Each training face crop is turned into patches; each patch is labelled by one fine-grained class built from the dataset metadata (capture device and presenting material). The patch features are L2-normalised and trained with an asymmetric angular-margin softmax plus a similarity loss §3.1 fig. 2. In CASIA-FASD, for example, two spoof media and three capture resolutions give nine patch types: three live and six spoof §3.1.
Step 1: patch views. Two patch views x^t1 and x^t2 are drawn from one face crop x. The augmentations are limited to ones that leave the pixels undistorted: a horizontal flip and a rotation, both applied at random, and a crop of fixed size. No resizing is used §3.2. The rotation range, flip probability and crop position distribution are not stated.
Step 2: encoder and normalisation. Each view goes through the encoder E and a normalisation layer, giving f = Normalize(E(x)) §3.2. The encoder is ResNet18 §4.2.
Step 3: fine-grained labels. For N classes, of which k are live and N-k are spoof, each patch has one ground-truth label y in {L_1..L_k, S_1..S_(N-k)} §3.3. Dataset statistics are in tab. 1; class counts per protocol are in §4.3-§4.5.
Step 4: AM-Softmax base. The softmax logit is written as a scaled cosine. For the target class the cosine is reduced by a margin m, and all logits are scaled by s eq. 1 eq. 2 eq. 3 eq. 4 eq. 5. Features and class weights must both be L2-normalised for this to hold §3.3.1.
Step 5: asymmetric margin. The margin depends on the target's live or spoof type. Live classes receive m_l and spoof classes receive m_s, with m_l larger than m_s, so live features are pushed into tighter clusters eq. 6 §3.3.2.
Step 6: two-view classification loss. The asymmetric loss is computed for both views and summed, then averaged over the batch eq. 7.
Step 7: similarity loss. The mean L2 distance between the two views' features is added as a regulariser eq. 8 §3.4.
Step 8: total loss. L = alpha_1 * L_Asym + alpha_2 * L_Sim, with alpha_1 = alpha_2 = 1.0 in all experiments eq. 9 §3.5.1.
Inference. A test face is cut into P uniformly placed patches of the training size. For each patch the live probability is the sum of the softmax over the live classes, using the scaled cosines. The face score is the mean over the P patches eq. 10 §3.5.2. P = 9 in all experiments, using 3 anchors per side §4.2.
# PatchNet training step [§3.2] [§3.3] [eq. 7] [eq. 8] [eq. 9]
# Input: face crops and per-view patch labels (device x material)
# Output: scalar loss
def training_step(face_crops, labels, is_live):
# Draw two non-distorted views per face crop [§3.2]
view_1 = augment(face_crops) # flip, rotate, crop 160
view_2 = augment(face_crops)
# Encode and L2-normalise each view [§3.2]
feat_1 = normalize(encoder(view_1)) # (B, d)
feat_2 = normalize(encoder(view_2)) # (B, d)
# Asymmetric margin classification loss, summed over views [eq. 7]
cls_loss = mean(
asym_am_softmax(feat_1, labels, is_live)
+ asym_am_softmax(feat_2, labels, is_live)
)
# Mean L2 distance between the two views of one face [eq. 8]
sim_loss = mean(norm(feat_1 - feat_2, dim=1))
# Weighted total, alpha_1 = alpha_2 = 1.0 [eq. 9] [§3.5.1]
return alpha_1 * cls_loss + alpha_2 * sim_loss# Asymmetric AM-Softmax on one view [eq. 4] [eq. 5] [eq. 6]
# Input: feat (B, d) normalised; class_weights (N, d) normalised
# Output: per-sample loss (B,)
def asym_am_softmax(feat, labels, is_live):
# Cosine between each feature and each class weight [eq. 4]
cosines = feat @ class_weights.T # (B, N)
# Margin m_l for live targets, m_s for spoof targets [eq. 6]
margins = where(is_live[labels], m_live, m_spoof) # (B,)
# Subtract the margin from the target cosine only [eq. 4] [eq. 6]
logits = cosines.copy()
logits[row, labels] = cosines[row, labels] - margins
# Scale by s, then cross-entropy over all N classes [eq. 5]
return cross_entropy(s * logits, labels) # s = 30.0, m_l = 0.4, m_s = 0.1# Test-time live score for one face image [§3.5.2] [§4.2] [eq. 10]
# Input: face_image (H, W) cropped face; patch size 160
# Output: scalar live probability
def live_probability(face_image, live_mask):
# Three anchors per side, evenly spaced across the valid range [§4.2]
# (spacing is implied, not stated)
patches = grid_crops(face_image, size=160, per_side=3) # P = 9
scores = []
for patch in patches:
feat = normalize(encoder(patch)) # (d,)
# Scaled cosines softmaxed over classes [eq. 10]
probs = softmax(s * (class_weights @ feat)) # (N,)
scores.append(sum(probs[live_mask]))
return mean(scores)Hyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| Patch crop size | 160 | §4.2 | Fixed square crop, training and test. Very small (64) degrades clearly; larger sizes differ little fig. 3 §4.4 |
| Scale s | 30.0 | §4.2 eq. 5 | Applied to all cosines |
| Live margin m_l | 0.4 | §4.2 tab. 5 | Chosen on OULU-NPU protocol 1 (see Pitfalls) |
| Spoof margin m_s | 0.1 | §4.2 tab. 5 | Same selection |
| alpha_1, alpha_2 | 1.0, 1.0 | §3.5.1 eq. 9 | Used in all experiments |
| Encoder | ResNet18 | §4.2 | Larger encoder reported as similar, in supplementary (not in source) |
| Optimiser | SGD | §4.2 | Momentum and weight decay not stated |
| Initial learning rate | 0.002 | §4.2 | |
| LR schedule | halve every 90 epochs | §4.2 | |
| Max epochs | 200 | §4.2 | |
| Frames per video clip | 3 random frames | §4.2 | Source frames for training |
| Face detector | RetinaFace | §4.2 | Used to crop training faces |
| Test patches P | 9 (3 per side) | §4.2 §3.5.2 | Coordinates: min = size/2, max = width - size/2 |
| Batch size | not stated | ||
| Rotation range, flip probability | not stated | §3.2 | Only the operation names are given |
| Input normalisation, colour scale | not stated | ||
| Feature dimension d | not stated | ||
| Training time, hardware | not stated |
Table 1: dataset statistics tab. 1
| Dataset | Subjects train / test | Clips train / test | Live classes | Spoof classes |
|---|---|---|---|---|
| OULU-NPU (O) | 20 / 20 | 1800 / 1800 | 1 | 4 |
| SiW (S) | 90 / 75 | 2442 / 2036 | 2 | 12 |
| CASIA-FASD (C) | 20 / 30 | 480 / 720 | 3 | 6 |
| Replay-Attack (I) | 30 / 20 | 360 / 240 | 1 | 3 |
| MSU-MFSD (M) | 15 / 20 | 120 / 160 | 2 | 6 |
Number of patch-type classes used per protocol: Oulu-NPU protocols 1 to 4 use 5, 3, 5 and 3 §4.3.1. SiW protocols 1, 2, 3-1 and 3-2 use 14, 8, 6 and 10 §4.3.2. C to I uses 9 and I to C uses 4 §4.5.1. The four domain-generalization protocols use 18, 17, 22 and 21 §4.5.2.
Results
Author-reported. Setup: training on the protocol's source datasets with a ResNet18 encoder, testing on the stated target §4.1-§4.5. The numbers below are copied from the tables; the bold and underline marks are the paper's own. Competitor rows are the best one listed per protocol, not all baselines.
Intra-dataset, OULU-NPU ACER (%) tab. 2
| Protocol | PatchNet ACER | Best listed baseline ACER |
|---|---|---|
| 1 | 0.0 | NAS-FAS 0.2 |
| 2 | 1.2 | NAS-FAS 1.2 (tie) |
| 3 | 1.18 ± 1.26 | NAS-FAS 1.7 ± 0.6 |
| 4 | 2.9 ± 3.0 | NAS-FAS 2.9 ± 2.8 (tie by mean) |
Intra-dataset, SiW ACER (%) tab. 3
| Protocol | PatchNet ACER | Best listed baseline ACER |
|---|---|---|
| 1 | 0.00 | SpoofTrace 0.00, DualStage 0.00 |
| 2 | 0.00 ± 0.00 | SpoofTrace 0.00 ± 0.00, DualStage 0.00 ± 0.00 |
| 3 | 2.45 ± 0.45 | NAS-FAS 1.52 ± 0.13 |
Cross-dataset HTER (%) tab. 6
| Protocol | PatchNet HTER | Best listed baseline HTER |
|---|---|---|
| Train C, test I | 9.9 | DC-CDN 6.0 |
| Train I, test C | 26.2 | DC-CDN 30.1 |
Domain generalization, HTER (%) / AUC (%) tab. 8
| Method | O&C&I to M | O&M&I to C | O&C&M to I | I&C&M to O |
|---|---|---|---|---|
| PatchNet | 7.10 / 98.46 | 11.33 / 94.58 | 13.4 / 95.67 | 11.82 / 95.07 |
| Lowest-HTER baseline listed | SSDG-R 7.38 / 97.17 | SSDG-R 10.44 / 95.94 | NAS-FAS 11.63 / 96.98 | NAS-FAS 13.16 / 94.18 |
The O&C&M to I cell for PatchNet differs between tables: tab. 7 lists 14.6 / 92.51 for the same protocol (see Pitfalls).
Ablation, OULU-NPU protocol 1, ACER (%) tab. 4
| Output class | Input | L_Asym | L_Sim | ACER |
|---|---|---|---|---|
| binary (cross-entropy) | resize 256 x 256 | no | no | 6.25 |
| binary (cross-entropy) | patch crop | no | no | 3.54 |
| fine-grained patch types | resize 256 x 256 | no | no | 5.63 |
| fine-grained patch types | patch crop | no | no | 1.88 |
| fine-grained patch types | patch crop | no | yes | 1.46 |
| fine-grained patch types | patch crop | yes | no | 0.63 |
| fine-grained patch types | patch crop | yes | yes | 0.0 |
Margin choice, OULU-NPU protocol 1, ACER (%) tab. 5
| m_l | m_s | ACER |
|---|---|---|
| 0.0 | 0.0 | 1.46 |
| 0.2 | 0.2 | 0.83 |
| 0.2 | 0.1 | 0.41 |
| 0.4 | 0.4 | 0.63 |
| 0.4 | 0.3 | 0.2 |
| 0.4 | 0.2 | 0.2 |
| 0.4 | 0.1 | 0.0 |
| 0.4 | 0.0 | 0.41 |
Domain-generalization ablation, HTER / AUC tab. 7: without margin, without L_Sim, and with the four-class SSDG split ("coarse cls") are all worse than the full model on each of the four protocols. Example, O&C&I to M: coarse-class 10.24 / 96.45, no margin 10.0 / 96.61, no L_Sim 8.9 / 97.42, full 7.10 / 98.46.
Cross-capture-device and few-shot tab. 9 (AUC %, averaged over 10 runs for the few-shot rows):
| Method | M1 | M2 | C1 | C2 | C3 |
|---|---|---|---|---|---|
| PatchNet | 99.54 | 98.63 | 94.29 | 88.49 | 98.13 |
| PatchNet + 5-shot live reference | 99.7 | 99.6 | 94.3 | 89.8 | 98.6 |
| PatchNet + 10-shot live reference | 99.8 | 99.6 | 95.3 | 90.7 | 99.2 |
The per-device rows are for O&C&I to M (M1, M2) and O&M&I to C (C1, C2, C3) §4.7. Overall AUC for those protocols is 98.46 and 94.58 tab. 9.
Patch retrieval. A patch-type retrieval experiment (one live query from O, 21-type training set from M, C and I) gives top-7 types M2_L, C3_L, M1_L, M1_S2, C2_L, M2_S2, I_L fig. 7 §4.7. Removing C2_L, C1_L and I_L at test time gives 95.27 AUC on O; relabelling them as spoof for training gives 95.87 §4.7. The paper does not give the baseline AUC for this comparison.
Paper vs code
No official code. Search steps:
- The LaTeX source contains no code or project-page links, apart from a template URL. The raw metadata has no comments field.
- GitHub repository search for
patchnet+anti-spoofing,patchnet+face+anti-spoofing,patchnet+org:microsoft,fine-grained+patch+anti-spoofingandPatchNet+spoof. The only hit,asd841018/Face-anti-spoofing-using-PatchNet, belongs to an account that is not an author and has no licence, so it was not used. - A general web search was not available in this run.
Because no code was used, every implementation detail above comes from the paper.
Pitfalls
- Similarity loss wording. Section 3.1 calls the similarity term "the positive part of the contrastive loss", but Eq. 8 is a plain mean L2 distance with no negatives, no squaring and no stop-gradient §3.1 eq. 8. Implement Eq. 8 as written unless the authors say otherwise.
- Margin tuning on a test protocol. m_l = 0.4 and m_s = 0.1 were chosen on OULU-NPU protocol 1 §4.4 tab. 5, which is also reported as a test result tab. 2. The "same in all protocols" claim is therefore not a held-out choice.
- Internal inconsistency on OULU-NPU. The text says PatchNet achieves the best performance in all four protocols §4.3.1, but tab. 2 bolds NAS-FAS on protocol 4 (2.9 ± 2.8 vs PatchNet 2.9 ± 3.0) and ties on protocol 2.
- Internal inconsistency on domain generalization. O&C&M to I for PatchNet is 14.6 / 92.51 in tab. 7 and 13.4 / 95.67 in tab. 8. For O&M&I to C, tab. 8 bolds ANRL (10.83) although SSDG-R (10.44) has lower HTER in the same table.
- Not stated in source. Batch size, optimiser momentum and weight decay, rotation range, flip probability, crop position distribution, normalisation of inputs, feature dimension, number of runs behind the ± values, and the classification threshold used for APCER/BPCER/ACER.
- Supplementary material is referenced but absent. The encoder-capacity claim and the dataset sample images are in the supplement §4.2 §4.1. Not in the source package, so they cannot be checked.
- Class definitions are dataset-specific. Patch-type classes come from each dataset's device and medium metadata §3.1. The paper says Oulu-NPU's six phone types are merged into five classes because they look alike §4.1. SiW per-protocol class groupings are not spelled out. In domain generalization the classes of the three training sets are simply concatenated, so a device seen in two datasets becomes two classes §4.5.2.
- Test-time crop placement. The text gives the outer bounds (size/2 from each edge) and three anchors per side, but not the spacing §4.2. Assume even spacing that includes both bounds, and say so in your code.
- Test input. The face image at test time is described as "a test face image" and the crops are taken from it §3.5.2. Whether test faces go through the same RetinaFace crop is implied but not stated.
- Margin-live and margin-spoof roles. m_l applies to live classes and m_s to spoof classes, and the larger margin is on live §3.3.2 eq. 6. Swapping them is a silent error that the notation does not catch.
- Cost. The classifier has one weight per patch type, so the head grows with the fine-grained class count (3 to 22 across the protocols above). Each test face needs P = 9 forward passes §4.2.
- Few-shot details missing. How the live reference features are combined (mean of embeddings, or per-sample scores) is not stated §4.7. The table reports averages over 10 runs tab. 9.
- Crop size ablation. Only the 64 result is described in words. Other sizes are only shown in a plot whose values are not given fig. 3 §4.4.
- Live probability at test. Eq. 10 sums softmax probabilities over live classes with no margin term eq. 10, which is consistent with the margin being used only during training. The paper does not say this explicitly.
Lineage
- Builds on: AM-Softmax, Wang et al., "Additive margin softmax for face verification" (cited in the paper as wang2018additive; no card). This paper adds separate live and spoof margins §3.3.2.
- Builds on: the SSDG class split of Jia et al. (CVPR 2020) (no card). The paper uses it as a baseline ("PatchNet w/ coarse cls") tab. 7 tab. 8.
- Motivated by: Yu et al., face anti-spoofing with human material perception (ECCV 2020, no card), which frames anti-spoofing as material recognition §1.
- Compared against: NAS-FAS, CDCN, BCN, Disentangle, SpoofTrace, DualStage, DC-CDN, SSDG-R, ANRL and others tab. 2 tab. 3 tab. 6 tab. 8. None has a card.
- Superseded by: not known from content in this wiki.
Source and license
Adapted from “PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition” by Chien-Yi Wang, Yu-Ding Lu, Shang-Ta Yang, Shang-Hong Lai (arXiv:2203.14325), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.