ArcFace: Additive Angular Margin Loss for Deep Face Recognition

In plain words

A face recognition system turns each face photo into a list of 512 numbers, an embedding, so that photos of the same person land close together and photos of different people land far apart. A common way to learn such embeddings is to train an ordinary classifier over thousands of known people and then throw the classifier away, keeping only the embedding. The catch is that a plain classifier only needs to get the training people right; it does not push them far apart, so unseen people get confused.

ArcFace puts every embedding and every person's "center" on the surface of a sphere and measures how similar they are by the angle between them. During training it pretends each photo is a little further from its own person's center than it really is, by adding a fixed angle. To still classify correctly, the network has to pull each person's photos tightly together and leave a clear gap to everyone else. It is a few lines of code on top of a normal softmax classifier.

The later version of the paper adds two things. Sub-center ArcFace gives each person three centers instead of one, so mislabeled or odd photos gather around a secondary center rather than dragging the main one around; those photos can then be found and removed, which lets the authors clean large scraped web collections automatically. And the trained network can be run in reverse: starting from random noise and nudging the pixels, it can produce a face that it recognizes as a given person, using statistics stored inside the network to keep the result looking like a face.

TL;DR

ArcFace replaces the logit of the true class in a normalized softmax with s\cos(\theta_{y_i}+m), i.e. an additive margin on the angle between the L2-normalized feature and class weight, with s=64, m=0.5 and 512-d embeddings. The authors argue this margin maps exactly to geodesic distance on the hypersphere and, unlike SphereFace, trains without an auxiliary softmax loss. This journal version (arXiv v4) adds sub-center ArcFace: K=3 weight vectors per class with max pooling over them, which tolerates label noise; after a first training round, non-dominant sub-centers and samples more than 75° from the dominant sub-center are dropped and the model is retrained from scratch. Combining cleaned MS1MV0 and Celeb500K gives IBUG-500K (11.96M images, 493K identities), on which ResNet100 ArcFace reports 96.02% / 97.27% TPR@FPR=1e-4 on IJB-B / IJB-C. It also inverts a trained model into face images using the ArcFace loss plus a BatchNorm-statistics prior. Read this if you train embedding models with a classification head (faces, re-identification, retrieval).

Problem

Softmax-trained features separate the closed set of training identities but are not discriminative enough for open-set recognition, and the classifier matrix W \in \mathbb{R}^{d\times N} grows linearly with the identity count N §1. Triplet loss avoids the classifier but suffers a combinatorial explosion of triplets and depends on hard semi-hard mining §1, §2. SphereFace introduced an angular margin, but its integer multiplicative margin makes the target logit curve steep, needs approximations and a softmax auxiliary loss to converge §1, §3.1. Margin-based softmax methods also assume clean labels, whereas web-scraped face datasets carry large noise (MS1MV0 is estimated at 47.1%-54.4% noisy) and manual cleaning is expensive §1, §3.2.

Method

ArcFace loss §3.1

Start from softmax cross-entropy eq. 1:

L_1=-\log\frac{e^{W^T_{y_i} x_i+b_{y_i}}}{\sum_{j=1}^{N}e^{W^T_j x_i+b_j}} eq. 1

Set biases to 0, write the logit as \|W_j\|\|x_i\|\cos\theta_j, L2-normalize each class weight W_j to norm 1, and L2-normalize the feature then rescale it to norm s §3.1. Predictions now depend only on angles eq. 2:

L_2=-\log\frac{e^{s\cos\theta_{y_i}}}{e^{s\cos\theta_{y_i}}+\sum_{j=1,j\neq y_i}^{N}e^{s\cos\theta_{j}}} eq. 2

ArcFace adds the margin m to the target angle only eq. 3:

L_3=-\log\frac{e^{s\cos(\theta_{y_i}+m)}}{e^{s\cos(\theta_{y_i}+m)}+\sum_{j=1,j\neq y_i}^{N}e^{s\cos\theta_{j}}} eq. 3

Algorithm: ArcFace head (forward)  [§1, §3.1, fig. 2]
Input: embedding x_i in R^512, class weights W in R^{512 x N}, label y_i, scale s, margin m
1. x_hat = x_i / ||x_i||;  W_hat_j = W_j / ||W_j|| for every class j
2. cos_j = W_hat_j^T x_hat for all j            (one matrix multiply)
3. theta_y = arccos(cos_{y_i})                   (target class only)
4. target logit = cos(theta_y + m); other logits unchanged = cos_j
5. multiply all logits by s
6. softmax cross-entropy with label y_i
Output: loss L3

Unified margin. SphereFace (multiplicative m_1), ArcFace (additive angular m_2) and CosFace (additive cosine m_3) are special cases of eq. 4:

L_4=-\log\frac{e^{s(\cos(m_1\theta_{y_i}+m_2)-m_3)}}{e^{s(\cos(m_1\theta_{y_i}+m_2)-m_3)}+\sum_{j=1,j\neq y_i}^{N}e^{s\cos\theta_{j}}} eq. 4

The authors reimplement SphereFace through arccos with a non-integer m_1=1.35, which they report converges without the softmax auxiliary loss §3.1. They argue that the additive angular margin gives a constant linear angular decision margin, while SphereFace and CosFace give a nonlinear one §3.1, fig. 5. During ArcFace training the target angle moves from about 90° at initialization to about 30° at the end §3.1, fig. 4.

Auxiliary losses compared (not part of ArcFace) §3.1:

L_5=L_2 + \frac{1}{\pi} \theta_{y_i} eq. 5

L_6=L_2 - \frac{1}{\pi ( N -1 )} \sum_{j=1,j\neq y_i}^{N} \arccos(W^T_{y_i}W_j) eq. 6

and an angular triplet constraint \arccos(x_i^{pos} x_i) + m \leq \arccos(x_i^{neg} x_i) §3.1.

Sub-center ArcFace §3.2

Each class gets K sub-centers, W \in \mathbb{R}^{512\times N\times K}, all L2-normalized. The scores W^T x_i \in \mathbb{R}^{N\times K} are max-pooled over k to one score per class, and the ArcFace loss is applied to those class scores §3.2, fig. 2:

L_7 = -\log\frac{e^{s\cos(\theta_{y_i}+m)}}{e^{s\cos(\theta_{y_i}+m)}+\sum_{j=1,j\neq y_i}^{N}e^{s\cos\theta_{j}}},\quad \theta_{j} = \arccos\Big(\max_k W^T_{j_k} x_i\Big),\ k\in\{1,\dots,K\} eq. 7

A noisy sample only has to be near one of its K positive sub-centers, so it tends to form a non-dominant sub-class instead of pulling the dominant one §1, §3.2. On MS1MV0 with ResNet50 and K=3, the authors measure 57.24% easy clean and 12.40% noisy samples in dominant sub-classes, and 4.28% hard clean and 26.08% noisy samples in non-dominant ones; with K=1 the noisy share near the center is 38.47% §3.2, fig. 7. No constraint ties sub-centers of one class together, so they may end up far apart, even orthogonal §3.2.

Algorithm: data cleaning with sub-center ArcFace  [§3.2, §4.1]
Input: noisy dataset D, K = 3, angle threshold 75 deg
1. Train sub-center ArcFace (K=3) on D with the normal LR schedule.
2. Offline: drop each class's non-dominant sub-centers (K=3 -> 1).
3. For each sample, compute the angle to its class's dominant sub-center;
   drop samples with angle > 75 deg ("high-confident noisy").
4. Retrain ArcFace (K=1) from scratch on the remaining data.
Output: cleaned dataset, final model

How the dominant sub-center of a class is chosen is not stated explicitly; the paper describes it as the sub-class holding the majority of (clean) samples §1, §3.2.

Inverting ArcFace §3.3, §4.5

Starting from Gaussian noise images, optimize pixels to minimize the ArcFace loss for a target class plus a BatchNorm statistics prior alg. 1, eq. 8:

L_8= \sum_{i=0}^L \|\tilde \mu_i^r - \mu_i\|_2^2 + \|\tilde \sigma_i^r - \sigma_i\|_2^2 eq. 8

where \tilde\mu_i^r, \tilde\sigma_i^r are mean / standard deviation of the activations at layer i for the generated batch and \mu_i, \sigma_i the values stored in the i-th BN layer. The objective is L_3 + \lambda L_8 with \lambda = 0.05 §3.3.

Algorithm 1: Face image generation from the ArcFace model  [alg. 1]
Input: model M with L BN layers, class label y_i
1. I_r <- random images from N(0, 1)
2. read mu_i, sigma_i from the BN layers of M
3. for j = 1..T:
     forward M(I_r), compute ArcFace loss
     compute batch mean/std of intermediate activations at each BN layer
     add statistic loss (eq. 8)
     backprop and update I_r
Output: generated images I_r

Open-set variant: replace the classification loss with an L2 loss between the generated image's embedding and a target embedding, so any feature vector (not only training class centers) can be inverted §3.3, §4.5. Total-variation and L2-norm image regularizers are also used §4.5.

Architecture and inference §4.1

Inputs are 112×112 crops aligned with five landmarks from RetinaFace. Backbones are ResNet50 / ResNet100 "without the bottleneck structure", followed by BN-Dropout-FC-BN to a 512-d embedding §4.1. At test time the FC classifier is dropped; template or video features are the mean of the per-image features §4.1.

Hyperparameters

Results

All numbers author-reported, verification accuracy in %, unless noted.

Loss ablation, [CASIA, ResNet50, loss] tab. 2:

Sub-center ablation, ResNet50, TPR@FPR=1e-4 tab. 3:

Benchmarks, ResNet100 ArcFace unless noted, with strongest listed baselines:

LFR2019 (MS1MV3 training, <1 GFlops track): EfficientNet-B0 ArcFace from scratch 86.44 (Image, TPR@FPR=1e-8) / 61.47 (Video, TPR@FPR=1e-4); with distillation from an R100 teacher (92.75 / 64.89), 88.65 / 63.60, versus challenge winners 88.78 (Image) and 63.23 (Video) tab. 9, §4.4.

Inversion with a [MS1MV3, ResNet50] model, scored by [IBUG-500K, ResNet100, ArcFace] tab. 10, tab. 11:

Real LFW faces score 99.81% with the inverting R50 model; its own inversions score 97.75% with the same model §4.5.

Paper vs code

Not checked. The paper says code and pre-trained models are available §1 and the authors' InsightFace project (github.com/deepinsight/insightface) is the likely source, but the repository was not cloned for this compilation, so no hyperparameter or step has been verified against code. code, code_commit and code_license are left empty until that comparison is done.

Pitfalls

• \theta_{y_i}+m can exceed π. For \theta_{y_i} > \pi - m (≈151° at m=0.5), \cos(\theta+m) starts increasing again, so the margin rewards moving away from the center. The paper never discusses this; it only plots target logits on [20°, 100°] §3.1, fig. 4. Decide how your implementation handles it (and check the official code once compared).
• arccos near ±1. Step 3 takes arccos of a cosine; its derivative diverges at ±1, and floating-point cosines can slightly exceed 1. Clamp the cosine before arccos, or compute \cos(\theta+m) from \cos\theta and \sin\theta directly. The paper does not say how it does this.
• Margin on target only. m is added only to the ground-truth class angle; non-target logits stay s\cos\theta_j eq. 3. At test time there is no margin and no classifier: compare normalized 512-d embeddings §4.1.
• s matters a lot. Normalized softmax with s=64 loses about 5 points on CFP-FP versus s=20 tab. 2; ArcFace's s=64 is tied to its margin. Don't transplant s across losses.
• Sub-center max pools non-target classes too. In eq. 7, every class's angle (including negatives) uses the closest sub-center, then the margin is added to the target. Memory and compute of the classifier scale by K; K=10 on MS1MV0 collapses IJB-C to 67.94 tab. 3.
• Sub-center is a cleaning tool, not the final model. The best numbers come from K=3 → drop → retrain from scratch with K=1 §3.2, tab. 3. On already clean MS1MV3, K=3 is slightly worse than K=1 tab. 3. The rule for picking the "dominant" sub-center is not spelled out.
• Under-specified training. Initial LR is given only for CASIA (0.1); for VGG2, MS1MV3 and IBUG-500K only the decay epochs are given §4.1. Dropout rate, whether batch 512 is global, warmup, and the exact "ResNet without bottleneck" block design are not stated §4.1.
• Center parallelism is advertised but not described. The introduction claims a "center parallel strategy" that trains millions of identities on one 8-GPU server §1; this version of the paper gives no details.
• Inversion statistics. eq. 8 uses standard deviations while the text says BN layers store "mean and variance" §3.3; convert running variance to std. The sum runs i=0..L for L BN layers, an off-by-one in notation. Which tensor (BN input vs output) supplies \tilde\mu, \tilde\sigma is not stated. The number of steps T is 20K §4.5.
• Typos in the ablation text. The last bullet of §4.3 compares (19) 96.91 with "(13) 96.50"; 96.50 is row (14) (MS1MV3, K=1) §4.3, tab. 3.
• MegaFace "R". Large-protocol identification numbers (98.98) use the authors' manually refined MegaFace; on the original set the same model scores 81.43 tab. 6. Compare only like with like.
• Template features. Template / video embeddings are a plain mean of per-image features; whether features are normalized before averaging is not stated §4.1.

Lineage

• Builds on: SphereFace (multiplicative angular margin), CosFace / AM-Softmax (additive cosine margin, s=64 setting), NormFace and L2-softmax (feature and weight normalization) §3.1. The conference version of ArcFace and the ECCV sub-center ArcFace paper are merged into this journal version §1.
• Compared against: Triplet / FaceNet, SphereFace, CosFace, CurricularFace, MV-Softmax, GroupFace, BroadFace, MC-FaceGraph; for noise robustness Co-mining, NT and NR; SoftTriple's multi-center softmax pooling and regularizer tab. 2-8.
• Superseded by: not known from content in this wiki.