One-class classification

Overview

The task is to decide whether a new input belongs to the one class seen during training. Only examples of that class are available, so the method has to describe "normal" from them alone 2011.02578 §1. Two shapes of solution appear in the sources:

• Two-stage detectors. A representation is learned from inliers without labels, then a density estimator or a one-class SVM is fitted on those features 2011.02578 §3. The paper cites hypersphere collapse as a reason not to train a deep one-class model end to end 2011.02578 §1.
• Novelty rejectors. A one-class model sees only the input and rejects it when its score is too low, before any predictor runs 2107.11277 §4.

A review of class imbalance names one-class learning as one of its algorithmic remedies, without describing a method 1305.1707 §2.2.

On the paper's image benchmarks, the choice of representation matters more than the choice of boundary. For the same features, KDE and OC-SVM differ by at most 1.1 mean AUC, while the representations span 80.4 to 89.9 [arxiv:2011.02578 tab. 2].

Caveat: the paper picks its temperature and model selection on CIFAR-10 with 500 inlier and 500 outlier images. It acknowledges that this may break the one-class assumption 2011.02578 app. A.4.

Variants

Two-stage: contrastive features with distribution augmentation

Contrastive learning on inlier images only. Each training image is also added in its 90, 180 and 270 degree rotations, treated as separate distributions. This enlarges the training set four times 2011.02578 §3.1.3. The authors report the best mean AUC in their comparison: 89.9 with OC-SVM and 89.8 with KDE over five settings, without test-time augmentation [arxiv:2011.02578 tab. 2]. Plain contrastive features reach 86.9 and 86.8 on the same rows 2011.02578 tab. 2. The authors argue that plain contrastive features spread every sample nearly uniformly, which leaves outliers no room to stand out 2011.02578 §3.1.3. They chose batch size 32 on CIFAR-10 validation among the sizes they tried 2011.02578 §5.1. Card: Learning and Evaluating Representations for Deep One-class Classification.

Two-stage: rotation-prediction features

The backbone is trained to predict which of four rotations (0, 90, 180, 270 degrees) was applied to an augmented inlier image 2011.02578 §3.1.2. With a KDE on top, the paper reports mean 88.2; with OC-SVM, 87.1 [arxiv:2011.02578 tab. 2]. That is 1.7 mean AUC below the distribution-augmented contrastive best. Per-dataset standard deviations for these rows run from 0.3 to 1.7, so the gap is small. The paper reports that rotation prediction holds up better when the training set contains outliers 2011.02578 §5.3. Card: Learning and Evaluating Representations for Deep One-class Classification.

Frozen pretrained backbone

No self-supervised training. An ImageNet-pretrained ResNet-50 supplies the features, and OC-SVM or KDE the detector. The paper reports mean 84.0 with OC-SVM and 83.7 with KDE 2011.02578 tab. 2. Its backbone is trained on ImageNet rather than on the inlier class alone, so the one-class setting applies only to the detector. Card: Learning and Evaluating Representations for Deep One-class Classification.

Separated novelty rejector

A one-class model (a density estimate, a one-class SVM, or a novelty score squashed into a probability) acts as a filter. The input is rejected when the score falls below a threshold, and the predictor is not called [arxiv:2107.11277 §4]. A novelty score in which higher means more novel must be inverted or transformed first, since the rule rejects on low scores 2107.11277 §4. The survey reports no numbers for this variant. Because the filter never sees the predictor's output, it cannot reject ambiguous inputs 2107.11277 §4. Card: Machine Learning with a Reject Option: A survey.

Imbalance review: one-class learning as one remedy

A review of class imbalance lists one-class learning next to cost-sensitive learning, SVM variants and boosting 1305.1707 §2.2. It says one-class learning can outperform other methods on multi-dimensional data under certain conditions. It names no condition and cites one reference for the claim 1305.1707 §2.2. The review's only comparison is qualitative, and it gives no numbers for one-class learning 1305.1707 tab. 2. Treat the claim as the authors' opinion. Card: Class Imbalance Problem in Data Mining Review.

When to use which

The MVTec numbers come from a defect dataset, not the semantic protocol. The released code has no MVTec loader, so that table is not reproduced from the release (see the card's Paper vs code).

Implementation sketch

Two-stage detector, the common core of the paper's method. Variants differ only in how the backbone is trained.

# Stage 1: backbone trained on inlier images only  [arxiv:2011.02578 §3.1]
# Option A: rotation prediction, 4 classes  [§3.1.2] [eq. 1]
# Option B: contrastive, rotated copies as extra data  [§3.1.3] [eq. 2]
def train_backbone(inliers):
    return backbone

# Stage 2: fit the detector on normalised inlier features  [§3.2]
def fit_detector(backbone, inliers):
    feats = l2_normalize(backbone(inliers))  # (N, d) unit vectors
    # Kernel width from the training features; see Contradictions
    # [arxiv:2011.02578 app. A.4]
    gamma = 10 / (variance(feats) * feats.shape[1])
    return feats, gamma

# Anomaly score, higher = more anomalous  [app. A.1] [eq. 6]
def anomaly_score(x, backbone, feats, gamma):
    # Cosine similarity of the test input to every inlier feature
    sims = l2_normalize(backbone(x)) @ transpose(feats)
    # Log-sum-exp KDE score, sign flipped from the paper's normality score
    return -logsumexp(2 * gamma * sims, axis=1) / gamma

Contradictions

• Kernel width, within one paper. Section 3.2 says the width is reduced ten times relative to the default 2011.02578 §3.2. Appendix A.4 says γ is ten times larger 2011.02578 app. A.4. The released code uses the larger width. The card records this discrepancy and does not resolve it.
• Ranking depends on the metric. On the semantic benchmarks, distribution- augmented contrastive features lead. On MVTec, rotation with an MLP head ties it on detection (86.3 against 86.5) and beats it on localisation (93.0 against 90.4) 2011.02578 tab. 4.
• Printed mean. The card's recomputation of the rotation-KDE mean from the printed rows gives about 87.8, not the printed 88.2 2011.02578 tab. 2. Treat the printed Mean column as the paper's value and check it before quoting.
• One-class claim without numbers. The imbalance review says one-class learning can beat other methods under conditions it does not name 1305.1707 §2.2. It gives no numbers, so it cannot be set against the AUC figures of the two-stage detectors 2011.02578 tab. 2.
• Rejection numbers are not comparable. The survey states there is no standard benchmark for models with rejection, and most papers use custom or proprietary data 2107.11277 §10.2.

Papers

• Class Imbalance Problem in Data Mining Review: a review that lists one-class learning among class-imbalance remedies. Qualitative only.
• Learning and Evaluating Representations for Deep One-class Classification: two-stage detector with rotation-prediction and distribution-augmented contrastive features, fitted with KDE or OC-SVM.
• Machine Learning with a Reject Option: A survey: taxonomy of rejectors, including separated one-class-style novelty rejectors. No experiments.