Out-of-distribution detection

Overview

A classifier trained on a fixed set of classes often answers with confidence on inputs from outside that set. Out-of-distribution (OOD) detection asks whether an input belongs to the training distribution, so the system can refuse or flag the rest 2203.10807 §1. The usual setup is a score computed from a trained network, where a higher score means more OOD, and a threshold chosen on held-out in-distribution (ID) data to keep 95% of ID inputs 2203.10807 §6 2205.09310 §2.1.

Scores read one or more of three signals: the feature vector, the logits and the normalised probabilities. Probability and logit scores miss outliers that the class weights throw away, and feature scores ignore class-specific evidence 2203.10807 §1 2203.10807 §3.

The methods differ in what they change. Post-hoc scores (ViM, max logit) leave the classifier's training alone. Training-time methods keep the score and change the loss (LogitNorm) or the recipe (max logit with a stronger recipe). MOS changes the output head: it adds an "others" class to each group of classes and retrains only the last layer 2105.01879 §3.1 §4.2. Self-supervised methods learn features without labels, then fit a distance to them (SSD) 2203.10807 §1 2205.09310 §3.2 2110.06207 app. D 2103.12051 §3.

The same idea appears in three neighbouring settings. Open-set classification asks a closed-set classifier to reject unseen classes and scores it with its largest logit 2110.06207 §4. Open-world object detection asks a detector to label unseen objects as unknown and learn them later 2103.02603 §3. In language-model work the ID data are helpful dialogue prompts and the OOD target is harmful prompts 2204.05862 §5.4. A robot planner can apply the same test to its skills: a skill whose learned Q-function ensemble disagrees too much is rejected 2303.12153 §4.5.

The sources use different ID data and different metrics, so their results are not directly comparable. This page rests on eight paper cards.

Variants

Feature residual with a virtual logit (ViM)

The feature vector is shifted so the logits carry no bias. Its component outside the top principal directions of the training features is measured. That norm, scaled by a constant fitted on training data, becomes one extra logit, and the softmax probability of that logit is the score 2203.10807 §4. Nothing is retrained and no outlier data is used 2203.10807 §1. It reads the feature and the logits together, labelled "feat+logit" in the paper's table 2203.10807 tab. 2. Card: ViM: Out-Of-Distribution with Virtual-logit Matching.

Maximum logit on a stronger closed-set classifier

The score is the largest raw logit of the last linear layer, with no softmax 2110.06207 §4. The paper's gains come mostly from training the classifier harder: 600 epochs, a cosine schedule with two warm restarts, RandAugment and warmup 2110.06207 app. D. The authors tie the score's strength to closed-set accuracy. Across the standard benchmarks the two correlate at Pearson 0.95, and the correlation is weaker across ImageNet-scale model families 2110.06207 §3.1 2110.06207 §3.2. The same recipe with the usual softmax score is the paper's own control, so the score effect can be read off directly 2110.06207 tab. 1. The open-set treatment is on the open-set recognition page. Card: Open-Set Recognition: a Good Closed-Set Classifier is All You Need?.

Energy score with Weibull fits in a detector (ORE)

The detector gets an extra unknown class, and its training examples come from background boxes. The paper picks the highest-objectness background proposals that do not overlap a labelled object and treats them as unknown 2103.02603 §4.2. A contrastive clustering loss pulls the RoI features of each class towards a prototype and keeps the prototypes apart 2103.02603 §4.1. At test time the energy of the known-class logits is computed 2103.02603 §4.3. Two Weibull densities are fitted to energies, one for known and one for unknown samples, and a detection is labelled unknown when the unknown density is larger there 2103.02603 §4.3. When new labels arrive, the model is fine-tuned on the new classes together with a balanced set of old examples, 50 per class, so earlier classes are not lost 2103.02603 §4.4 2103.02603 §5.2. Card: Towards Open World Object Detection.

Mahalanobis-style distance to in-distribution prompts

The RLHF study scores each prompt with a Mahalanobis-style distance to its helpfulness data, using the language model's activations. The authors report that this score can flag most harmful prompts, and that a few harmful examples improve it further 2204.05862 §5.4 fig. 22 fig. 23. The reported AUROC grows with model size, and middle layers work best for large models 2204.05862 fig. 22 fig. 23. The card gives these as trends only. It does not state the layer, the covariance estimator or the AUROC values. Card: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. This is a side analysis in a study of RLHF, not its main result.

Self-supervised features with a single-Gaussian distance (SSD)

Stage one trains a backbone on unlabeled images, with a contrastive loss that pulls together two augmented views of the same image and pushes other images apart 2103.12051 §3 eq. 1. Stage two fits one Gaussian to the normalised training features and scores each test input by its Mahalanobis distance to it 2103.12051 §3.1 eq. 2. The core method uses no OOD data for training or tuning 2103.12051 §5. A few-shot variant with five OOD samples per class subtracts an OOD term from the score and reaches 92.5 average AUROC on the four CIFAR-10 pairs, against 88.7 without it 2103.12051 tab. 1. A label-aware variant replaces the contrastive loss with a supervised one 2103.12051 §3.3. Card: SSD: A Unified Framework for Self-Supervised Outlier Detection.

Training-time logit normalisation (LogitNorm)

The score stays the usual maximum softmax probability, but training changes. The cross-entropy loss is applied to the logit vector divided by its L2 norm and by a temperature tau, so every training logit vector has the same length 2205.09310 §3.2 eq. 4. The authors link overconfidence on OOD inputs to logit norms that keep growing during ordinary training 2205.09310 §3.1 fig. 1. On CIFAR-10 with WRN-40-2 the average FPR95 across six OOD sets falls from 49.52 to 15.65, and on CIFAR-100 from 79.82 to 58.49 2205.09310 tab. 1. The paper's default tau for CIFAR-10 is 0.04 2205.09310 §4.1. Card: Mitigating Neural Network Overconfidence with Logit Normalization.

Grouped "others" classes for large label spaces (MOS)

The classes are split into groups, here the 8 ImageNet super-classes as the default taxonomy. Each group gets its own softmax with an extra "others" class, and the network is trained on the sum of the per-group cross-entropy losses. Every training image is a target for every group, so it also teaches the groups it does not belong to, and no outside data is used [arxiv:2105.01879 §3.1] eq. 1 eq. 2. The score is the negated minimum "others" probability over the groups 2105.01879 §3.2 eq. 3. The main results freeze a pretrained ImageNet backbone and retrain only the last layer [arxiv:2105.01879 §4.2]. The paper motivates the method by showing that flat-softmax scores fall as the class count grows: MSP's FPR95 rises from 17.34% at 50 classes to 76.94% at 1,000 2105.01879 §2 fig. 2. Card: MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space.

Ensemble variance rejecting learned skills (Text2Motion)

Each skill in this robot planner comes with an ensemble of Q-functions. A skill is treated as out of distribution when the variance of the ensemble's Q-values at its optimised parameters reaches a per-skill threshold, and plans that contain such a skill are rejected 2303.12153 §4.5 eq. 13. The thresholds are fixed per skill and calibrated on 100K invalid state-action pairs for each skill 2303.12153 app. A.2. The paper reports planning success, not an OOD metric. Its headline is 82% success on six simulated tabletop tasks, against 13% for prior language-based planners 2303.12153 §1. Card: Text2Motion: From Natural Language Instructions to Feasible Plans.

When to use which

The rows follow the constraints the papers state. Numbers are author-reported. The ViM rows use ImageNet-1K as the in-distribution set, with four OOD sets 2203.10807 §6. The last rows are for different settings: an RLHF language model, and a robot planner.

Implementation sketch

The ViM score needs the classifier's weight matrix W and bias b, and the feature vector x. The origin shift uses the pseudo-inverse of W 2203.10807 §3 eq. 1. Fitting is offline; scoring runs per input.

# Fit the residual basis and alpha, offline  [arxiv:2203.10807 §4]
# Inputs: train_feats (K, N), train_logits (K, C), origin (N,), D
# Output: residual (N, N - D), alpha (scalar)
def fit_vim(train_feats, train_logits, origin, D):
    N = train_feats.shape[-1]
    # Shift so the logits are bias-free  [arxiv:2203.10807 §3 eq. 1]
    centred = train_feats - origin                     # (K, N)
    # Eigenvectors of X^T X, smallest eigenvalues first  [§4 eq. 4]
    _, eigvecs = eigh(centred.T @ centred)             # (N, N)
    # The N - D smallest-eigenvalue directions span the residual space  [§4]
    residual = eigvecs[:, : N - D]                     # (N, N - D)
    # Match the residual norm to the mean maximum logit  [§4 eq. 6]
    residual_norm = norm(centred @ residual, axis=-1)  # (K,)
    alpha = train_logits.max(axis=-1).mean() / residual_norm.mean()
    return residual, alpha

# Score one input; higher means more out-of-distribution  [§4 eq. 5] [eq. 8]
def vim_score(feature, logits, origin, residual, alpha):
    # Shift to the bias-free origin  [arxiv:2203.10807 §3 eq. 1]
    centred = feature - origin                         # (N,)
    # Virtual logit: scaled norm of the residual  [§4 eq. 5]
    virtual_logit = alpha * norm(centred @ residual)
    # Log-odds of the ViM probability; same ranking  [§4 eq. 8]
    return virtual_logit - logsumexp(logits)

# Maximum logit score; higher means more in-distribution  [arxiv:2110.06207 §4]
def mls_score(logits):
    # Largest raw logit, no softmax; the sign is the opposite of ViM's  [§4]
    return max(logits)

# MOS score; higher means in-distribution  [arxiv:2105.01879 §3.2] [eq. 3]
# Input:  logits split into K groups, each with its "others" logit
# Output: score for one test input
def mos_score(group_logits):
    others_probs = []
    for g in group_logits:
        # Softmax within one group, including its "others" logit  [§3.1]
        probs = softmax(g)
        # Probability of the "others" slot in this group  [§3.2]
        others_probs.append(probs[others_index])
    # Negated minimum "others" probability over the groups  [eq. 3]
    return -min(others_probs)

# LogitNorm training loss  [arxiv:2205.09310 §3.2] [eq. 4]
# Input:  logits (B, k); labels (B,); temperature tau
# Output: scalar loss
def logit_norm_loss(logits, labels, tau):
    # Length of each logit vector, with a small epsilon for stability  [§3.2]
    lengths = norm(logits, axis=-1, keepdims=True) + 1e-7   # (B, 1)
    # Rescale to length 1/tau, then ordinary cross-entropy  [§3.2] [eq. 4]
    scaled = logits / (lengths * tau)                        # (B, k)
    return mean(cross_entropy(scaled, labels))

# SSD score; higher means more out-of-distribution  [arxiv:2103.12051 §3.1] [eq. 2]
# Input:  train_feats (N, d) from unlabeled in-distribution training images
# Output: score for one test feature
def ssd_score(train_feats, test_feat):
    # Unit-length features  [arxiv:2103.12051 alg. 1]
    train_z = normalise_rows(train_feats)              # (N, d)
    test_z = test_feat / norm(test_feat)               # (d,)
    # Mean and sample covariance of the training features  [§3.1]
    mu = mean(train_z, axis=0)                         # (d,)
    sigma = covariance(train_z)                        # (d, d)
    # Squared Mahalanobis distance to the training Gaussian  [eq. 2]
    diff = test_z - mu                                 # (d,)
    return diff @ inverse(sigma) @ diff

Notes for implementers:

• Threshold. Set it on held-out ID data, not on the set used to report FPR95 or AUROC. The paper says tuning on the evaluation ID set inflates results 2203.10807 app. E.
• Principal-space dimension D. The paper's rule is 1000 when N > 1500 and 512 otherwise 2203.10807 §6. The reported runs use N = 2048 (BiT) and N = 768 (ViT). For ViT-style models, the paper suggests D between N/3 and 2N/3 2203.10807 §6.4. The results change little across a wide range of D 2203.10807 fig. 4.
• Eigen-solver. The paper gives only the eigendecomposition of X^T X. A symmetric solver is the natural choice for a covariance matrix; whether the official code's general solver gives the same subspace was not checked 2203.10807 §4 eq. 4.
• Scale. The constant alpha is a ratio of two averages over training data. Scaling it by a factor lowered average performance in the paper's sweep 2203.10807 §6.4 fig. 5.
• Max logit depends on the recipe. The gains in the max-logit rows of 2110.06207 tab. 1 come from the stronger training recipe as well as from the score. Compare against the same recipe with the softmax score before crediting the score alone.
• Prompt scores. The RLHF card gives no threshold rule and no layer choice for its distance score. Fix both on your own held-out data 2204.05862 §5.4 fig. 23.
• LogitNorm changes only training. The primary test score is the maximum softmax probability of the raw logits, unnormalised 2205.09310 §4.2. Normalising logits at test time changes the scores, and the paper does not test that.
• LogitNorm tau. The paper's default for CIFAR-10 is 0.04, and it tunes tau on Gaussian noise 2205.09310 §4.1. Noise is not part of the released repository, and its defaults differ; see the LogitNorm card.
• SSD normalisation. The paper's algorithm normalises features by L2 norm only. The released evaluation code also standardises each dimension, a step the paper does not describe; see the SSD card.
• MOS grouping and training. The paper's grouping experiments are not reproducible from the release, which ships only the taxonomy grouping. The released training also adds mixup and a random flip, which the paper does not describe, so following eq. 2 alone does not match the authors' run; see the MOS card.
• MOS fine-tuning depth. Unfreezing more residual blocks raises accuracy but can hurt OOD detection in some cases, which the authors link to label overfitting 2105.01879 §4.3.5 fig. 8.

Contradictions

Papers do not share one benchmark, so their results should not be ranked against each other. Two papers do test CIFAR-10 in and SVHN out: SSD reports AUROC 99.6 with ResNet-50 2103.12051 tab. 1, and LogitNorm reports AUROC 98.47 with WRN-40-2 after its loss change 2205.09310 tab. 1. The backbones and training budgets differ, so the two numbers are not a comparison.

ViM's table also lists a maximum-logit baseline, with 79.27 average AUROC on BiT and 94.57 on ViT 2203.10807 tab. 2. Those are not the MLS numbers in 2110.06207 tab. 1, which use other models and data. Both papers find that the max logit beats the softmax score, so the direction agrees.

Within ViM, the paper's rule for D 2203.10807 §6 and the rule in its released code differ for some values of N. Both give the same D for the two models reported, so the reported numbers are not affected.

SSD's own tables disagree in two places. The CSI baseline on CIFAR-10 in, CIFAR-100 out is 89.2 in one table and 92.2 in another 2103.12051 tab. 1 tab. 4. The SSD and supervised ImageNet AUROCs are swapped between two tables 2103.12051 tab. 2 tab. 6. Treat those baseline numbers as unsettled.

SSD's single-cluster choice was made with CIFAR-100 as the OOD set, and CIFAR-100 is also one of its reported test sets 2103.12051 app. B.2 tab. 1.

ViM and MOS both use ImageNet-1K as the in-distribution set with a BiT-S-R101x1 backbone, and both report on iNaturalist and Texture, but their OOD sets differ: ViM also uses OpenImage-O and ImageNet-O, while MOS uses SUN and Places 2203.10807 tab. 2 2105.01879 tab. 1. ViM's Texture set has four ImageNet-overlapping classes removed. Where the setups overlap, the numbers differ: MSP's iNaturalist AUROC is 87.92 in ViM's BiT row and 87.59 in MOS's table, and its Texture AUROC is 79.80 and 74.45. KL matching's iNaturalist FPR95 is 33.28 in ViM and 27.36 in MOS 2203.10807 tab. 2 2105.01879 tab. 1. The cards do not say whether the two papers used the same sample or the same KL-matching setup, so the gaps are not explained.

Papers

• ViM: Out-Of-Distribution with Virtual-logit Matching: a post-hoc score that adds a virtual logit from the feature residual to the class logits. No retraining and no OOD data are needed.
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?: a stronger closed-set training recipe with a maximum-logit score, which matches or beats prior open-set methods on standard benchmarks.
• Towards Open World Object Detection: an energy score with fitted Weibull densities, plus pseudo-labelled unknowns and exemplar replay, for detectors that must learn new classes later.
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback: an RLHF study whose side analysis uses a Mahalanobis-style distance on activations to flag harmful prompts, with trends reported only.
• SSD: A Unified Framework for Self-Supervised Outlier Detection: contrastive self-supervised features on unlabeled data, scored by a single-Gaussian Mahalanobis distance, with few-shot and label-aware variants.
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space: group softmax with an "others" class per semantic group, retraining only the last layer, scored by the minimum "others" probability for ImageNet-scale label spaces.
• Mitigating Neural Network Overconfidence with Logit Normalization: cross-entropy on L2-normalised logits with a temperature, which keeps logit norms fixed during training and lowers OOD FPR95 on CIFAR.
• Text2Motion: From Natural Language Instructions to Feasible Plans: a robot planner whose learned skills are rejected when their Q-function ensembles disagree, judged by planning success rather than OOD metrics.