--- kind: concept slug: mahalanobis-distance title: Mahalanobis distance summary: >- A distance that uses the inverse covariance, so correlated or differently scaled features are not treated alike. Three uses: choosing similar past cases before fitting a local model (one clinical paper, where weighting helped Naive Bayes but not a Bayesian network), and two OOD scores, one on labelled class centroids (ViM baseline) and one on unlabelled self-supervised features (SSD). tags: [similarity-matching, anomaly-detection, clinical-decision-support, ood-detection, self-supervised-learning] papers: ["2103.12051", "2203.10807", "2605.04664"] related: [anomaly-detection, out-of-distribution-detection, bayesian-network, nearest-neighbor-search, contrastive-learning] status: draft updated_at: 2026-10-31 --- ## Overview The task is to measure how far apart two points are, given their feature vectors. The distance is a squared quadratic form. For two points x_i and x_j, with a matrix Γ, it is [arxiv:2605.04664 p. 3]: $$ r^2(\mathbf{x}_i, \mathbf{x}_j) = (\mathbf{x}_i - \mathbf{x}_j)\, \Gamma^{-1}\, (\mathbf{x}_i - \mathbf{x}_j)^T $$ The shared core is this quadratic form. Setting Γ^{-1} to the identity gives the Euclidean case. Setting Γ to the population covariance Σ gives the Mahalanobis case [arxiv:2605.04664 p. 3]. The three papers below use the same form for different jobs. - **Neighbour selection.** Rank past cases by distance to a target and fit a local model on the closest ones [arxiv:2605.04664 p. 2] [arxiv:2605.04664 p. 3]. - **Out-of-distribution (OOD) scoring on labelled classes.** Use the distance from a feature to the nearest class mean as an OOD score. The Mahalanobis baseline in the ViM paper is this use [arxiv:2203.10807 §1] [arxiv:2203.10807 app. E]. - **OOD scoring on unlabelled data.** Fit one Gaussian to the normalised features of unlabelled in-distribution images and score by distance from its mean. No class labels are used [arxiv:2103.12051 §1] [arxiv:2103.12051 §3.1]. ## Variants ### Euclidean form (Γ^{-1} = I) The paper writes this as the special case of its general form [arxiv:2605.04664 p. 3]. It is not among the reported experiments, which compare only the other two selections [arxiv:2605.04664 tab. 2]. Card: [Evidence-based anomaly detection in clinical domains](/papers/2605.04664). ### Mahalanobis neighbour selection (Γ = Σ) The population covariance Σ replaces Γ. The paper does not state which sample Σ is estimated from [arxiv:2605.04664 p. 3]. The k = 40 nearest cases are kept [arxiv:2605.04664 p. 4]. Reported with a Naive Bayes model: AUC PR 0.49 and AUC ROC 0.77, against 0.42 and 0.74 with all cases. Reported with a Bayesian network: AUC PR 0.52 and AUC ROC 0.79, against 0.56 and 0.81 with all cases [arxiv:2605.04664 tab. 2]. Card: [Evidence-based anomaly detection in clinical domains](/papers/2605.04664). ### Weighted Mahalanobis (w scales each difference) Each attribute difference is multiplied by an importance weight w before the quadratic form. The weights come from the univariate dependence between each attribute in C(x) and the target. The experiments measure that dependence with the Wilcoxon rank-sum statistic [arxiv:2605.04664 p. 3] [arxiv:2605.04664 p. 4]. The paper does not state how the statistic becomes a weight: its scaling, sign or normalisation [arxiv:2605.04664 p. 3]. Reported with a Naive Bayes model: AUC PR 0.60 and AUC ROC 0.80. Reported with a Bayesian network: AUC PR 0.56 and AUC ROC 0.80 [arxiv:2605.04664 tab. 2]. Card: [Evidence-based anomaly detection in clinical domains](/papers/2605.04664). ### Class-centroid Mahalanobis OOD score The score is the minimum Mahalanobis distance from a feature to the class centroids [arxiv:2203.10807 §1]. It uses the final feature only; the paper reports that adding lower-level features degrades the BiT result, to 56% average AUROC [arxiv:2203.10807 app. E]. The class-wise mean vectors and the precision matrix are estimated from 200,000 random training samples, and the ground-truth class labels are used [arxiv:2203.10807 app. E]. Results are author-reported on ImageNet-1K ID data with four OOD sets: average AUROC 86.62% and FPR95 53.34% with BiT-S-R101x1, and 96.02% and 19.45% with ViT-B/16 [arxiv:2203.10807 tab. 2]. Scoring time, excluding the forward pass, is 2135.13 s on OpenImage-O against 1.49 s for ViM, on a Skylake CPU with numpy [arxiv:2203.10807 tab. 8]. Card: [ViM: Out-Of-Distribution with Virtual-logit Matching](/papers/2203.10807). ### Self-supervised feature Mahalanobis score (SSD) No labels are needed for the normal data [arxiv:2103.12051 §1]. A backbone is first trained with a contrastive loss on unlabelled in-distribution images [arxiv:2103.12051 §3, eq. 1]. The score then uses one cluster: the mean and sample covariance of the L2-normalised training features, and the squared distance of a normalised test feature from them [arxiv:2103.12051 §3.1, eq. 2]. The paper's few-shot extension subtracts a second Mahalanobis term built from OOD samples, with Ledoit-Wolf shrinkage for that covariance [arxiv:2103.12051 §3.2, eq. 3, eq. 4]. Author-reported, averaged over four CIFAR-10/100 pairs: AUROC 88.7 against 80.9 for the best unlabelled baseline with a full row, and 92.5 with five OOD samples per class [arxiv:2103.12051 tab. 1]. The backbone is ResNet-50 trained for 500 epochs at batch size 512 [arxiv:2103.12051 §4.1]. The paper states no compute cost. Temperature matters: on a ResNet-18 ablation, AUROC is 66.7 at tau = 0.001 and 89.5 at tau = 0.5 [arxiv:2103.12051 tab. 5]. Card: [SSD: A Unified Framework for Self-Supervised Outlier Detection](/papers/2103.12051). ## When to use which | Situation | Use | Why | |---|---|---| | Naive Bayes model on a small labelled set, neighbourhood wanted | Weighted Mahalanobis | Highest Naive Bayes AUC PR (0.60) and AUC ROC (0.80) of the three selections [arxiv:2605.04664 tab. 2]. One leave-one-out run over 100 cases [arxiv:2605.04664 p. 4] | | Bayesian network model, parameters must fit many cases | All cases, no neighbourhood | The network on all cases had the highest AUC ROC (0.81) and tied the weighted variant on AUC PR (0.56) [arxiv:2605.04664 tab. 2]. The authors link the drop on 40 neighbours to small samples [arxiv:2605.04664 p. 5] | | Weights must not see the case being scored | Unweighted Mahalanobis, or weights from training data only | The paper does not say which data the rank-sum statistic used. If it included the scored case, the weights leak its outcome [arxiv:2605.04664 p. 3] [arxiv:2605.04664 p. 4] | | Deciding between neighbour-selection variants on small differences | Do not rely on the ranking alone | The gaps are small, such as 0.56 vs 0.52 AUC PR, and no variance is reported [arxiv:2605.04664 p. 4] | | OOD detection on a trained classifier, ground-truth labels available, and scoring cost matters | ViM over the class-centroid score, if the paper's numbers transfer | ViM has higher average AUROC on BiT (90.91% vs 86.62%) and is close on ViT (96.23% vs 96.02%) [arxiv:2203.10807 tab. 2]. The class-centroid score is far slower to score than ViM [arxiv:2203.10807 tab. 8]. Both results are on ImageNet-1K only | | OOD detection with unlabelled in-distribution data, and you can train a contrastive backbone | SSD-style score | Label-free and above the unlabelled baselines on average [arxiv:2103.12051 tab. 1]. It is not uniformly better than supervised Mahalanobis: on CIFAR-100 in with Texture OOD it reaches 82.9 AUROC against 98.8 [arxiv:2103.12051 tab. 2] | | Labels for the normal data exist and a contrastive backbone can be trained | SSD with labels, or the class-centroid score | With labels, the paper's supervised variant reaches 92.4 AUROC against 91.6 for contrastive plus supervised training [arxiv:2103.12051 tab. 4]. The gap is small and the paper reports no variance | | OOD detection without ground-truth labels, and no training is allowed | Mahalanobis is not applicable as the paper sets it up | The ViM paper's class means use ground-truth labels [arxiv:2203.10807 app. E]. The SSD variant needs training, but it uses no labels [arxiv:2103.12051 §3] | Open choices for any implementation: the sample used for Σ, the weight formula, and whether to regularise Σ. The paper leaves all three unstated [arxiv:2605.04664 p. 3]. With 18 binary attributes and 40 neighbours, a covariance estimate can be unstable. This is an implementer's concern, not a result of the paper. For the SSD variant, the paper's Algorithm 1 normalises features by L2 norm only [arxiv:2103.12051 alg. 1]. Whether to also standardise each dimension is an implementer's choice; the SSD card lists what the released code does [SSD card](/papers/2103.12051). ## Implementation sketch Adapted from the papers' descriptions; neither gives pseudocode. The first sketch is neighbour selection, the second the OOD score. The SSD variant differs from the second only in its inputs: features are L2-normalised, the training set is unlabelled, and there is a single cluster with no minimum over classes [arxiv:2103.12051 §3.1] [arxiv:2103.12051 alg. 1]. ```pseudocode # Weighted Mahalanobis neighbour selection [arxiv:2605.04664 p. 3] # Input: target: vector C(x) for the scored case, shape (d,) # past: past cases, one row each, shape (n, d), target excluded # weights: w, shape (d,); all ones gives plain Mahalanobis # k: neighbourhood size (40 in the paper) # Output: indices of the k nearest past cases def select_neighbours(target, past, weights, k): # Population covariance. The paper does not say which sample; using # past is an implementer's choice [arxiv:2605.04664 p. 3] cov_inv = inverse(covariance(past)) # Scale each attribute difference by its importance weight [p. 3] diff = weights * (past - target) # element-wise, (n, d) # Squared distance per row; same ranking as the unsquared form [p. 3] dist = sum((diff @ cov_inv) * diff, axis=1) # Keep the k nearest cases as the training set E [p. 4] return argsort(dist)[:k] ``` ```pseudocode # Class-centroid Mahalanobis OOD score [arxiv:2203.10807 §1] [app. E] # Input: train_feats (K, N), train_labels (K,): ground-truth labels # feature (N,): the input to score # Output: scalar, higher = more OOD def fit_centroids(train_feats, train_labels, classes): # Class-wise mean vectors, from ground-truth labels [app. E] means = stack([train_feats[train_labels == c].mean(axis=0) for c in classes]) # (C, N) # One precision matrix from the class-centred features [app. E] centred = train_feats - means[train_labels] # (K, N) precision = inverse(centred.T @ centred / len(train_feats)) return means, precision def mahalanobis_ood_score(feature, means, precision): # Squared distance to every class centroid [§1] diffs = feature - means # (C, N) dists = sum((diffs @ precision) * diffs, axis=1) # (C,) # Distance to the nearest centroid; the paper's score is this minimum [§1] return min(dists) ``` The paper names the precision matrix but not its estimator beyond the sample size, so the centred empirical covariance above is an implementer's choice [arxiv:2203.10807 app. E]. ## Contradictions Papers that use the distance for different tasks, neighbour selection and OOD scoring, measure different things, so their numbers should not be compared directly. The two OOD papers also use different ID data and networks. The SSD paper's supervised Mahalanobis rows [arxiv:2103.12051 tab. 2] are on CIFAR-100 and are not the ViM baseline on ImageNet-1K [arxiv:2203.10807 tab. 2]. Within the clinical paper, the neighbourhood step does not help every model. It raised the Naive Bayes AUC PR but lowered the Bayesian network's AUC ROC from 0.81 to 0.79 [arxiv:2605.04664 tab. 2]. The authors attribute the Bayesian network's drop to its complexity and small samples [arxiv:2605.04664 p. 5]. Within the SSD paper, the reported gains depend on the OOD set. Against the supervised Mahalanobis baseline, SSD is ahead on CIFAR-10 OOD from CIFAR-100 (69.6 against 55.3) and behind on Texture (82.9 against 98.8) [arxiv:2103.12051 tab. 2]. ## Papers - [SSD: A Unified Framework for Self-Supervised Outlier Detection](/papers/2103.12051): uses the Mahalanobis distance of normalised self-supervised features, fitted on unlabelled in-distribution images, as a label-free OOD score. - [ViM: Out-Of-Distribution with Virtual-logit Matching](/papers/2203.10807): uses a class-centroid Mahalanobis score as one of its baselines for OOD detection; the paper's own method, ViM, scores with a virtual logit instead. - [Evidence-based anomaly detection in clinical domains](/papers/2605.04664): uses a weighted Mahalanobis distance to choose the k most similar past patients, then scores a decision with a Bayesian network or Naive Bayes model fitted on them.