---
kind: concept
slug: uncertainty-estimation
title: Uncertainty estimation
summary: >-
  Scoring how unsure a model is about an input, to rank unlabeled examples, flag unreliable answers or set an act-or-ask rule. Families: softmax margin and entropy (cheap, often overconfident), MC dropout with BALD, ensembles, GPs and SNGP, sampled-answer agreement, LLM self-grading, conformal sets, reject thresholds, per-box variance heads. Evidence is thin: eighteen cards, mostly author-reported.
tags: [uncertainty, calibration, active-learning, sampling, conformal-prediction, rejection, llm-self-evaluation, annotator-disagreement, evidential-learning, imprecise-probability, object-detection, lidar]
papers: ["2009.00236", "2203.11171", "2305.04933", "2401.01301", "2307.01928", "2006.10108", "2107.11277", "2410.12405", "2006.12024", "2107.03342", "2207.05221", "2110.05719", "2204.11423", "2201.07766", "2602.01667", "1810.02890", "2404.18416", "1903.08701"]
related: [active-learning, monte-carlo-dropout, self-consistency, llm-self-evaluation, hallucination-detection, calibration, ensembles, conformal-prediction, llm-planning, gaussian-process, spectral-normalization, learning-with-rejection, anomaly-detection, prompt-engineering, llm-evaluation, bayesian-neural-network, variational-inference, evidential-deep-learning, multimodal-fusion, multi-task-learning, toxicity-detection, imprecise-probability, physics-informed-neural-network, neural-operator, human-in-the-loop, imitation-learning, retrieval-augmented-generation, object-detection, bev-representation, gaussian-mixture-model]
status: draft
updated_at: 2026-11-01
---

## Overview

Uncertainty estimation turns a model's outputs on one input into a number that
says how unsure the model is. The number is used in three ways: to rank unlabeled
examples so that a human labels the most useful ones (active learning), to
decide whether to trust an answer, or to set a threshold that decides when a
system acts and when it asks a person [arxiv:2307.01928 §3.2]. The survey frames the core difficulty: a
softmax output looks like a confidence score but is often overconfident, so
"label what the model is least sure about" can pick badly [arxiv:2009.00236 §2].
The families differ in what they read from the model: one deterministic pass,
the spread across stochastic passes or several models, distance from the
training data, or the agreement among sampled generations [arxiv:2009.00236
§3.1] [arxiv:2305.04933 §3.1] [arxiv:2203.11171 §3.5]. In active learning the
score is computed on the unlabeled pool, before the oracle labels the chosen
batch [arxiv:2009.00236 §3].

A second distinction is between noise in the data and a lack of data. The
tutorial splits total predictive variance into aleatory (irreducible) and
epistemic (reducible) parts, and argues the split matters for deciding whether
more data or a different model will help [arxiv:2305.04933 §2.1] [§2.2]. The
broader survey adds a third kind, distributional uncertainty, for inputs that
come from a different distribution than the training data [arxiv:2107.03342
§2.4.1] [§2.4.2]. Work on physics-informed networks uses the same two-part
split: the total variance is a noise term plus an epistemic term estimated
from posterior samples [arxiv:2201.07766 §3.1].

Regression models can also output a spread for each prediction, and use it as
the uncertainty directly rather than through repeated passes or several models.
A LiDAR detector predicts a log standard deviation for each box it proposes
[arxiv:1903.08701 §3.3], and the spread then drives how duplicate boxes are
pruned [§3.6].

Calibration, whether a stated 95% interval holds about 95% of the time, is a
separate property from ranking quality. The tutorial measures it with
reliability curves, expected calibration error (ECE) and NLL [arxiv:2305.04933
§6.1]. The survey names temperature scaling, a single value fitted after
training that divides the logits, as a common post-hoc fix for overconfident
softmax outputs. It keeps the predicted class but has limited expressiveness
[arxiv:2107.03342 §5.1].

## Variants

### Softmax margin and entropy

Rank by the gap between the two most likely labels (margin; smallest gap first),
or by the entropy over all classes (largest first), both from one deterministic
pass [arxiv:2009.00236 §3.1.2] [eq. 5]. No change to the model and no extra
passes are needed. The survey warns that softmax response has been reported to
do worse than random sampling in cited work [arxiv:2009.00236 §2], so treat it
as a baseline to beat, not a default.

### Monte Carlo dropout with BALD

Keep dropout active at prediction time, average several stochastic passes, and
score each example by the mutual information between prediction and model
weights (BALD) [arxiv:2009.00236 §3.1.1] [eq. 3]. The survey notes that dropout
has been shown equivalent to variational inference, which is the justification
for reading the passes as posterior samples [arxiv:2009.00236 §3.1.3]. BALD
scores each example on its own, so a top-K batch can be full of near-duplicates
[arxiv:2009.00236 §3.1.1] [fig. 2]. The joint score BatchBALD [eq. 4], or an
explicit redundancy term [eq. 7], addresses this. The number of passes and the
dropout rate are not stated in the survey. See
[Monte Carlo dropout](/concepts/monte-carlo-dropout).

The tutorial treats MC dropout as the scalable Bayesian option. Its case study
uses a dropout rate of 0.10 and 15 passes [arxiv:2305.04933 §6.1]. It warns that
the result depends strongly on the dropout rate, the number of dropout layers
and the training length [arxiv:2305.04933 §3.2]. On its case studies, MC dropout
was overconfident and had the largest run-to-run variation of the methods
compared [arxiv:2305.04933 §6.1] [§6.2].

The Bayesian neural network survey places MC dropout inside variational
inference, with a Bernoulli variable on each unit's input [arxiv:2006.12024
§2.4]. Its toy regression shows MC dropout with variance that is too large
inside the data and larger outside it, against a GP baseline [arxiv:2006.12024
§3, fig. 7]. That is one run with no dropout rate given. The mutual-information
score has a blind spot: if every pass agrees on the same wrong class, the score
stays small even for an input far from the training data [arxiv:2107.03342
§4.1.3]. Pair it with a distribution-shift check if out-of-distribution inputs
matter. See [Bayesian neural network](/concepts/bayesian-neural-network).

### Ensembles

Train several models and use their disagreement as the uncertainty. The survey
reports that Beluch et al. found ensembles gave better-calibrated uncertainty
than MC-dropout and density-based selection, and also offset class imbalance
during acquisition [arxiv:2009.00236 §3.1.4]. The DEBAL line of work combines
ensembles with MC-dropout, motivated by overconfidence from mode collapse in
variational inference [arxiv:2009.00236 §3.1.3]. The cost is several trainings;
the ensemble size is not stated in the survey.

The tutorial's variant gives each member a predicted mean and variance, trained
on the Gaussian negative log-likelihood. The average member variance is read as
aleatory uncertainty and the spread of the member means as epistemic
uncertainty [arxiv:2305.04933 §3.3]. Its case study uses 15 members, chosen by an
elbow plot [arxiv:2305.04933 §6.1]. The authors report that this ensemble had
the lowest NLL on each battery test set. Its ECE was not uniformly lowest: on
turbofan unit 15 it was 14.9%, against 8.5% for SNGP and 11.5% for MC dropout
[arxiv:2305.04933 §6.2]. The epistemic term depends on how diverse the seeds and
mini-batch orders make the members [arxiv:2305.04933 §3.3].

### Ensemble doubt at human takeovers

In interactive imitation learning, the learner drives until a human expert takes
over at a point they judge unsafe, and labels are collected only during those
takeovers. The doubt at each takeover is the norm of the diagonal of the
ensemble's output covariance. The threshold is the mean of the last quarter of
the logged doubts [arxiv:1810.02890 §II] [eq. 3] [eq. 4]. As printed, that sum
and its divisor count different numbers of entries, so an implementation has to
pick a convention [eq. 4]. The paper uses the threshold only to draw an
estimated safe set for evaluating the trained policy [§IV-B] [eq. 5]. Using doubt
as a control gate at test time is proposed as future work [§V]. The ensemble
size, network, optimiser and covariance estimator are not stated [§II].

Results are author-reported. In simulation, collisions and road departures are
about 12 and 20 times lower inside the estimated safe set than outside it
[§IV-B] [tab. 1]. On a real vehicle, five fixed obstacle scenarios gave zero
collisions and zero road departures for this method, against one collision and
one to six departures for the baselines [§IV-C] [tab. 2]. The authors call that
test too limited for statistical significance [§IV-C]. Card:
[HG-DAgger](/papers/1810.02890).

### Posterior samples for physics-informed networks and neural operators

A physics-informed network (PINN) is trained on data and on the residual of a
differential equation. A DeepONet learns a mapping from input functions to
solutions. The scientific-ML paper applies the same posterior machinery to
both [arxiv:2201.07766 §4.1] [§4.2]. Its Bayesian options are HMC, Langevin
dynamics, mean-field variational inference, MC dropout and a Laplace
approximation. Its ensemble options are independent members, snapshots from
one cyclic-learning-rate run, and SWAG, which fits a Gaussian to those
snapshots. Each method yields M parameter samples, and the predictive variance
is the noise variance plus the spread of the sample predictions [§3.1].

On its prototype problems the paper reports author-reported results. On a
one-dimensional regression with known noise, MC dropout had the lowest
relative L2 error (21.9 against 22.9 for HMC, on a ×10² scale), and MFVI had
the lowest calibrated RMSCE, 3.8 against 5.7 to 8.0 for the other methods
[arxiv:2201.07766 §6.1]. The paper states that post-training calibration
leaves all methods roughly equally calibrated on that problem [§6.1]. Its
cheaper methods (LA, MFVI and snapshot ensembles) matched costlier ones mainly
when the prior and the noise scale were learned during inference [§7]. With
fixed priors, the paper reports that epistemic uncertainty did not grow
reliably with less data or more noise [§6.1]. Costs are indicative: 50 HMC
steps take about four to five times standard training, and M serial ensemble
members take M times standard training [§6.6].

Post-training calibration uses a left-out set and offers three methods:
scaling the variance by one constant, fitting a correction to the predictive
CDF, or fitting an empirical distribution to scaled residuals [arxiv:2201.07766
§5.2]. The calibration set comes from the training distribution, so the
authors warn that performance may fall on out-of-distribution inputs
[§5.2]. The same paper reports that MC dropout's uncertainty did not rise on
out-of-distribution inputs, which it flags as a risk [§6.1]. The authors have
not released code. Card: [Uncertainty Quantification in Scientific Machine Learning](/papers/2201.07766).

### Per-annotator heads and annotator disagreement

Train one shared encoder with one output head per annotator. Each head learns
only from the instances its annotator labelled, and the majority over the heads
gives the label [arxiv:2110.05719 §3.4] [§4.3.1]. The uncertainty is the
disagreement among the heads: the product of the counts of positive and
negative head labels, divided by the square of their total. It therefore lies
between 0 and 0.25 [arxiv:2110.05719 §4.3.2] [eq. 1] [§5.2]. The authors report
that this measure tracks annotator disagreement better than softmax confidence
and MC dropout on the GHC hate-speech data and on GoEmotions, with joy on the
full GoEmotions data as the exception [arxiv:2110.05719 §4.3.2] [fig. 2] [§4.4]
[fig. 4]. Its mean is 0.170 on wrong predictions and 0.049 on correct ones
[§5.2] [fig. 5]. These results measure agreement with annotators, not
calibration, so they do not show that a stated probability is correct. On GHC,
majority-vote F1 is 59.49 against 57.32 for majority-label training. The authors
call the gap significant but report no statistical test [arxiv:2110.05719
§4.3.1] [tab. 1]. The method needs labels from each annotator, and the authors
suggest clustering annotators to limit the number of heads [§6.2]. Card:
[Dealing with Disagreements](/papers/2110.05719).

### Gaussian processes and SNGP

Gaussian process regression (GPR) gives a posterior variance that rises away
from the training inputs, so its uncertainty reflects distance from the data
[arxiv:2305.04933 §3.1]. Exact GPR costs O(N³) in training time and O(N²) in
storage, which the tutorial cites as the reason to leave it out of its larger
turbofan study [arxiv:2305.04933 §3.1] [§6.2]. On the battery training set GPR
fits exactly but generalizes poorly, and its test-set uncertainty is high
[arxiv:2305.04933 §6.1].

Spectral-normalized Gaussian processes (SNGP) keep distance information in a
single deterministic pass. They use residual hidden layers with spectral
normalization, bounded by a constant γ that must be tuned, and replace the
output layer with a Gaussian process approximated by random Fourier features
and a Laplace approximation [arxiv:2305.04933 §3.4]. SNGP cannot separate
aleatory from epistemic uncertainty [arxiv:2305.04933 §3.4]. On the battery
data it was often underconfident [arxiv:2305.04933 §6.1]. The tutorial also
reports that SNGP-type methods gave higher ECE than MC dropout and ensembles on
other benchmarks [arxiv:2305.04933 §3.4].

The SNGP paper introduces the method and reports its own results. It keeps the
network deterministic: spectral normalization on the hidden weights, and an
output layer that is a random-feature Gaussian process with a closed-form
posterior covariance [arxiv:2006.10108 §3.1] [§3.2]. Its results are averaged
over 10 seeds on CIFAR-10 and CIFAR-100 (Wide ResNet-28-10) and on CLINC OOS
(BERT-Base), against a 10-model deep ensemble [arxiv:2006.10108 §5]. On CIFAR-10
clean inputs, SNGP's ECE was 0.018, against 0.010 for the ensemble and 0.023 for
the deterministic network. Its per-example latency was 6.25 ms, against 38.10 ms
for the ensemble [arxiv:2006.10108 tab. 2]. Prediction averages the softmax over
10 Gaussian logit samples [arxiv:2006.10108 §5] [app. A]. The released code, as
recorded on [the paper's card](/papers/2006.10108), uses a mean-field
approximation at prediction time and a squared-loss likelihood in the Laplace
step, which differ from the text. Implement from the text and check the code
before reusing its defaults.

### Per-box variance heads with adaptive NMS for LiDAR detection

The detector runs on the sensor's range image in one pass. Each point predicts
a mixture of oriented box distributions, and each component has a log standard
deviation trained with a Laplace-style box loss [arxiv:1903.08701 §3.3] [§3.5]
[eq. 9]. Points that fall on one object are merged as a product of Gaussians,
weighted by inverse variance [§3.4] [eq. 6]. Duplicate boxes are then pruned by
an adaptive NMS. Its overlap limit comes from the two predicted spreads, so
uncertain boxes may overlap more before one is dropped [§3.6] [eq. 11]. The
number of mixture components depends on the class: three for vehicles and one
for pedestrians and bikes on the large dataset [§4.1].

The uncertainty here is a regression output, not a classifier score, and the
paper reads it as localisation uncertainty. Results are author-reported. On the
ATG4D validation set, vehicle AP0.7 over 0-70 m is 85.34, against 80.99 for
PIXOR and 85.17 for ContFuse with LiDAR and RGB [arxiv:1903.08701 tab. 1]. In
the ablation, the adaptive threshold with soft NMS scored best [tab. 2]. The
calibration check passes on ATG4D but not on the smaller KITTI set [§4.5]
[fig. 6], and on KITTI the detector trails the BEV baselines [tab. 4]. The paper
does not say how to obtain ATG4D [§4.1]. Card:
[LaserNet](/papers/1903.08701).

### Evidential Dirichlet opinions fused across views

For an input described by several views, such as sensors or feature sets, each
view's network outputs non-negative evidence for each class. The evidence gives
a Dirichlet distribution, and its strength S yields an uncertainty mass of K/S
for K classes [arxiv:2204.11423 §3.1] [§3.2] [eq. 11]. The views are merged
with a reduced Dempster-Shafer rule that discounts conflicting mass. An
uncertain view therefore counts for less, and the joint opinion carries its own
uncertainty [arxiv:2204.11423 §3.3] [eq. 13]. The loss adds a KL term that pulls
the evidence for wrong classes toward zero. The weight of that term is raised
during training, and the paper does not give the schedule [§3.1] [§3.6]. Both
fused models are close to late fusion in cost [tab. 5]. Author-reported: test
samples with added Gaussian noise get higher uncertainty [arxiv:2204.11423
§4.1] [fig. 5]. On SUN RGB-D, dropping samples whose uncertainty exceeds the
largest belief lowers the misclassification rate from 0.387 to 0.187, using the
paper's own rule [§4.2] [fig. 8] [fig. 9]. The card notes that the released
RGB-D scripts pick checkpoints on the test loader, so these numbers may be
optimistic. Card: [Trusted Multi-View Classification](/papers/2204.11423).

### Agreement across sampled answers

For a language model that ends in a short answer that can be compared for
equality, draw m reasoning samples, take the majority answer, and use the share
of samples that agree with it as an uncertainty signal. On GSM8K the agreement
share tracked accuracy [arxiv:2203.11171 §3.5] [fig. 5]. This is not an
active-learning score: the method returns the majority answer and its agreement
ratio. The cost scales with m, and the headline results use 40 full decodes per
question [arxiv:2203.11171 §3.2]. See [Self-consistency](/concepts/self-consistency).

The legal-hallucination study applies the same idea to open-ended answers. Two
temperature-1 answers to one prompt are compared by a GPT-4 judge. A
contradiction shows that at least one answer is wrong, so the contradiction rate
is only a lower bound on hallucination [arxiv:2401.01301 §4.3] [eq. 2]. For
calibration, confidence is the share of 10 samples that match the greedy answer
[arxiv:2401.01301 app. D.1]. On that setup all four models were overconfident,
with pooled ECE from 0.057 (PaLM 2) to 0.421 (Llama 2) [arxiv:2401.01301 §5.3].
The judge's agreement with two human coders was κ 0.87 and 0.77 on 100 queries
[arxiv:2401.01301 app. C].

The Med-Gemini study uses a related gate: entropy over sampled answers decides
whether the model searches again. For each question it samples several reasoning
paths and takes the Shannon entropy of the answer-choice distribution across
them. At or below a threshold, the majority answer is returned. Above it, the
model writes three queries that would settle the disagreement, and their results
join the next round's prompt [arxiv:2404.18416 §2.1]. The paper describes the
measure in words, and the log base and threshold value are not stated [§2.1].
The gate is applied to the three text benchmarks only [§2.1]. On MedQA, accuracy
rises from 87.2% to 91.1% across search rounds, and self-training is credited
separately with 3.2 points [§4.1] [fig. 4]. Card:
[Capabilities of Gemini Models in Medicine](/papers/2404.18416).

### LLM self-evaluation: P(True) and P(IK)

Two signals come from the model grading itself. P(True) asks the model whether
one of its own sampled answers is correct, as a True/False question, and uses
the probability it puts on "True". The comparison variant draws five answers at
temperature 1, judges one, and shows all five in the prompt. With 20 few-shot
examples it is the variant the paper reports as best [arxiv:2207.05221 §4.2]
[fig. 11] [app. A.5]. The zero-shot prompt is poorly calibrated and sits near
50% for typical samples [arxiv:2207.05221 §4.1] [app. B].

P(IK) is a value head trained on the model's own labels. For each question,
30 samples at temperature 1 are marked right or wrong, and every sample becomes
one training example for a head that reads the last input token
[arxiv:2207.05221 §5] [app. A.6]. Trained on TriviaQA, P(IK) reaches AUROC 0.864
on held-out TriviaQA questions [tab. 1]. When the head is trained on TriviaQA
only, AUROC on the other tasks is 0.928 for mixed arithmetic, 0.687 for Python
function synthesis, 0.624 for GSM8k and 0.606 for LAMBADA [tab. 1]. Adding the
other tasks to training raised the GSM8k AUROC to 0.752 [tab. 1]. Calibration
is good on TriviaQA and poor elsewhere, where the head is often underconfident
[fig. 13]. Prepending a relevant Wikipedia article raised P(IK) for an obscure
question from 18% to 78% at 52B [arxiv:2207.05221 §5.3].

Both signals were measured on the authors' own pretrained models, from 0.8B to
52B parameters. No weights or code are linked, so the numbers cannot be
reproduced from the card [arxiv:2207.05221 §1.2] [§7].

### Conformal sets over LLM option scores

The model sees a few candidate next steps as a multiple-choice question, with
an extra "none of these" option, and its probability for each option is the
uncertainty input. Each calibration case scores one minus the normalized
probability of the correct option. The threshold is the empirical quantile of
those scores at level ⌈(N+1)(1−ε)⌉/N [arxiv:2307.01928 §3.1] [eq. 1]. A new
case keeps every option whose probability clears 1 − q̂. One kept option means
act; more means ask a person to choose [arxiv:2307.01928 §3.2]. The user sets ε,
so the target success rate is a dial rather than a tuned threshold. The paper's
dataset-conditional form [eq. 2] adjusts ε with an inverse Beta quantile and
gives a stated probability over the calibration sample. The guarantee assumes
calibration and test cases come from the same distribution, and that a human
answers correctly when asked [arxiv:2307.01928 §3.2]. For multi-step plans, each
episode gets the minimum per-step score, one threshold is shared across steps,
and the sets are rebuilt one step at a time at test time [arxiv:2307.01928 §3.3]
[eq. 4] [alg. 1]. Card: [Robots That Ask For Help](/papers/2307.01928).

Results are author-reported. On a mobile manipulator, help was 0.67 against 0.81 for a cumulative-
threshold baseline, with plan success 0.87 for both [arxiv:2307.01928 tab. 2].
On a tabletop arm, step-wise help was 0.58 against 0.72 at equal plan success
0.76 [arxiv:2307.01928 tab. 1]. The baseline's ε was tuned to match plan
success, not the target, so the comparison is not at equal target
[arxiv:2307.01928 §4.2] [§4.3]. Realized task success fell short of the 0.85
target on the mobile robot (0.76), which the authors partly attribute to the
low-level controller [arxiv:2307.01928 tab. 2] [app. A.8]. Raw sentence
probabilities gave larger sets than the multiple-choice score [arxiv:2307.01928
app. A.9]. Calibration used 400 labelled scenarios [arxiv:2307.01928 app. A.7].

### Credal summaries of conformal p-values

Set size at one error level keeps only the labels above one cut-off, and so
discards most of the p-value profile. The credal-geometry paper treats all
split-conformal p-values for one input as a family of compatible
distributions. It rescales the largest p-value to one, then derives summaries
that can be computed after sorting the labels: fuzziness, a generalised
Hartley measure, a family of Tsallis-penalised summaries, TV-MMI (equal to the
second-largest p-value) and entropy imprecision [arxiv:2602.01667 §3.1]
[§3.3] [app. B]. The Tsallis form at q = 1 is the normalised Hartley measure
U₁, which the paper uses as its main summary [§5].

On LAC scores, the paper reports that U₁ beat every tested fixed-level set
size on four image datasets for selective prediction, by 0.85 to 1.86 AUARC
points over the strongest set size [arxiv:2602.01667 §5] [tab. 1]. For
distribution-shift detection the picture is mixed: entropy imprecision was
largest on CIFAR-10.1 by a narrow margin, and fuzziness led on Fashion-MNIST
[§5] [tab. 1]. Under APS, RAPS and SAPS scores, the baseline 1 − q₍₁₎ beat all
five summaries on selective prediction [§5] [app. E]. Under LAC, TV-MMI is a
monotone transform of the second-largest probability, so it adds nothing to
ranking beyond that probability [app. D] [app. F]. For regression with a range
condition on the score, every input gets the same rank core, so the summaries
cannot separate inputs [arxiv:2602.01667 §3.4] [app. C]. All results are
author-reported on fixed, cached classifiers, with standard errors over
calibration splits only. Card: [Beyond Prediction-Set Size](/papers/2602.01667).

### Reject option with a threshold

A model with a reject option answers only when a score clears a threshold τ,
and otherwise passes the case on. The survey separates two causes for rejecting:
ambiguity, where the features do not determine the target, and novelty, where
the input is rare or absent from training [arxiv:2107.11277 §2.2]. Two
architectures carry this out. A separated rejector scores the input alone, for
example with a density or one-class model, and runs before the predictor
[arxiv:2107.11277 §4] [eq. 3]. A dependent rejector reads a confidence value from
the predictor, such as its largest class posterior, and rejects when that value
is below τ [arxiv:2107.11277 §5] [eq. 4] [eq. 5]. Because a separated rejector
never sees the predictor's output, it cannot reject ambiguous inputs
[arxiv:2107.11277 §4].

The threshold can be set from a target rejection rate ρ, from a rejection-rate
bound, from an error bound, or by Chow's cost rule [arxiv:2107.11277 §5]
[eq. 7]. The survey says Chow's rule needs calibrated posteriors and advises a
cost-based choice of τ when posteriors are estimated [arxiv:2107.11277 §5]. It
also warns that coverage and accuracy on answered inputs must be read together,
since either one alone favours a trivial setting [arxiv:2107.11277 §3.1]. The
survey runs no experiments and reports no benchmark numbers; it names the lack
of a standard benchmark as an open problem [arxiv:2107.11277 §10.2]. Card:
[Machine Learning with a Reject Option](/papers/2107.11277).

### Next-token confidence and prompt sensitivity

Prompt sensitivity measures how much one question's score changes when only the
wording of the prompt changes. PromptSensiScore averages, over instances, the
mean absolute score gap between the prompt variants of each instance
[arxiv:2410.12405 §2.2] [eq. 1] [eq. 2]. The same paper defines a decoding
confidence: for each instance, the mean over its prompts of the probability of
the most likely next token [arxiv:2410.12405 §5.1] [eq. 3]. On CommonsenseQA the
authors report that lower sensitivity goes with higher confidence across
instances, for three models [arxiv:2410.12405 §5.2] [fig. 7]. This is a
correlation on one dataset, and the figure gives no numbers. The paper does not
say whether the maximum runs over the whole vocabulary or over option tokens
[arxiv:2410.12405 §5.1]. Its authors also say confidence values cannot be
compared across models [arxiv:2410.12405 §5.2]. Treat it as a cheap signal to
check, not as a calibrated probability. Card: [ProSA](/papers/2410.12405).

## When to use which

| Situation | Use | Why |
|---|---|---|
| One model, one pass per example, need a cheap ranking | Margin or entropy [arxiv:2009.00236 §3.1.2] | No extra compute, but softmax confidence can be overconfident [arxiv:2009.00236 §2] |
| Model has dropout and can run several passes per example | MC dropout with BALD [arxiv:2009.00236 §3.1.3] | Gives a spread over predictions; use a joint or diversity-aware batch score so batches are not redundant [arxiv:2009.00236 §3.1.1]. Its results depend on dropout rate and training length [arxiv:2305.04933 §3.2] |
| Batch must be diverse, many classes, large batches | Diversity methods such as core-set [arxiv:2009.00236 §3.1.2] | The survey says diversity methods suit more classes and larger batches; core-set needs a pool-wide distance matrix [arxiv:2009.00236 §3.1.4] |
| Budget for several trained models, calibration matters | Ensembles [arxiv:2009.00236 §3.1.4] | Reported as better calibrated than MC-dropout in the survey; costs several trainings. In the tutorial, lowest NLL on battery data but not lowest ECE everywhere [arxiv:2305.04933 §6.1] [§6.2] |
| Regression on small data, need distance-aware uncertainty | GPR [arxiv:2305.04933 §3.1] | Variance grows away from training inputs; O(N³) cost in training points rules it out for large sets |
| Regression on larger data, one pass, distance awareness wanted | SNGP [arxiv:2305.04933 §3.4] | Single deterministic pass; γ must be tuned, and it was often underconfident on battery data [arxiv:2305.04933 §6.1] |
| Image or text classifier, one model, latency matters more than the last point of ECE | SNGP [arxiv:2006.10108 §3] | Author-reported: about 6 ms per example against 38 ms for a 10-model ensemble on CIFAR-10 [arxiv:2006.10108 tab. 2]; its clean ECE was above the ensemble's [tab. 2] |
| Need to split noise from missing data | Gaussian-head ensemble [arxiv:2305.04933 §3.3] | Gives aleatory and epistemic terms; the split depends on member diversity |
| LLM with short, comparable final answers, no training allowed | Agreement across samples [arxiv:2203.11171 §3.5] | Agreement tracked accuracy on GSM8K; costs about m full decodes per question [arxiv:2203.11171 §3.2] |
| LLM with open-ended answers and a judge model available | Judge-flagged contradiction between two samples [arxiv:2401.01301 §4.3] | Gives a lower bound on errors, not a calibrated probability; the study found overconfidence [arxiv:2401.01301 §5.3] |
| LLM planner that picks from a few listed options, a person can answer when asked, and a target success rate is needed | Conformal sets over option scores [arxiv:2307.01928 §3.1] | Sets how often the system asks from calibration data; needs labelled calibration cases from the test distribution [arxiv:2307.01928 §3.2] [app. A.7] |
| Abstain on inputs unlike the training data, scored from the input alone, before the predictor runs | Separated rejector with a density or one-class score [arxiv:2107.11277 §4] | Cannot catch ambiguous inputs; the density or one-class model is a design choice of its own |
| Abstain on ambiguous inputs using the predictor's own confidence, with a target rejection rate | Dependent rejector, τ set from ρ [arxiv:2107.11277 §5] | Chow's cost rule needs calibrated posteriors; a rate target fixes τ only on the data used to set it [arxiv:2107.11277 §5] |
| Check whether LLM answers stay stable across prompt wording, per item | Prompt sensitivity (PSS) on your own prompt set [arxiv:2410.12405 §2.2] | The paper used 12 prompt variants per item [arxiv:2410.12405 §3.1]; 0.1 is a rule-of-thumb threshold [§2.2]; subjective scores need a judge |
| Log a cheap per-answer confidence next to a model's answers | Next-token confidence [arxiv:2410.12405 §5.1] | Correlated with lower sensitivity on one dataset; not comparable across models [§5.2] |
| Open-weights LLM answering short factual or multiple-choice questions, no training allowed, token probabilities readable | P(True) with comparison samples [arxiv:2207.05221 §4.2] | Use few-shot prompting, since zero-shot P(True) is poorly calibrated [§4.1]; needs about five samples per question. Reported results are on the authors' own models, mainly 52B |
| Can fine-tune the model, have labelled in-domain questions, and need a "know it or not" score before answering | P(IK) value head [arxiv:2207.05221 §5] | Good on the training distribution, poor elsewhere [fig. 13]; needs 30 labelled samples per question and no reproducible code [§1.2] |
| Trained classifier with a labelled validation set whose confidences are too high | Temperature scaling [arxiv:2107.03342 §5.1] | One fitted number; keeps the predicted class, and the survey notes its expressiveness is limited [§5.1] |
| Several inputs describe one sample, some may be noisy or out of distribution, and the fused prediction needs its own uncertainty | Evidential Dirichlet fusion [arxiv:2204.11423 §3.3] | One network per view plus a fusion step; cost close to late fusion [tab. 5]. Gains are author-reported, and the RGB-D checkpoint choice used the test set (see [the card](/papers/2204.11423)) |
| Several annotators label each item and disagree, and each annotator has enough labels | Per-annotator heads [arxiv:2110.05719 §3.4] | Keeps annotator structure and gives a disagreement score [§4.3.2]; the F1 gain over majority labels is 2.2 points with no test [§4.3.1]; head count grows with annotators [§6.2] |
| Physics-informed network or neural operator, error bars needed, prior and noise scale can be learned | Cheap posterior methods (LA, MFVI, snapshot ensembles) [arxiv:2201.07766 §7] | Matched costlier methods when prior and noise were learned; 50 HMC steps cost about 4 to 5 times training [§6.6] |
| Same models, a left-out calibration set, intervals too narrow or too wide | Post-training variance scaling or CDF correction [arxiv:2201.07766 §5.2] | Fitted on training-distribution data, so it may not hold out of distribution (same section) |
| Split-conformal classifier with LAC scores, want a ranking finer than set size at one error level | Normalised generalised Hartley, U₁ [arxiv:2602.01667 §3.3] [§5] | Author-reported gain of 0.85 to 1.86 AUARC points over the strongest set size; only tested with LAC on four image datasets |
| Split-conformal classifier with APS, RAPS or SAPS scores | Max-probability baseline 1 − q₍₁₎ [arxiv:2602.01667 §5] [app. E] | The paper found it above every credal summary on selective prediction under these scores |
| Split-conformal regression, absolute residuals or conformalised quantile regression | Not the rank-core summaries [arxiv:2602.01667 §3.4] | Under the range condition every input gets the same core; response-space geometry would be needed |
| LLM answering multiple-choice questions with a search tool, can afford several samples per question, and should search again only when samples disagree | Entropy over sampled answer choices with a threshold gate [arxiv:2404.18416 §2.1] | Adds retrieval only on disagreement; the threshold and log base are not stated, and the reported gains combine search with self-training [§4.1] |
| Interactive imitation learning where a human can take over and judge danger in real time, and a doubt level is wanted from the takeover moments | Ensemble output-covariance doubt, threshold from late takeovers [arxiv:1810.02890 §II] | The threshold only draws an estimated safe set in the paper, not a control gate [§IV-B] [§V]; ensemble size not stated [§II]; the on-vehicle test has five scenarios [§IV-C] |
| Dense 3D detection from a spinning LiDAR, one pass wanted, per-box spread needed to tune duplicate removal, large labelled dataset available | Per-point box mixture with variance-aware NMS [arxiv:1903.08701 §3.6] | Calibration held on the large dataset but not on KITTI [§4.5]; trailed BEV detectors on KITTI [tab. 4]; dataset access not stated [§4.1] |
| Open-ended LLM output with no judge | Not covered by these sources | The self-consistency paper leaves a consistency metric for generations to future work [arxiv:2203.11171 §2] |

## Implementation sketch

Pool scoring for active learning, covering the margin, entropy and BALD
variants. Each step names its source. The number of passes T is not stated in
the survey, so it is a parameter here.

```pseudocode
# Score an unlabeled pool by predictive uncertainty, then take the top K
# [arxiv:2009.00236 §3.1] [§3.1.2]
# Input: model f, pool U, batch size K, method in {margin, entropy, bald}

def uncertainty_scores(model, pool, method, T):
    # Softmax probabilities from one deterministic pass  [§3.1.2]
    if method in ("margin", "entropy"):
        probs = softmax(model(pool))
    if method == "margin":
        # Gap between the two most likely labels; small gap = uncertain  [§3.1.2]
        top2 = sort_desc(probs)[:, :2]
        return -(top2[:, 0] - top2[:, 1])
    if method == "entropy":
        # Entropy over the k classes; large entropy = uncertain  [eq. 5]
        return -(probs * log(probs)).sum(axis=1)
    # Keep dropout on and draw T stochastic passes  [§3.1.3]
    passes = stack([softmax(model(pool, dropout=True)) for _ in range(T)])
    # Average softmax probabilities, not logits  [§3.1.3] [eq. 9]
    mean_p = passes.mean(axis=0)
    # BALD: entropy of the mean minus mean of the entropies  [eq. 3]
    return entropy(mean_p) - entropy(passes).mean(axis=0)

def select_batch(scores, K):
    # Top-K per-sample scores; near-duplicates can all rank high  [§3.1.1]
    return argsort_desc(scores)[:K]
```

For language models, the per-question analogue is the agreement share from
[arxiv:2203.11171 §2] and [§3.5], not a pool score. For open-ended answers, a
judge call that flags contradictions between two samples replaces the equality
test, giving a lower bound on errors [arxiv:2401.01301 §4.3].

For option-scored LLM choices, the threshold comes from calibration cases
rather than a pool. This sketch uses the marginal quantile [eq. 1] only; the
dataset-conditional adjustment [eq. 2] is left out.

```pseudocode
# Conformal threshold and act-or-ask rule  [arxiv:2307.01928 §3.1] [eq. 1]
# Input:  option_probs (N, k) normalized confidences, labels (N,) correct index
# Input:  epsilon in (0, 1), the user-set error level
# Output: qhat, and per case either "act" with an option or "ask" with a set

def conformal_threshold(option_probs, labels, epsilon):
    n = len(labels)
    # Nonconformity: one minus confidence of the correct option  [§3.1]
    scores = 1 - option_probs[range(n), labels]
    # Quantile level ceil((n+1)(1-eps))/n, capped at 1, rounded up  [eq. 1]
    level = min(ceil((n + 1) * (1 - epsilon)) / n, 1.0)
    return quantile(scores, level, method="higher")

def act_or_ask(probs, qhat):
    # Keep every option whose confidence clears 1 - qhat  [§3.1]
    kept = [i for i, p in enumerate(probs) if p >= 1 - qhat]
    # One option: act. Several: ask a person to choose  [§3.2]
    if len(kept) == 1:
        return ("act", kept[0])
    return ("ask", kept)
```

## Contradictions

- **Which family is better calibrated.** The survey reports that Beluch et al.
  found ensembles better calibrated than MC dropout [arxiv:2009.00236 §3.1.4].
  The tutorial reports MC dropout with the lower ECE on the battery secondary
  test set (9.5% against 13.5% for the ensemble), while on turbofan unit 15 the
  ensemble had the higher ECE (14.9% against 11.5% for MC dropout and 8.5% for
  SNGP) [arxiv:2305.04933 §6.1] [§6.2]. No single ranking follows.
- **Direction of error.** The tutorial reports that MC dropout was overconfident
  and that the ensemble and SNGP were underconfident on battery data
  [arxiv:2305.04933 §6.1]. The legal study reports that all four LLMs were
  overconfident [arxiv:2401.01301 §5.3]. The survey says softmax confidence is
  often overconfident [arxiv:2009.00236 §2]. The domains and metrics differ, and
  the sources do not reconcile them.
- **MC dropout direction.** The BNN survey's toy regression reports MC dropout
  variance too large inside the data [arxiv:2006.12024 §3] [fig. 7]. The
  tutorial reports MC dropout as overconfident on battery data [arxiv:2305.04933
  §6.1]. The two are different settings, and neither is a benchmark.
- **LLM calibration across formats.** The self-evaluation paper reports large
  models as well calibrated on letter-labelled multiple choice [arxiv:2207.05221
  §2] [app. A.1]. The legal study reports all four LLMs as overconfident on
  open-ended court questions [arxiv:2401.01301 §5.3]. The formats and tasks
  differ, and the sources do not reconcile them.
- **Agreement and accuracy.** The self-consistency paper reports that the
  agreement share tracked accuracy on GSM8K [arxiv:2203.11171 §3.5]. The legal
  study uses the same kind of agreement as confidence and finds it overconfident
  on legal questions [arxiv:2401.01301 §5.3] [app. D.1]. The tasks differ
  (arithmetic against legal recall), so neither result settles the other.
- **SNGP against ensembles on calibration.** The tutorial reports that SNGP-type
  methods gave higher ECE than MC dropout and ensembles on other benchmarks
  [arxiv:2305.04933 §3.4]. The SNGP paper reports SNGP's ECE on CIFAR-10 as
  0.018 clean and 0.090 corrupted, against 0.010 and 0.087 for the ensemble
  [arxiv:2006.10108 tab. 2], and on corrupted CIFAR-100 as lower than the
  ensemble's (0.117 against 0.138) [arxiv:2006.10108 tab. 3]. The benchmarks
  differ, so neither result settles the other.
- **Uncertainty against diversity.** The core-set and DFAL papers report that
  diversity methods always beat uncertainty methods, and that uncertainty can
  do worse than random sampling; the LLAL paper reports otherwise
  [arxiv:2009.00236 §5].
- **Conformal summaries against the probability baseline.** The credal-geometry
  paper reports that under LAC its U₁ summary beats every fixed-level set size
  [arxiv:2602.01667 §5] [tab. 1]. It also reports that under APS, RAPS and
  SAPS the max-probability baseline beats all of its summaries on selective
  prediction [§5] [app. E]. The score family decides the winner, and the
  sources do not reconcile the two results.
- **Random baselines.** Random-sampling accuracy differs by 13% between the
  LLAL and BGADL papers at the same CIFAR-10 setting, and one method's results
  differ by 8% between core-set and VAAL papers on CIFAR-100
  [arxiv:2009.00236 §5].

## Papers

- [A Survey of Deep Active Learning](/papers/2009.00236): taxonomy of uncertainty, Bayesian, ensemble and diversity query strategies, with a reproducibility warning. No experiments of its own.
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](/papers/2203.11171): samples many reasoning paths and votes; the agreement share tracks accuracy on GSM8K.
- [Liu et al., SNGP for distance-aware deterministic networks](/papers/2006.10108): SNGP, one deterministic network with spectral normalization and a random-feature Gaussian-process output layer. Author-reported results against a 10-model ensemble.
- [Nemani et al., a UQ tutorial for engineering design and prognostics](/papers/2305.04933): compares GPR, MC dropout, Gaussian-head ensembles and SNGP on battery and turbofan prognostics, using NLL and ECE. Author-reported results.
- [Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models](/papers/2401.01301): judge-based contradiction lower bound and ECE for four LLMs on US federal court questions. Author-reported results.
- [KnowNo, LLM planners that ask for help](/papers/2307.01928): conformal sets over LLM option scores that decide when a robot acts and when it asks. Author-reported robot results; the released code differs from the paper (see the card).
- [Hendrickx et al., Machine Learning with a Reject Option: A survey](/papers/2107.11277): taxonomy of ambiguity and novelty rejection, three rejector architectures and ways to set the threshold. No experiments of its own.
- [Zhuo et al., ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](/papers/2410.12405): PromptSensiScore for prompt wording, and a next-token confidence correlation on one dataset. Author-reported results.
- [Goan and Fookes, Bayesian Neural Networks: An Introduction and Survey](/papers/2006.12024): variational inference, Hamiltonian Monte Carlo, Bayes by Backprop and MC dropout as approximations. Two small single-run experiments.
- [Gawlikowski et al., A Survey of Uncertainty in Deep Neural Networks](/papers/2107.03342): data, model and distributional uncertainty; the main method families and calibration metrics. No experiments of its own.
- [Kadavath et al., Language Models (Mostly) Know What They Know](/papers/2207.05221): P(True) self-grading and a P(IK) value head; in-distribution AUROC 0.864 on TriviaQA, partial transfer. Author-reported; no code or weights.
- [Psaros et al., Uncertainty Quantification in Scientific Machine Learning](/papers/2201.07766): Bayesian samplers, ensembles, functional priors and post-training calibration for PINNs and DeepONets, compared on prototype problems. Author-reported; no official code.
- [Chau et al., credal summaries of conformal prediction sets](/papers/2602.01667): credal summaries of split-conformal p-values, with a regression impossibility result. Author-reported on image classifiers; no official code.
- [Mostafazadeh Davani et al., Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](/papers/2110.05719): one head per annotator on a shared BERT encoder; head disagreement as uncertainty. Author-reported on hate-speech and emotion data.
- [Han et al., Trusted Multi-View Classification with Dynamic Evidential Fusion](/papers/2204.11423): Dirichlet evidence per view, fused with a reduced Dempster-Shafer rule. Author-reported on RGB-D, vector and image-text data.
- [Saab et al., Capabilities of Gemini Models in Medicine](/papers/2404.18416): Med-Gemini models with an entropy gate that searches again only when sampled answers disagree. Author-reported; models not released.
- [Kelly et al., HG-DAgger: Interactive Imitation Learning with Human Experts](/papers/1810.02890): human-controlled takeovers with a doubt threshold learned from them. Author-reported simulation and a five-scenario vehicle test.
- [Meyer et al., LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving](/papers/1903.08701): per-point box mixtures on LiDAR range images, fused by mean shift and pruned by variance-aware NMS. Author-reported on ATG4D and KITTI; no official code.