A Survey of Uncertainty in Deep Neural Networks

In plain words

A neural network almost always gives an answer, and it often sounds sure of itself even when it is wrong or when the input looks nothing like its training data. This survey is about putting a number on how much to trust such an answer.

It separates two kinds of doubt. Some doubt comes from the data itself, because the measurement is noisy or two classes look alike. Other doubt comes from the model not knowing enough, because it was trained on too little or on different data. A third kind covers inputs that come from a different distribution than the training data.

The survey then walks through the main ways to get these numbers: a single network that reports its own doubt, Bayesian networks that keep a range of plausible weights, ensembles of several networks, and running one input through several augmented copies. It also covers how to measure whether the numbers are any good, and how to correct a network whose stated confidence is too high.

TL;DR

A survey of uncertainty estimation for deep neural networks, with no experiments of its own. It defines data, model and distributional uncertainty and groups the methods into single deterministic networks, Bayesian neural networks, ensembles and test-time data augmentation §2, §3. It then covers uncertainty measures and calibration metrics §4, §5. Use it as a map of the field and as a checklist of the costs and open problems; do not take any of its numbers as your own result.

Problem

Standard networks produce softmax probabilities that are often poorly calibrated, and a single softmax output cannot separate data uncertainty from model uncertainty §3.1. For out-of-distribution inputs, ReLU networks with a softmax output can become more confident the further the input lies from the training data §3.1. The survey argues that for mission- and safety-critical uses this missing information is the main obstacle §2.4.1.

Method

This is not a method paper. The card records its taxonomy, the methods it reviews and the metrics it defines, so an implementer can find the right technique and check the formulas.

Uncertainty taxonomy §2

• A network is a function f_θ from inputs to outputs, trained on a finite dataset D of N input and target pairs §2 intro.
• The predictive distribution integrates over the parameters, p(y*|x*, D), and is approximated because the true parameter distribution is unavailable §2.4.
• Data uncertainty (aleatoric) comes from information lost when the world is measured, for example label noise or overlapping classes. It is not reduced by more training data §2.4.1.
• Model uncertainty (epistemic) comes from the architecture, the training procedure or a lack of data. In principle it can be reduced §2.4.1.
• Distributional uncertainty is modelled as a separate layer between the model and the data terms. It describes the input distribution shifting away from training, and only some approaches represent it explicitly §2.4.2.
• By input domain, uncertainty is in-domain, domain-shift or out-of-domain. Model uncertainty covers all three; data uncertainty covers only the in-domain part caused by the data §2.5.

Single deterministic networks §3.1

• Each approach needs one forward pass or at most two. Training may be more complex, and the results can be sensitive to architecture and data §3.1.
• Internal methods change training so that the network outputs the parameters of a distribution over predictions. The paper groups the Dirichlet-based methods here, along with interval and quantile-based regression methods and RBF networks §3.1.
• External methods attach a second component to an already trained network. Examples estimate the error of the main prediction, use training-data density in representation space, or use gradients §3.1.
• Dirichlet density, with concentration parameters alpha_c > 0 and precision alpha_0, the sum of the concentrations §3.1. In practice the concentrations come from a positive transform of the logits, with exp as one example §3.1.
• Dirichlet prior networks train towards a sharp Dirichlet for in-domain samples and a flat one for out-of-domain samples, by minimising an expected KL divergence §3.1. A reverse-KL variant is discussed as an alternative §3.1.
• Evidential networks treat the outputs as evidence from subjective logic. Total evidence is set against the number of classes, which gives an "I do not know" signal, and a regulariser discourages evidence for several classes at once §3.1.

Bayesian neural networks §3.2

• Posterior: p(θ|x,y) is proportional to p(y|x,θ) p(θ), the likelihood times the prior §3.2.
• Prediction: marginalise over the posterior, p(y*|x*,x,y) = ∫ p(y*|x*,θ) p(θ|x,y) dθ. The Monte Carlo approximation averages N networks, each sampled from the posterior §3.2.
• The posterior is intractable, so an approximation is used §3.2. Three families are named:
  • Variational inference: pick a tractable family q(θ) and minimise KL(q||p), equivalently maximise the evidence lower bound (ELBO). Mean-field Gaussians, Bayes by backprop, the local reparameterisation trick and normalising flows are covered §3.2.
  • Sampling: MCMC, with Hamiltonian Monte Carlo as the gold-standard variant. HMC uses leapfrog integration plus a Metropolis accept step and needs the full dataset per iteration. Stochastic-gradient Langevin dynamics (SGLD) makes the step minibatch-based §3.2.
  • Laplace approximation: a Gaussian centred at the MAP estimate, with covariance from the Hessian and a prior precision tau §3.2.
• MC dropout treats dropout layers as variational inference. Its main practical point is that dropout stays on at test time, and the spread over stochastic passes gives the uncertainty §3.2.

Ensembles §3.3

• Prediction: f(x) = (1/M) Σ f_i(x), the average over M members §3.3.
• Diversity is the key requirement. Sources listed are random initialisation, data shuffling, bagging, boosting, data augmentation, different architectures, snapshot learning-rate schedules and attention masks §3.3.
• Uncertainty is read from the disagreement between members. Deep ensembles add a second output head that predicts data uncertainty for each member §3.3.
• Efficiency options: pruning, distillation into one network (including distribution distillation, which keeps the spread), sub-ensembles with a shared trunk, and BatchEnsembles §3.3.

Test-time augmentation §3.4

• Make M augmented copies of the input, run the network on each, and aggregate the predictions. Majority voting is used in one cited segmentation study §3.4.
• Learned aggregation functions and greedy policy search are cited as ways to choose valid augmentations §3.4.

Sample-based uncertainty measures §4.1.2

Given M predictive distributions p^i, the expectations are replaced by averages.

# Sample-based uncertainty measures  [§4.1.2]
# Input:  probs (M, K): one softmax vector per sample or ensemble member
# Output: mean prediction (K,) and mutual information (scalar)

def sample_uncertainty(probs):
    # Mean prediction, the average over the M samples  [§4.1.2]
    mean_probs = probs.mean(axis=0)              # (K,)
    # Entropy of the mean prediction  [§4.1.2, MI definition]
    total_entropy = entropy(mean_probs)
    # Average entropy of the individual predictions  [§4.1.2, MI definition]
    expected_entropy = 0.0
    for member_probs in probs:
        expected_entropy += entropy(member_probs) / len(probs)
    # Mutual information, read as model uncertainty  [§4.1.2]
    mutual_information = total_entropy - expected_entropy
    return mean_probs, mutual_information

• Mutual information is the entropy of the mean prediction minus the mean of the entropies. The paper reads it as model uncertainty §4.1.2.
• The expected KL divergence between each sample and the mean prediction is an alternative. The predictive variance of each class probability is a third §4.1.2.
• The paper notes that these measures are blind to distributional shift. If all samples agree on the same wrong class, the spread is small even though the input is out of distribution §4.1.3.

Regression measures §4.2

• Data uncertainty is the predicted standard deviation, or a prediction interval. Coverage is measured by PICP, the fraction c/n of ground-truth values inside the predicted intervals. Average width is measured by MPIW §4.2.
• Model uncertainty is measured as for classification, by the spread of predictions §4.2.
• The paper gives a quantile-based interval as ŷ ± ½ Φ⁻¹(α) σ. See Pitfalls before using it §4.2.

Calibration §5

• A model is calibrated when the accuracy in each confidence bin equals the mean confidence in that bin §5.2.
• ECE (top label, equal-width bins): the bin-size-weighted mean of |acc − conf| over M bins §5.2.
• SCE (all labels): averages the per-class calibration error over classes. The paper reports that all-label measures are more effective than ECE §5.2.
• aECE and aSCE: the same measures with equal-mass bins. The paper says these are more robust to the number of bins §5.2.
• Reliability diagrams plot confidence against accuracy per bin §5.2.

# Expected calibration error, top label, equal-width bins  [§5.2, ECE]
# Input:  confidences (N,): top-label probabilities; correct (N,): booleans
# Output: scalar ECE in [0, 1]

def expected_calibration_error(confidences, correct, bin_count):
    # bin_count is M; the paper does not state a default  [§5.2]
    error = 0.0
    for m in range(1, bin_count + 1):
        # Bin I_m = ((m-1)/M, m/M]  [§5.2]
        lower = (m - 1) / bin_count
        upper = m / bin_count
        in_bin = (confidences > lower) & (confidences <= upper)
        if not in_bin.any():
            continue
        # Gap between mean accuracy and mean confidence in the bin  [§5.2]
        gap = abs(correct[in_bin].mean() - confidences[in_bin].mean())
        # Weight by the bin's share of all samples  [§5.2]
        error += in_bin.sum() / len(confidences) * gap
    return error

• Temperature scaling replaces the softmax with softmax(z / T), with T > 0 fitted after training. T > 1 raises the entropy and lowers the confidence; T between 0 and 1 does the reverse. It keeps the predicted class, and the paper notes its expressiveness is limited §5.1.
• Other post-hoc methods named are a class-wise one-vs-all method that is equivalent to a dense layer on log-probabilities §5.1, standard-deviation scaling for regression §5.1 and a Gaussian-process calibration map §5.1. Regularisation-based and uncertainty-estimation-based calibration are also covered §5.1.

Datasets, baselines and applications §6, §7

• A table lists the benchmark tasks and datasets, and the baselines per taxonomy class tab. 4. Official implementations are starred. Where none exists, the survey links the highest-ranked GitHub implementation it found §6.
• Application areas: active learning, reinforcement learning, medical analysis, robotics and Earth observation §7.

Hyperparameters

The survey gives no training settings of its own. The values below are the ones the text states; anything else is marked not stated.

Results

Author-reported. The survey runs no experiments. The claims below are the survey's summaries of cited works. Check the original before relying on any of them.

Paper vs code

No official code. The survey has no method implementation to compare. The authors link a resource list on GitHub (JakobCode/UncertaintyInNeuralNetworks_Resources, commit 04ffc156f634df3370864bbcaac149a97712d2df). It is a README of links with no LICENSE file, so it is not recorded as code §6. Checked: the paper, its linked project resources, and the repository above.

Pitfalls

• No experiments of its own. Every number and claim belongs to a cited work. The survey explicitly says a broad real-world comparison is missing, so there is no ranking of methods for a given application §8.1.
• Regression interval §4.2: the printed formula is ŷ ± ½ Φ⁻¹(α) σ. The same paragraph defines Φ⁻¹(α) as the α-quantile of the standard normal. Under that definition, a central interval covering probability α uses Φ⁻¹((1 + α)/2) and Φ⁻¹((1 − α)/2). The ½ factor looks like a typo. This is our reading and the paper does not confirm it. Check the original before you implement it.
• ELBO §3.2: the printed expectation is E_q[log p(y|x,θ)/q(θ)]. The identity KL(q||p) = −L + log p(y|x) holds only if the denominator is the prior p(θ). Our derivation: the ELBO is E_q[log p(y|x,θ)] − KL(q||p(θ)). Implement the standard form. The paper does not flag this.
• Laplace sign convention §3.2: H is defined as the Hessian of log p(θ|x,y). At a maximum that matrix is negative definite, yet the paper writes the covariance as (H + τI)⁻¹ and treats H + τI as a precision. The convention is not stated. Use the negative Hessian as the precision, plus τI, and check the sign against the source of any method you adopt.
• Model uncertainty by softmax disagreement §4.1.2, §4.1.3: mutual information and related measures can be small on a wrong, out-of-distribution input when all samples agree. Pair them with a distributional-uncertainty method if OOD detection matters.
• Calibration measures §5.2: ECE and SCE depend on the number of bins, which the paper does not fix. The paper reports adaptive binning as less sensitive to this. Class imbalance can hide poor calibration in small classes, because the errors are weighted by sample count.
• Ensemble cost §3.3: memory and compute grow linearly with M, for training and inference. BatchEnsembles reduce the parameter count, but the members then cannot be trained independently. The paper says the time saving is not as large as the compute saving.
• Augmentation pitfalls §3.3, §3.4: test-time augmentation can turn correct predictions into wrong ones, so the augmentations must stay within the target distribution. The paper also cites work reporting that augmentation can harm the calibration of an ensemble, which means post-hoc calibration may need adjusting for ensembles.
• Benchmark baselines §6: where no official implementation exists, the survey links third-party code. Treat those links as unverified.
• Unstated values: no default number of members, samples, bins, interval level or training setting is given. See the Hyperparameters table.

Lineage

• Builds on: none stated; the survey covers the cited works in §3 to §5.
• Compared against: its own comparison is qualitative and summarises cited studies §3.3.
• Superseded by: not stated.