Imprecise probability

Overview

A classifier that outputs one probability per label hides how much its ranking of the labels is trusted. Imprecise probability keeps a family of distributions instead of a single one. In the one paper here, that family is built from split-CP p-values: each input gives a credal set, a set of distributions that all agree with the p-value profile 2602.01667 §3.1. Its upper probability of an event is the largest normalised p-value inside that event 2602.01667 §3.1. The conformal prediction regions are the imprecise highest density regions of this set, so the threshold at each level gives one cut of the set 2602.01667 §3.2.

The point of the construction is to use the whole profile. A fixed-level set size keeps only the labels above one cut and ignores how they are ordered 2602.01667 §1. The summaries below use every p-value.

Variants

All variants read the same sorted, normalised p-values. They differ in the penalty applied to each cut or to each rank 2602.01667 §3.3. Each card link points to the same paper.

Generalised Hartley and entropy imprecision

Generalised Hartley weights each rank by the log of its size, and costs linear time after sorting 2602.01667 §3.3. Entropy imprecision is the largest entropy over the core, found by pooling adjacent masses until block averages are nonincreasing 2602.01667 §3.3. Under LAC scores, the paper reports U_1 (generalised Hartley) and U_2 above every tested fixed-level set size on selective prediction, on all four datasets 2602.01667 §5 tab. 1. Entropy imprecision scores close to U_1 on several conditions tab. 1. Card: Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry.

Tsallis-penalised summaries

A family of normalised weighted sums over the sorted p-values, indexed by q. q = 0 gives fuzziness, q = 1 gives Hartley normalised by log K, q = 2 gives a saturating form, and q going to infinity gives TV-MMI, which equals the second-largest p-value 2602.01667 §3.3 app. B. The paper does not tune q on evaluation labels 2602.01667 app. G. Card: Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry.

Probability baselines

Entropy, one minus the top probability, and the second-largest probability are the comparison points the paper uses 2602.01667 §5 app. D. They are not credal summaries. They appear here because the paper reports them against the credal summaries and they sometimes win 2602.01667 app. E.

When to use which

Implementation sketch

The common core is: sort the normalised p-values, take the gaps between consecutive values as masses, then apply a rank penalty. The paper gives this in words and formulas, not as code 2602.01667 §3.3. The card's pseudocode is a sketch and has not been run against the paper's numbers 2602.01667 §3.3 app. B.

# Shared core of the credal summaries  [arxiv:2602.01667 §3.1] [§3.3]
# Input:  pvals: (K,) split-CP p-values for one input
# Output: masses and sorted values, from which each summary is a sum

def credal_core(pvals):
    # Sort largest first; this order is shared by every summary  [§3.3]
    alphas = sorted(pvals, reverse=True)
    # Raise the maximiser to one so the core is nonempty  [§3.1]
    alphas[0] = 1.0
    # Mass of rank j is alpha_j minus alpha_(j+1), with a zero pad  [§3.3]
    padded = alphas + [0.0]
    masses = [padded[j] - padded[j + 1] for j in range(len(alphas))]
    return alphas, masses

Variants differ only in the penalty applied after this step. The generalised Hartley and integrated penalties weight each rank's mass by g(j) 2602.01667 §3.3. Entropy imprecision needs an extra isotonic step on the masses first 2602.01667 §3.3. The paper's Tsallis summaries are normalised sums of the sorted values 2602.01667 §3.3 app. B.

Contradictions

Only one paper cites this concept, so there are no disagreements between papers yet. Within that paper, the headline result does not hold across score families. With LAC scores, U_1 beats fixed-level set size everywhere and beats the strongest probability baseline on Caltech-101 and ImageNetV2, but matches entropy on CIFAR-100 and loses to probability baselines on Tiny ImageNet 2602.01667 §5 app. E. With APS, RAPS and SAPS, the simple 1 - q_(1) baseline is above all five credal summaries on selective prediction 2602.01667 §5 app. E. The paper reports both results, and so does this page.

Papers

• Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry: defines credal sets from split-CP p-values and gives linear-time uncertainty summaries, with author-reported gains under LAC and a regression impossibility result.