Conformal prediction

Overview

Conformal prediction wraps a model with a rule that returns a set of candidate answers instead of one answer. The set is sized so that the true answer is in it with at least a chosen probability, 1−ε, on new inputs drawn from the same distribution as the calibration data 2307.01928 §3.1 eq. 1. The method needs a score for each candidate and a labelled calibration set. Each paper here plugs in a different score: the LLM's confidence in each option 2307.01928 §3.1, a classifier's probability for each label 2602.01667 app. D, a failure detector's score per control step 2606.21386 §3.3, or the error of a learned value function on held-out states 2412.09989 §V-A.4. The last one returns a level, not a set.

The split form used here works in three steps. Score each calibration example by how unlike its correct answer the model found it. Take the empirical quantile of those scores at a level set by N and ε, giving one threshold q̂. At test time, keep every candidate whose score clears that threshold [arxiv:2307.01928 §3.1] eq. 2.

Four papers apply this so far. The first applies it to language-model planners: a planner lists candidate next steps, the candidates become a multiple-choice question, and the set size decides whether a robot acts or asks for help 2307.01928 §2 2307.01928 §3.2. The paper reports this with author-reported results on tabletop and mobile-manipulation hardware 2307.01928 §4. The second uses the same kind of calibration differently. It does not form sets. It sets alarm thresholds on the scores of a failure detector for finetuned robot policies 2606.21386 §3.3. The third uses the split-CP p-values of each input to build uncertainty summaries. It does not make a decision set either. It ranks inputs by how uncertain they are, and compares its summaries with set size 2602.01667 §3.3 §5. The fourth calibrates a learned reachability value function for a quadruped. It takes a quantile of prediction errors on simulated states, and uses the result as the level at which a safety filter starts to act 2412.09989 §V-A.4 §V-B.2. This is the only paper here where the output is a level that gates a controller, not a set or an alarm.

The evidence is narrow. The planner paper's results come from its own tabletop and mobile-manipulation runs with PaLM-2L 2307.01928 §4. The detector paper's come from six real-world tasks and Libero-Plus, with two VLA models 2606.21386 §4. The credal paper's come from image classifiers on four datasets 2602.01667 §5. The quadruped paper's come from simulation with 100 trials per row and ten hardware trials per row 2412.09989 §VI-D §VII.

Variants

Single-step set over multiple-choice options

Options A to D come from the LLM, and a fifth option E ("an option not listed") is appended so that the right action can fall outside the list. The score is one minus the normalised confidence of the correct option. The threshold is set from N calibration scenarios and the chosen ε 2307.01928 §2 2307.01928 §3.1. A singleton set means the robot acts; a larger set means it asks a human to pick 2307.01928 §3.2. Card: Robots That Ask For Help.

Dataset-conditional bound

The marginal guarantee holds on average over draws of the calibration set. The dataset-conditional form instead holds with probability 1−δ over the calibration set itself. It needs an adjusted ε̂ taken from an inverse Beta quantile, with v = ⌊(N+1)ε̂⌋ 2307.01928 §3.1 eq. 2. The paper uses δ = 0.01 and N = 400 2307.01928 §3.1 app. A.7.

Sequence-level calibration for multi-step plans

Calibration uses whole correct episodes. Each episode gets a score equal to the lowest per-step confidence of its correct options, and one threshold covers the episode 2307.01928 §3.3 eq. 4. At test time the set is rebuilt at each step from the context reached so far, using the same threshold. The robot acts on singletons and asks otherwise 2307.01928 §3.3 alg. 1. The paper's hardware tabletop run reached task success 0.74 against a target of 0.75, with the multi-step sets 2307.01928 §4.2 tab. 1.

Several acceptable options

When more than one option is correct, the calibration label is the correct option that the LLM ranks highest. The set then has to contain that option, and the human is asked to choose the highest-ranked correct one [arxiv:2307.01928 §3.3] app. A.3 app. A.4. The appendix also reports a bimanual run on an all-ambiguous distribution 2307.01928 app. A.5.

Constant band for a failure-detector threshold

A detector gives a score per control step, and one threshold is set per score from calibration data. The alarm fires when either score is at or above its threshold 2606.21386 §3.3. The paper calibrates each threshold with a constant conformal band at significance level 0.05, using 20 successful calibration rollouts, which are removed from the test set [arxiv:2606.21386 app. D]. It does not use a time-dependent band, because episode lengths vary, for example in its Drawer task 2606.21386 §3.3 app. D. The band's formula is not on the card, so read the paper's appendix D before implementing it. Card: VLA-FAIL.

Calibrated level for a learned value function

A learned value function estimates how close a robot state is to failure. Its prediction error is measured on held-out states whose true value comes from a reachability solver 2412.09989 §V-A.2. The paper picks a confidence level β and a violation rate ε, finds the largest k that a binomial tail bound allows, and takes the (N − k)/N quantile of the errors as a level δ. With probability at least 1 − β, the fraction of samples whose predicted value overestimates the true value by more than δ is at most ε [arxiv:2412.09989 §V-A.4] eq. 11 eq. 12. The paper calls the proof a split-conformal argument with a Beta distribution 2412.09989 app. A eq. 17. At run time, the safety filter passes the nominal command through while the calibrated value is above δ, and solves a QP below it 2412.09989 §V-B.2 eq. 15. With β = 10^-12 and N = 10^7 samples, the paper tabulates δ = 0.22, 0.49 and 0.99 m for ε = 10^-1, 10^-2 and 10^-3, and uses 0.49 m 2412.09989 tab. II §V-A.4. Card: One Filter to Deploy Them All.

Credal summaries of the p-value profile

Keep all the split-CP p-values for an input, not only the labels above one cut-off. The p-values define a family of distributions that are all compatible with them, called a credal set. Its spread gives uncertainty numbers that take every label into account, and these come out in linear time once the p-values are sorted 2602.01667 §3.1 §3.3. Before the summaries are computed, the largest p-value is raised to one. The raw maximum is kept apart as conformal credibility 2602.01667 §3.1.

The summaries include generalised Hartley, entropy imprecision, TV-MMI and Tsallis-penalised forms 2602.01667 §3.3 app. B. TV-MMI equals the second-largest p-value 2602.01667 §3.3. Under LAC scores, it is only a relabelling of the second-highest probability, so it adds no new ranking 2602.01667 app. D app. F. The paper reports these on image classifiers. With LAC, the Tsallis summaries U_1 and U_2 beat every fixed-level set size on selective prediction, with paired gains of 0.50 to 1.86 points 2602.01667 §5 tab. 1. With APS, RAPS and SAPS, the simple baseline 1 − q_(1) has a higher mean than all five summaries on every tested condition 2602.01667 §5 app. E. Card: Beyond Prediction-Set Size.

When to use which

The paper reports 0.67 help for the conformal method against 0.81 for the Simple Set baseline at the same plan success of 0.87 on a mobile manipulator 2307.01928 §4.3 tab. 2. The baseline is cumulative thresholding on LLM confidence, not conformal, and its ε was tuned to match plan success 2307.01928 §4.2 2307.01928 §4.3. The comparison is therefore not at an equal target level.

For the failure detector, the paper's threshold-dependent tables use the constant band, and the authors say these tables are less reliable for comparing methods than their threshold-free tables 2606.21386 app. D. Cite the threshold-free results when comparing detectors. The paper claims no formal safety guarantee, and says its method is empirical 2606.21386 §6. Do not read the band as the 1−ε guarantee from the planner paper. The two papers use different calibration data and different score types, so their results are not directly comparable.

Implementation sketch

The core is split conformal calibration plus a singleton test. Notation follows the paper: N calibration scenarios, five option labels, and ε set by the user 2307.01928 §2 2307.01928 §3.1.

# Calibrate a threshold from labelled scenarios  [§3.1] [eq. 1]
# Input:  probs (N, 5) normalised option confidences
# Input:  labels (N,) index of the correct option
# Input:  eps, the user-chosen error level

def calibrate(probs, labels, eps):
    # Nonconformity: one minus confidence of the correct option  [§3.1]
    scores = 1 - probs[range(N), labels]
    # Quantile level ceil((N+1)(1-eps))/N, capped at 1  [eq. 1]
    level = min(1.0, ceil((N + 1) * (1 - eps)) / N)
    # Take the upper empirical quantile of the scores  [§3.1]
    return quantile(scores, level, method="higher")

def prediction_set(probs, qhat):
    # Keep each option whose confidence clears 1 - qhat  [§3.1]
    return [y for y in options if probs[y] >= 1 - qhat]

def act_or_ask(probs, qhat):
    # Singleton: act. Otherwise ask a human to choose  [§3.2]
    kept = prediction_set(probs, qhat)
    if len(kept) == 1:
        return act(kept[0])
    return ask_human(kept)

Where variants differ:

• Dataset-conditional. Replace the level in calibrate with the adjusted ε̂ from the Beta quantile in eq. 2 2307.01928 §3.1 eq. 2.
• Multi-step. Score each correct episode by the minimum step confidence eq. 4, set one threshold on episodes, and call act_or_ask at each step with the current context alg. 1 2307.01928 §3.3.
• Several acceptable options. Set the calibration label to the highest-ranked correct option 2307.01928 app. A.3.
• Constant band on a detector score. Calibrate one threshold per score on successful rollouts at significance 0.05, then alarm when score >= threshold for either score 2606.21386 §3.3 app. D. The band's exact formula is not on the card, so read the paper's appendix D before implementing it.
• Credal summaries. Take the same (K,) vector of split-CP p-values. Sort it in decreasing order and raise the largest to one. The summaries are then weighted sums over the sorted values 2602.01667 §3.1 §3.3 app. B. The entropy term needs an isotonic step (pool adjacent violators) on the masses. The paper describes that step in words, not as code 2602.01667 §3.3.
• Calibrated value level. Compute the errors s_i = V_pred − V_true on held-out states with ground truth. Choose β and ε, then take δ from the binomial rule in eq. 11 and the error quantile in §V-A.4. The paper gives the k-rule in eq. 11 and app. A, and this card gives only its outline, so read them before coding. Switch the filter when the calibrated value is at or below δ 2412.09989 §V-B.2 eq. 15.

Contradictions

• Dataset-conditional guarantee: paper vs released code. The paper states the eq. 2 guarantee with δ = 0.01 and an adjusted ε̂ 2307.01928 §3.1. The released notebooks use the requested ε directly, which gives only the marginal guarantee. The card's Paper vs code section records the search that found no Beta-quantile adjustment (card).
• Calibration size: paper vs released code. The paper uses N = 400 2307.01928 app. A.7. The card records 100 in the tabletop notebook and 200 in the mobile one (card).
• LLM: paper vs released code. The paper's main results use PaLM-2L 2307.01928 §4. The released notebooks call GPT-3.5, which the README discloses (card). Results from one model do not carry over to the other without a check.
• Set size as an uncertainty signal depends on the score. Under LAC, the credal summaries beat fixed-level set size on classifiers. Under APS, RAPS and SAPS, the raw baseline 1 − q_(1) beats the summaries 2602.01667 §5 app. E. The planner paper uses set size as its act-or-ask rule and does not compare summaries of the set 2307.01928 §3.2. These are different tasks and scores, so the results do not conflict. The choice of score still changes which signal ranks best.
• Value-function level: paper vs released config. The paper switches the filter at δ = 0.49 m on the uncalibrated output 2412.09989 §V-B.2 tab. II. The released hardware config adds a calibration offset and switches at a different level, which the card puts at about 0.85 m in raw network units (card). The paper tabulates ε = 10^-3, which the released calibration grid does not cover (card).

Papers

• Robots That Ask For Help: applies split conformal prediction to LLM next-step choices, so a robot acts on singleton sets and asks for help otherwise.
• Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry: turns split-CP p-values into credal uncertainty summaries that, with LAC scores, beat fixed-level set size on selective prediction (author-reported).
• VLA-FAIL: sets alarm thresholds for a failure detector on robot policies with a constant conformal band fitted on successful rollouts.
• One Filter to Deploy Them All: calibrates a learned reachability value network with a binomial-tail quantile rule and uses the resulting level to switch a quadruped safety filter (author-reported).