---
kind: concept
slug: adaptive-halting
title: Adaptive halting
summary: >-
  Deciding, per input, how much computation to run before stopping. Three sources: stop a looped model when its output settles (KL and entropy thresholds, no retraining), train a halting head on answer correctness, or skip remaining layers once dominant sub-updates agree. All are author-reported on small or synthetic tasks. None compares them with fixed depth on a common benchmark.
tags: [recurrence, inference, test-time-compute, latency]
papers: ["2203.14680", "2604.07822", "2605.19376"]
related: [multi-hop-reasoning, variational-autoencoder, activation-steering, mechanistic-interpretability]
status: draft
updated_at: 2026-10-19
---

## Overview

A recurrent model runs the same block several times over its hidden state.
Adaptive halting asks when to stop. Instead of a fixed number of passes for
every input, the model stops once it judges that more passes will not help.
The same question appears in a feed-forward transformer, where the unit is a
layer rather than a pass: the model can skip its remaining layers once the
answer is already settled [arxiv:2203.14680 §6.2].

Two reasons drive the idea. Some inputs need more passes than others, and
the number needed is often unknown in advance. The second reason is
overthinking: more passes can make the answer worse. One source reports that
for its dynamic model, OOD accuracy stopped improving beyond about 15
iterations, and that the margin peaked and then declined as iterations grew
[arxiv:2604.07822 §6.2]. That source also says adaptive halting does not fix
this effect [arxiv:2604.07822 §6.2].

The sources differ in what they use as the stop signal. One uses the model's
output distribution alone, so it needs no extra training. Another trains a
dedicated head to predict whether the current answer is correct
[arxiv:2605.19376 app. A.3]. The third reads which large sub-updates of each
feed-forward layer already promote the final token, and compares them with
clusters seen at earlier saturation events [arxiv:2203.14680 §6.2].

## Variants

### Output settling (KL and entropy)

Stop when the prediction stops moving and is confident. At each pass after the
first, compare the new output distribution with the previous one by KL
divergence, and measure the entropy of the new distribution. Halt when both
fall below their thresholds [arxiv:2604.07822 §6.2]. The paper sets the KL
threshold to 0.01 and the entropy threshold to 3.00 [arxiv:2604.07822 app. C].
The paper's rule is described as an extension of an earlier KL-based halting
method, to which the entropy term is added [arxiv:2604.07822 §6.2]. Card:
[Loop, Think, & Generalize](/papers/2604.07822).

Two details matter for reproduction. The rule never halts on the first pass,
so at least two passes always run [arxiv:2604.07822 §6.2]. The released code
uses a different KL default in its adaptive evaluation than the paper states,
so the effective threshold depends on which code path is used
[arxiv:2604.07822 app. C].

### Learned halting head

Train an auxiliary head on the recurrent state to output the probability that
the current answer is correct. At inference, stop when this probability passes
one half [arxiv:2605.19376 app. A.3]. Training uses a binary cross-entropy
loss on the head's output, with the label set by whether the current answer is
correct [arxiv:2605.19376 alg. 1]. The paper says the head follows a
Q-style target [arxiv:2605.19376 app. A.3]. Card:
[Generative Recursive Reasoning](/papers/2605.19376).

The paper marks halting as optional in its method [arxiv:2605.19376 app. A.3].
The card reports no run with halting switched off, so the cost and accuracy
effect of the head alone is not known from the card.

### Dominant-cluster early exit (layer skipping)

This applies to a standard transformer with no recurrence. For each layer, the
10 largest sub-updates of its feed-forward output are mapped to clusters of
value vectors. Clusters seen at saturation events in that layer are stored.
At inference, the model exits at a layer when the clusters of its current
dominant sub-updates overlap the saturation sets of that layer more than the
overlap with the non-saturation sets of every later layer, on both the mean and
the maximum [arxiv:2203.14680 §6.2] [arxiv:2203.14680 app. A.5]. Card:
[Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](/papers/2203.14680).

The clusters are precomputed. The paper uses k = 2e2 clusters for this
rule [arxiv:2203.14680 app. A.5]. The paper's text describes the rule more
loosely than its notebook, which requires both mean and maximum overlaps to
win [arxiv:2203.14680 §6.2] [code: early_exit/wikilm-earlyexit-ours.ipynb].

## When to use which

| Situation | Use | Why |
|---|---|---|
| A trained recurrent model, no retraining allowed | Output settling | Needs only the per-pass output distribution. The paper's thresholds were set for a synthetic knowledge-graph task, so re-tune them [arxiv:2604.07822 app. C] |
| You can retrain and want the stop decision learned | Learned halting head | The head is trained on correctness labels [arxiv:2605.19376 alg. 1]. Costs a training run that is sequential across supervision steps [arxiv:2605.19376 §5] |
| Several valid answers per input | Learned head, with care | The paper defines the label as matching the stored answer [arxiv:2605.19376 app. A.3]. The released code's label instead accepts any valid solution on the puzzle tasks (see the card's *Paper vs code*). Which label you use changes what the head learns |
| Standard feed-forward transformer, you want to skip layers at inference | Dominant-cluster early exit | Reported 94.1% accuracy and 20.0% of layers saved on WikiLM, on par with per-layer classifiers [arxiv:2203.14680 tab. 6]. Needs precomputed value-vector clusters. The code for building them is not in the released repository [code: clustering/README.md] |
| Chains deeper than training | Fixed passes or the dynamic model, not adaptive halting alone | Fixed higher recurrence gives reported extrapolation (R = 8 reaches 24 hops in one setup) [arxiv:2604.07822 §6.2]. The paper says the dynamic model goes further but gives no number in the text, and the card reports no adaptive-halting number for depth |
| Latency is the constraint | Measure it yourself | No source reports the wall-clock effect of halting. The latency numbers in the learned-head source come from parallel sampling, not from halting [arxiv:2605.19376 tab. 10]. The layer-skipping source reports saved layers, not time [arxiv:2203.14680 tab. 6] |

## Implementation sketch

The output-settling rule is the simplest to add to an existing looped model,
so the sketch follows it. The learned head adds one training loss and a
threshold of one half at inference. The layer-skipping rule needs the cluster
sets and per-layer saturation sets from its card, so it is not sketched here.

```pseudocode
# Output-settling adaptive halting  [arxiv:2604.07822 §6.2]
# Stop once successive output distributions agree and are confident

def adaptive_predict(prefix, max_recurrence, eps_kl, entropy_thresh):
    hidden = embed(prefix)
    previous_logp = None
    for t in range(max_recurrence):
        # One more pass through the shared layers  [arxiv:2604.07822 §4]
        hidden = apply_shared_layers(hidden)
        logp = log_softmax(readout(hidden))  # answer position only
        if t > 0:
            # How far this pass moved the prediction  [arxiv:2604.07822 §6.2]
            kl = sum(exp(logp) * (logp - previous_logp))
            # How unsure the current prediction is  [arxiv:2604.07822 §6.2]
            entropy = -sum(exp(logp) * logp)
            if kl < eps_kl and entropy < entropy_thresh:
                return logp, t + 1
        previous_logp = logp
    # No halt before the cap: use the last pass  [arxiv:2604.07822 app. C]
    return logp, max_recurrence
```

## Contradictions

No direct conflict between the three sources. They measure different things:
output settling, learned correctness and sub-update clusters. No card compares
any two of them on the same task.

Within one source, the paper and its released code disagree. The paper sets
the KL threshold to 0.01 [arxiv:2604.07822 app. C], while the adaptive
evaluation uses a wrapper default of 0.001 (see the card's *Paper vs code*
section). The same card notes that the paper gives the halting entropy as a
fixed threshold, while the code also contains a variant that scales it with
sequence length [arxiv:2604.07822 app. C].

For the learned-head source, the paper states that the halting label is whether
the prediction equals the target [arxiv:2605.19376 app. A.3]. The released code
accepts any valid solution on the puzzle tasks instead (see the card's
*Paper vs code*). The two readings give different training signals on
multi-solution tasks.

For the layer-skipping source, the paper reports its rule as "on-par" with
per-layer logistic-regression classifiers [arxiv:2203.14680 §6.2]. The
classifiers train on 8k examples and the rule on 10k, so the two are not
compared on identical data [arxiv:2203.14680 app. A.5]. The table's rows for
two of the classifier inputs show identical numbers, which the text does not
explain [arxiv:2203.14680 tab. 6].

## Papers

- [Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers](/papers/2604.07822): output-settling halting with KL and entropy thresholds, with overthinking observed in a dynamic-recurrence model.
- [Generative Recursive Reasoning](/papers/2605.19376): a learned halting head trained on correctness, inside a latent-variable recursive reasoner.
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](/papers/2203.14680): a layer-skipping early exit from dominant feed-forward sub-update clusters, reported on par with per-layer classifiers.