Chain-of-thought compression

Overview

Chain-of-thought compression shortens the text a reasoning model writes before its answer, so that fewer tokens go to steps that add little. The method on this page splits the thinking into blank-line-separated steps, scores each step by the mean entropy of its tokens, and marks the lowest-scoring 80% as [SKIP] 2508.03346 §3.2 §4.1. The rest of the trace stays verbatim.

Both variants share one invariant: the model's own per-token uncertainty ranks the steps, and the lowest-ranked steps are the ones that get removed. They differ in when the removal happens. Static pruning applies it to traces that already exist. The learned variant trains the model to write the [SKIP] markers itself 2508.03346 §3.3.

Only one paper on wikillm covers this method so far. Other ways to shorten reasoning, such as length caps and parallel sampling, are on Chain of thought. They do not use step entropy.

Variants

Static step pruning on existing traces

Generate a full trace, score its steps, replace the lowest 80% with [SKIP], and place the result before the answer. The model then writes only the final answer 2508.03346 §3.2. The value κ = 0.8 comes from a sweep of 10% to 100% on 50 DeepScaleR problems with DeepSeek-R1-7B. The paper does not say how the sweep was stepped §4.1. Card: Paper card for arXiv:2508.03346.

On DeepSeek-R1-Distill-Qwen 7B and 14B and on Qwen3-8B, thinking tokens fall by 27% to 45% on MATH500 and AIME 2508.03346 tab. 1. Accuracy changes vary by benchmark. On GSM8K and MATH500 the changes are within about 2 points either way. On AIME 2024 the two DeepSeek models lose 6.7 and 6.9 points, and Qwen3-8B gains 2.2 points 2508.03346 tab. 1. On the full DeepScaleR and OpenR1-Math sets, the 7B model loses 1.6 and 1.1 points 2508.03346 tab. 2. The paper does not state how many samples it draws per problem, and set sizes are not stated for every benchmark, so small gaps may be within noise 2508.03346 §4.2.

The paper also reports where the signal comes from. Removing the highest-entropy steps hurts accuracy at once, and random removal falls between the two rules 2508.03346 fig. 1. Masking individual tokens instead of whole steps loses accuracy after a 20% mask ratio, while step pruning holds to 40% 2508.03346 fig. 3. At 90% pruning on the 7B model, the reported accuracy is 68% for plain deletion or [MASK] and 70% for [SKIP] or "and then" 2508.03346 app. E fig. 4. The choice of marker therefore moves accuracy by a few points at this ratio.

Learned skipping with SFT and GRPO

The model is taught to emit [SKIP] itself. Stage 1 is supervised fine-tuning on (problem, compressed trace, answer) triples, built from 130k problems and filtered to 70k samples of at most 4096 tokens 2508.03346 §4.2 app. C. Stage 2 is GRPO on 10k of those samples. For each sampled completion the reward adds four parts 2508.03346 §3.3 eq. 4:

• correctness: 2.0 for the right final answer eq. 5;
• skip ratio: 1.0 if at least 80% of steps are skipped, 0.5 if between 50% and 80% eq. 6 §4.3;
• skip count: −1.0 above 100 [SKIP] tokens §3.3;
• length: −1.0 above 3500 response tokens §3.3 §4.3.

For the 7B model, the paper's SFT plus GRPO row uses 35% to 57% fewer tokens than the full trace on four benchmarks 2508.03346 tab. 3. Accuracy changes from the full trace are GSM8K +0.6 points, MATH500 −3.2, AIME 2024 −6.2 and AIME 2025 −2.4 2508.03346 tab. 3. That table is the only place the learned variant is reported, and it covers one model.

The reward terms matter. With only correctness and the skip ratio, MATH500 accuracy falls to 51.00 and the completions get longer. Adding the skip count restores 85.00 and the length term reduces tokens further [arxiv:2508.03346 tab. 5].

When to use which

Implementation sketch

The scoring and pruning core is the same for both variants. The learned variant adds training around it 2508.03346 §3.2 §3.3.

# Steps are the blank-line blocks of the thinking  [arxiv:2508.03346 §3.2]
# Each token has an entropy over the full vocabulary  [arxiv:2508.03346 eq. 1]
def step_score(step):
    # Mean token entropy, so long steps are not favoured  [eq. 3]
    return sum(step.token_entropies) / len(step.token_entropies)

def prune_steps(steps, kappa=0.8):
    # Rank the steps from lowest to highest score  [§3.2]
    scores = [step_score(s) for s in steps]
    order = sorted(range(len(steps)), key=lambda i: scores[i])
    # Number to prune is kappa times N; rounding is not stated  [§3.2]
    pruned = set(order[: int(kappa * len(steps))])
    # Replace pruned steps with the marker, keep the rest verbatim  [§3.2]
    return "".join(
        "[SKIP]" if i in pruned else steps[i].text
        for i in range(len(steps))
    )

Where the variants differ: the static variant calls this function on each trace. The learned variant calls it on the SFT data and then rewards the model for skip counts and ratios that match the rules above 2508.03346 §3.3.

Contradictions

No other wikillm card reports this method, so there is no cross-paper disagreement. Within the paper, and between the paper and its released code, these points differ. The paper's values are given first.

• Entropy definition. The paper uses the full-vocabulary entropy 2508.03346 eq. 1. The released code scores only the sampled token's term. See the paper card, section Paper vs code.
• Skip-count penalty. The paper gives −1.0 2508.03346 §3.3. The released code uses −2.0 (see the paper card).
• GRPO group size and SFT epochs. The paper gives 14 samples per prompt and 3 SFT epochs 2508.03346 app. C. The released code uses 8 samples and 1 epoch (see the paper card).
• Baseline tokens. The 7B model's full-trace GSM8K token count is 298.33 in 2508.03346 tab. 1 and 296.09 in tab. 5.
• MATH500 token reduction. The paper states 35% for SFT plus GRPO 2508.03346 tab. 3 §4.3. Its table values give 34.1%, which the appendix reports app. F.

Papers

• Paper card for arXiv:2508.03346: step-entropy pruning of reasoning traces, applied statically and learned with SFT and GRPO, on DeepSeek-R1 distills and Qwen3 models.