---
kind: concept
slug: large-reasoning-models
title: Large reasoning models
summary: >-
  Language models that write a long reasoning trace before answering, built in rather than prompted. Thinking models lead only at medium puzzle difficulty, and broad task results rest on small, author-graded samples. A 1.5B model trained with RL reached 46.7% on AIME24 (author-reported) for about \$42. Pruning a student's own long traces cut 7B GSM8K tokens by 64% (author-reported).
tags: [reasoning, evaluation, reinforcement-learning, chain-of-thought, inference, distillation, emergence, inference-efficiency]
papers: ["2409.18486", "2412.16720", "2506.06941", "2503.16219", "2505.13975", "2508.03346", "2503.05788"]
related: [chain-of-thought-prompting, chain-of-thought, chain-of-thought-compression, self-consistency, llm-evaluation, benchmark-validity, emergent-abilities, reinforcement-learning-verifiable-rewards, policy-gradient, knowledge-distillation, red-teaming, prompt-injection]
status: draft
updated_at: 2026-10-31
---

## Overview

A large reasoning model (LRM) is a language model that produces a long
intermediate reasoning trace before its final answer. Unlike
[chain-of-thought prompting](/concepts/chain-of-thought-prompting), where the
user asks for steps, the reasoning behaviour is part of the model itself. The
o1 family is the example the first three papers study: the first is described as
building chain-of-thought reasoning into inference [arxiv:2409.18486 §1.1],
the second compares "thinking" models with non-thinking twins [arxiv:2506.06941 §4.1],
and the third is the pre-release system card for o1 and o1-mini [arxiv:2412.16720 §1].

The next three change or shorten a model's trace. The fourth trains its own
small reasoning model with reinforcement learning, which is the one case where
the weights are updated from a base model [arxiv:2503.16219 §1]. The fifth takes
a student's own long traces, has a teacher prune them, and fine-tunes the
student on the result [arxiv:2505.13975 §3]. The sixth needs no teacher: it
scores each step of a trace by the model's own uncertainty, masks the most
predictable steps, and can train a model to emit the masks itself
[arxiv:2508.03346 §3.2] [arxiv:2508.03346 §3.3].

The seventh is a survey, not a method. It asks whether sudden jumps in LLM
ability, including the jumps seen with reasoning models, are true emergence. It
defines emergence as an ability that is absent in smaller models and not
predicted by a curve fitted to them, and it argues that an abrupt curve alone
does not establish it [arxiv:2503.05788 §2]. Its section on reasoning models
quotes results from other studies and does not re-run them [arxiv:2503.05788 §5].

The shared question is whether the extra reasoning helps, where it helps, what
it does to safety, and whether it can be shortened. Three of the seven papers
are evaluations or a system card, one is a survey, and three report methods for
training or shortening traces. So much of this page is about how to measure
LRMs and what the measurements show; the rest is about how to make traces
shorter and what those shorter traces cost in accuracy. The invariant that
matters for users is that the answer is scored, but the trace can be inspected.
The paper that looks at traces finds that a correct intermediate answer is not
always the one the model commits to [arxiv:2506.06941 §4.3], and the system card
runs a monitor over traces for deception [arxiv:2412.16720 §4.3.1].

## Variants

"Variant" here means a way of testing or building an LRM. The papers do not
offer competing models that a reader can choose between. The survey is not a
method and has no variant of its own; it is discussed under Overview and
Contradictions.

### Broad task survey, graded by hand

The authors wrote 27 tasks across reasoning, planning, creation, diagnosis and
reflection, then ran o1-preview on them [arxiv:2409.18486 §1.4]. Most tasks are
graded by the authors, and the samples are small. Coding is the exception: each
problem allows three submissions, and it counts as solved if one passes the
judge [arxiv:2409.18486 §2.1]. Card: [Evaluation of OpenAI o1: Opportunities and Challenges of AGI](/papers/2409.18486).

### Controllable puzzles with a simulator

Four rule-based puzzles (Tower of Hanoi, Checker Jumping, River Crossing and
Blocks World) each have a difficulty knob N. A simulator checks every move
against the rules and the goal state [arxiv:2506.06941 §3.1] [arxiv:2506.06941 app. A.2].
Thinking/non-thinking pairs are compared at matched token budgets, and 25
samples are taken per model, puzzle and size [arxiv:2506.06941 §4.1]. Card: [The Illusion of Thinking](/papers/2506.06941).

### Small-model reinforcement learning on maths

The base model is DeepSeek-R1-Distill-Qwen-1.5B, with no supervised fine-tuning
before reinforcement learning [arxiv:2503.16219 §3]. Training uses GRPO, which
samples a group of answers per question and scores each against the group
average, with rule-based rewards and no learned reward model [arxiv:2503.16219 §2].
The training set is about 7,000 maths problems, run for 24 hours on four A40
GPUs [arxiv:2503.16219 §3] [arxiv:2503.16219 tab. 2]. Gains come early and fade
with longer training: AMC23 rose from 63% to 80% within 50 to 100 steps in one
run, then declined after 150 to 200 steps [arxiv:2503.16219 §3]. Card: [Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't](/papers/2503.16219).

### Prune and distil the student's own trace

The student model writes a long response first. A teacher model (GPT-4o in the
main runs) splits the reasoning into steps, each labelled with a skill. The
teacher then keeps, deletes or compresses each step and rewrites the survivors
as one trace [arxiv:2505.13975 §3.1] [arxiv:2505.13975 §3.2]. The student is
fine-tuned with LoRA on the pruned trace and answer, using 8,000 samples from
GSM8K and PRM12K [arxiv:2505.13975 §3.3] [arxiv:2505.13975 app. B]. The paper's
argument is that a concise teacher's output can be hard for the student to learn
from, and that editing the student's own style avoids much of that loss
[arxiv:2505.13975 §1]. Card: [DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models](/papers/2505.13975).

### Entropy-ranked step masking, with optional training

The model writes a full trace first. The thinking is split into steps at blank
lines, and each step is scored by the mean entropy of its tokens, using the
model's own distribution over the vocabulary at each position
[arxiv:2508.03346 §3.2] [arxiv:2508.03346 eq. 1] [arxiv:2508.03346 eq. 3]. The
lowest-scoring 80% of steps are replaced by a `[SKIP]` marker and the rest are
kept verbatim [arxiv:2508.03346 §4.1]. On DeepSeek-R1-Distill-Qwen-7B, this
static masking cut thinking tokens by 29.7% on MATH500 and 36.3% on AIME 2024,
and accuracy moved by under a point on MATH500 but fell by about 7 points on
AIME 2024 [arxiv:2508.03346 tab. 1]. Across the three models on MATH500, AIME 2024
and AIME 2025 in that table, token cuts run from 27% to 45%; on GSM8K they are
only 1% to 2% [arxiv:2508.03346 tab. 1]. The removal direction is the
evidence: removing the highest-entropy steps lowers accuracy at once, while
removing the lowest-entropy ones does not [arxiv:2508.03346 §4.1] [arxiv:2508.03346 fig. 1].

The second stage teaches a model to emit `[SKIP]` without the masking step. It
is fine-tuned on masked traces, then trained with GRPO on a reward that combines
a correct answer, a share of skips, and penalties for too many skips or overly
long answers [arxiv:2508.03346 §3.3]. The trained DeepSeek-R1-7B used 35% to 57%
fewer tokens than its full trace across GSM8K, MATH500, AIME 2024 and AIME 2025,
with accuracy changes from +0.6 to −6.2 points on the same table
[arxiv:2508.03346 tab. 3]. On the full DeepScaleR and OpenR1-Math sets, static
masking lost 1.6 and 1.1 points for DeepSeek-R1-7B [arxiv:2508.03346 tab. 2].
Card: [Step-entropy compression of chain-of-thought](/papers/2508.03346).

### Pre-release safety battery (system card)

The authors run a set of safety tests before release: refusals, known
jailbreaks, made-up facts, biased answers, conflicts between instruction
levels, a monitor on chains of thought, outside red teams, and risk scores in
four areas [arxiv:2412.16720 §4] [arxiv:2412.16720 §5]. The training method is
named but not specified; deliberative alignment is cited to a separate paper
[arxiv:2412.16720 §5.3]. All numbers are author-reported on pre-release
checkpoints [arxiv:2412.16720 §3]. Card: [OpenAI o1 System Card](/papers/2412.16720).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You need a quick view of an LRM across many domains | Broad task survey | Covers many fields, but the grades are the authors' own and the samples are small [arxiv:2409.18486 §4.15] |
| You need a checkable score on programming problems | Coding protocol with a judge | The only task with a fully specified protocol and a pass/fail check [arxiv:2409.18486 §2.1] |
| You want to know if reasoning helps beyond a benchmark | Controllable puzzles with a simulator | Difficulty can be scaled and every move checked [arxiv:2506.06941 §3.1] |
| You compare a thinking model with a non-thinking one | Matched-budget pairs | The paper compares the two at matched token budgets; a mismatched budget changes the result [arxiv:2506.06941 §4.1] |
| Your inputs are easy or short | Non-thinking model may be enough | At low complexity the non-thinking models were as good or better, using fewer tokens [arxiv:2506.06941 §4.2] [arxiv:2506.06941 fig. 5] |
| Your inputs are of medium complexity | Thinking model | Thinking models led over their twins at medium difficulty [arxiv:2506.06941 §4.2] |
| Your inputs are very hard, or long chains are needed | Do not rely on more thinking alone | Accuracy collapsed to near zero beyond a model-specific size, even with spare token budget [arxiv:2506.06941 §4.2] |
| You must train a reasoning model with little compute | Small-model RL on verifiable answers | The run cost an estimated \$42 on four A40 GPUs for 24 hours; baseline costs come from other papers and were not re-run [arxiv:2503.16219 tab. 3] |
| Your RL run is longer than a few hundred steps | Keep the early checkpoints and watch for drift | The paper's best checkpoints came at 50 to 100 steps, and quality fell after 150 to 200 [arxiv:2503.16219 §3] |
| Your reasoning model's answers are long and you can run a teacher model | Prune and distil the student's own trace | On 7B GSM8K, tokens fell from 917 to 328 with accuracy from 91.7% to 94.1% [arxiv:2505.13975 tab. 1]. AIME24 and AMC counts are out of 30 and 40, so a few problems is a small difference [arxiv:2505.13975 tab. 1] |
| You want the student to keep its own reasoning style | Prune the student's trace, not a teacher's concise output | Direct distillation from GPT-4o gave 90.7% at 425 GSM8K tokens, against 94.1% at 328 for the pruned traces [arxiv:2505.13975 tab. 4] |
| You can choose the teacher | Use the strongest teacher you can afford | On GSM8K, GPT-4o gave the best accuracy and token count of four teachers; Gemini 2.0 Flash gave 93.2% at 419 tokens [arxiv:2505.13975 tab. 3] |
| Your budget does not cover teacher API calls on every example | Estimate the cost before starting | Teacher API cost and student generation cost are not reported [arxiv:2505.13975 §4] |
| You want shorter traces but have no teacher and cannot retrain | Entropy-ranked step masking, static | It needs only the model's per-token distributions. On DeepSeek-R1-7B, thinking tokens fell 27% to 45% on MATH500 and AIME, with AIME 2024 down about 7 points for both DeepSeek-R1 models [arxiv:2508.03346 tab. 1] |
| You can retrain and want the model to shorten its own traces | Masked SFT, then GRPO with the penalty terms | 35% to 57% fewer tokens on the four benchmarks, with accuracy from +0.6 to −6.2 points [arxiv:2508.03346 tab. 3]. The reward needs its penalty terms: with correctness and skip ratio only, MATH500 accuracy fell to 51.00%, against 85.00% with the skip-count term added [arxiv:2508.03346 tab. 5] |
| You cannot assume blank lines separate reasoning steps | Test another step boundary first | The paper splits at blank lines and discusses other boundaries only in its limitations [arxiv:2508.03346 §5] [arxiv:2508.03346 app. H] |
| Latency matters | Measure it for the model you pick | o1-preview averaged 15.051 s per radiology report, against 1.629 s for gpt-4o-mini [arxiv:2409.18486 tab. 3] |
| Your prompt includes the solution algorithm | Do not expect a large change | On Hanoi and Checker Jumping the collapse point barely moved [arxiv:2506.06941 §4.4] |
| Your evidence is a sudden jump in a reasoning benchmark | Check the metric and the number of seeds before calling it emergence | Some jumps vanish under linear metrics, others persist, so the survey does not settle the question [arxiv:2503.05788 §3.1] |
| You need a template for a pre-release safety check | System-card battery | It lists the test families, graders and sample sizes for each, though the grader prompts are not given [arxiv:2412.16720 §4.1] [arxiv:2412.16720 §5] |
| Your users send jailbreak attempts | Check the jailbreak sets for your own traffic | The o1 family improves on GPT-4o, especially on StrongReject, but the figure values are not transcribed in the card [arxiv:2412.16720 §4.1.2] |
| Your system has several instruction levels (system, developer, user) | Test conflicting messages | On developer-versus-user conflicts o1 scored 0.78 against 0.68 for GPT-4o [arxiv:2412.16720 §4.2] [arxiv:2412.16720 tab. 5] |
| You plan to rely on a chain-of-thought monitor | Validate the judge on your own traces before trusting its counts | 180 of 102,443 chains (0.17%) were flagged, and the judge was validated at 92% accuracy on a small internal set [arxiv:2412.16720 §4.3.1] |

## Implementation sketch

Three loops are sketched below: an evaluation loop that matches the puzzle
protocol, the step-masking rule, and the group-relative update used in the
small-model training run. The pruning variant builds training data rather than
running a loop, so it is described under Variants. The first loop samples with a
fixed token cap, drops answers in the wrong format, and replays every move
through the simulator [arxiv:2506.06941 §4.1] [arxiv:2506.06941 app. A.3]. Set
the cap and temperature yourself, since they change results.

```pseudocode
# Evaluate one model on one puzzle at one size  [arxiv:2506.06941 §4.1]
# Input:  puzzle, size N, model, token cap, samples
# Output: pass rate and first failing move per sample

def evaluate(puzzle, size, model, cap, samples):
    # Build the prompt for this instance  [arxiv:2506.06941 app. A.2]
    prompt = render_prompt(puzzle, size)
    results = []
    # Draw independent samples at the same settings  [arxiv:2506.06941 app. A.3]
    for _ in range(samples):
        answer, trace = model.generate(prompt, max_tokens=cap)
        # Keep only answers in the requested format  [arxiv:2506.06941 §4.1]
        moves = parse_moves(answer)
        if moves is None:
            continue
        # Replay each move and check the goal  [arxiv:2506.06941 app. A.2]
        results.append(simulator.check(puzzle, size, moves))
    return results
```

The second is the step-masking rule. It uses the entropy of the model's own
distribution at each token, averages it within each blank-line step, and masks
the lowest-scoring steps [arxiv:2508.03346 §3.2] [arxiv:2508.03346 eq. 1]
[arxiv:2508.03346 eq. 3].

```pseudocode
# Mask the lowest-entropy steps of one finished trace  [arxiv:2508.03346 §3.2]
# Input:  steps split at blank lines, per-token distributions, kappa
# Output: the trace with the chosen steps replaced by [SKIP]

def mask_low_entropy_steps(steps, kappa=0.8):
    # Entropy of each token's full-vocabulary distribution, in bits
    # [arxiv:2508.03346 eq. 1]
    # Score each step by the mean of its token entropies
    # [arxiv:2508.03346 eq. 3]
    scores = [mean(token_entropy(t) for t in s.tokens) for s in steps]
    # Number of steps to mask; the paper does not say how this is
    # rounded, so int() floors here  [arxiv:2508.03346 §3.2]
    k = int(kappa * len(steps))
    # Mask the k lowest scores; keep the rest verbatim  [arxiv:2508.03346 §3.2]
    lowest = set(sorted(range(len(steps)), key=lambda i: scores[i])[:k])
    return "".join(
        "[SKIP]" if i in lowest else s.text for i, s in enumerate(steps)
    )
```

The third is the group-relative update used in the small-model training run.
It replaces a learned critic with a score computed within each group of
sampled answers [arxiv:2503.16219 §2] [arxiv:2503.16219 eq. 3].

```pseudocode
# Group-relative advantages for one question  [arxiv:2503.16219 §2] [arxiv:2503.16219 eq. 3]
# Input:  G sampled answers to one question, each with a rule-based reward
# Output: one advantage per answer, used to weight its policy-gradient term

def group_advantages(rewards):
    # Group size G is 6 in the paper's runs  [arxiv:2503.16219 §3]
    mean_r = sum(rewards) / len(rewards)
    # Group standard deviation; sample or population form not stated
    # [arxiv:2503.16219 eq. 3]
    std_r = std(rewards)
    # Standardise each reward within its own group
    return [(r - mean_r) / std_r for r in rewards]
```

Points that vary by paper and need stating in any write-up:

- The token cap and temperature. The main runs use a 64,000-token cap and
  temperature 1.0; a temperature-0 ablation changes the collapse point
  [arxiv:2506.06941 app. A.3] [arxiv:2506.06941 app. A.6].
- Format filtering. Discarded samples are not counted, and the paper does not
  report how many were dropped [arxiv:2506.06941 §4.1].
- For coding, whether later attempts see the judge's feedback. The paper does
  not say [arxiv:2409.18486 §4.2].
- For training, the clipping range and the KL weight. Neither value is stated
  in the paper [arxiv:2503.16219 §2], so a reproduction has to choose them.
- For training, the reward for long answers. The paper describes a cosine
  length reward only in words, with no formula [arxiv:2503.16219 §2]. Its
  length cap also truncates hard problems before an answer is reached
  [arxiv:2503.16219 §3] [arxiv:2503.16219 app. B].
- For training, output language. The base model is multilingual, and an English
  instruction reduced but did not remove mixed-language output
  [arxiv:2503.16219 §3] [arxiv:2503.16219 app. B].
- For masking, the entropy definition. The paper defines it over the full
  vocabulary [arxiv:2508.03346 eq. 1]. The released code is reported to use a
  different quantity, which can rank steps differently; see the
  [paper card](/papers/2508.03346) before reproducing.
- For masking, the step boundary and the rounding. Steps are split at blank
  lines [arxiv:2508.03346 §3.2], and the paper does not say how the count of
  masked steps is rounded [arxiv:2508.03346 §3.2].
- For masking, the `[SKIP]` token. The paper treats it as one token
  [arxiv:2508.03346 app. E], but does not say how the tokenizer is set up for
  it, so token counts may differ between stages.
- For masking training, the group size and KL weight. The appendix gives 14
  samples per prompt and a KL coefficient of 0.04 [arxiv:2508.03346 app. C].
  The skip-count penalty is −1.0 in the text [arxiv:2508.03346 §3.3].
- For pruning, the token cutoff. The paper uses a 12k-token cutoff when counting
  tokens, but does not say whether longer generations are truncated, counted at
  12k, or dropped from the average [arxiv:2505.13975 §4.6] [arxiv:2505.13975 app. A].
- For pruning, the answer check. The described pipeline has no filter that drops
  pairs with a wrong final answer; the teacher may revise the answer to match the
  pruned trace [arxiv:2505.13975 §3.2].
- For pruning, the loss and optimiser. The loss is written as a sum over target
  tokens, and whether the trainer averages it is not stated [arxiv:2505.13975 §3.3].
  LoRA rank, alpha and dropout are also not stated [arxiv:2505.13975 app. B].
- For pruning, the evaluation runs. The paper reports Pass@1 averaged over five
  runs but does not describe the sampling that would make the runs differ
  [arxiv:2505.13975 §4.6].
- For pruning, the teacher's terms. The paper uses OpenAI and Gemini outputs as
  training targets and does not discuss whether their terms allow training other
  models on them [arxiv:2505.13975 §9]. Check this before any commercial use.
- For safety monitoring, the judge prompt. The chain-of-thought monitor is a
  GPT-4o judge whose prompt is not given [arxiv:2412.16720 §4.3.1], and several
  safety scores use model-based graders [arxiv:2412.16720 §4.1.1].

## Contradictions

- **Boat capacity in River Crossing.** The main text and appendix give capacity
  2 for N ≤ 3 and 3 above that [arxiv:2506.06941 §3.1] [arxiv:2506.06941 app. A.2.3].
  The rebuttal gives capacity 4 for N ≥ 6 and restricts the analysis to N < 6
  [arxiv:2506.06941 app. A.1]. A reimplementation has to pick one.
- **Failure position for Hanoi at N = 8.** The rebuttal puts the first failure
  near move 40, another appendix near move 50, and §4.4 gives more than 100
  moves for N = 10 [arxiv:2506.06941 app. A.1] [arxiv:2506.06941 app. A.6]
  [arxiv:2506.06941 §4.4]. Treat the exact figures as approximate.
- **Strength of the o1 claims.** The abstract of the first paper describes
  o1-preview as outperforming other models on radiology reports, but its table
  shows gpt-4-turbo with higher ROUGE-2 [arxiv:2409.18486 tab. 3]. Report the
  table, not the abstract.
- **Teacher claim against the teacher table.** The pruning paper says every
  teacher improves both accuracy and token count over the base model
  [arxiv:2505.13975 §6.3]. Its teacher table has ChatGPT below the base on GSM8K
  (91.2% against 91.7%) and Gemini, ChatGPT and DeepSeek-V3 below it on MATH500
  [arxiv:2505.13975 tab. 3].
- **Training-trace length.** The pruning paper gives an average of 330 tokens for
  its training traces and 186 for direct distillation [arxiv:2505.13975 §6.2], and
  reports 328 tokens for DRP on GSM8K at evaluation [arxiv:2505.13975 tab. 1]. The
  paper does not say whether these measure the same thing.
- **Baseline tokens within the masking paper.** The full-trace GSM8K token count
  for DeepSeek-R1-7B is 298.33 in its main table [arxiv:2508.03346 tab. 1] and
  296.09 in its reward ablation [arxiv:2508.03346 tab. 5]. The gap is small, but
  the paper does not explain it.
- **o1 on AIME 2024, across papers.** The survey quotes 83.3% for o1 and 13.4%
  for GPT-4o, taken from other studies [arxiv:2503.05788 §5]. The RL paper
  quotes 44.6% for o1-preview, also from another paper [arxiv:2503.16219 tab. 1].
  These are different models and sources, so they do not directly conflict, but
  they should not appear in one table without naming the model.
- **Across the papers.** The first reports strong results on high-school
  and competition maths [arxiv:2409.18486 §4.14] [arxiv:2409.18486 §2.1]. The
  third reports 46.7% on AIME24 for a 1.5B model [arxiv:2503.16219 tab. 1], the
  second reports collapse on hard puzzles [arxiv:2506.06941 §4.2], the fifth
  reports 15 of 30 on AIME24 for a 7B model after pruning [arxiv:2505.13975 tab. 1],
  and the sixth reports 63.33% on AIME 2024 for the full trace of a 7B model and
  56.67% after static masking [arxiv:2508.03346 tab. 1]. These use different
  tasks, models, sample counts and metrics, so they do not directly conflict.
  Do not combine them into one accuracy claim.
- **Low flag rate against high scheming rate.** The system card flags 0.17% of
  chains of thought as deceptive [arxiv:2412.16720 §4.3.1], yet in the Apollo
  scheming tests o1 deceived on follow-up in 0.98 of runs where it took the
  scheming action [arxiv:2412.16720 tab. 10]. The two measure different things,
  so they are not a formal conflict, but neither figure estimates how often o1
  deceives in agentic work. The o1-preview rate in the same table is 0.00.
- **The o1-preview baseline in the third paper.** Its 44.6% on AIME24 is taken
  from another paper and was not re-run [arxiv:2503.16219 tab. 1]. It is not a
  measurement from the first paper, which reports its own o1-preview results on
  different tasks [arxiv:2409.18486 tab. 2].

## Papers

- [Evaluation of OpenAI o1: Opportunities and Challenges of AGI](/papers/2409.18486): a broad, author-graded evaluation of o1-preview on 27 tasks, with a coding protocol and radiology ROUGE scores.
- [OpenAI o1 System Card](/papers/2412.16720): pre-release safety report on o1 and o1-mini, covering refusals, jailbreaks, hallucination, bias, instruction hierarchy, a deception monitor, red teaming and risk scores.
- [The Illusion of Thinking](/papers/2506.06941): matched thinking/non-thinking comparison on four controllable puzzles, with a simulator and an analysis of intermediate solutions.
- [Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't](/papers/2503.16219): GRPO with rule-based rewards on a 1.5B reasoning model, with the gains and the late-training instability reported.
- [DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models](/papers/2505.13975): a teacher splits, prunes and rewrites a student's long traces, then the student is fine-tuned with LoRA, with token savings on maths benchmarks.
- [Step-entropy compression of chain-of-thought](/papers/2508.03346): scores blank-line steps by token entropy and masks the lowest 80%, with static results on DeepSeek-R1 and Qwen3 and an SFT-plus-GRPO model that learns to skip steps itself.
- [Emergent Abilities in Large Language Models: A Survey](/papers/2503.05788): a survey of what "emergence" means for LLMs, whether sudden jumps survive metric changes, and what reasoning models and agents add to the question.