Chain of thought

Overview

Chain of thought (CoT) here means text that a model writes before its final output, which is then used as a training target or paid for at inference. The three cards on wikillm that use the term do this in different settings. In a robot policy, the written chain is a plan (the subtask, a target and a movement) that is trained to come before each motor command 2412.11974 §3.3. On a reasoning language model, the written chain is the bulk of the work, and it sets the latency of every answer 2511.01866 §4.1. A third card treats that chain as something to shorten: it scores each step of a finished trace and replaces the most predictable steps with [SKIP] markers [arxiv:2508.03346 §3.2].

This page does not cover few-shot prompting with exemplars, self-consistency voting over prompt chains or tree search. Those are on Chain-of-thought prompting. The invariant shared by the variants below is that the chain is a sequence of tokens, so its length sets the cost of each answer or action.

Variants

Grounded chain of thought as a training target

Emma-X fine-tunes a 7B OpenVLA policy so that, for each frame, it writes a reasoning text and a subtask, then a 2D gripper target, then a 3D movement plan, and only then the 7-dimensional action 2412.11974 §3.3 app. F. The labels are built from existing demonstrations. The trajectory is split into segments wherever the cluster label or the gripper state changes [arxiv:2412.11974 §3.1]. A vision-language model then labels each segment from its images and the task instruction, returning a subtask and a justification 2412.11974 §3.2. The authors use images rather than text alone because a text-only labeller, as in ECoT, can hallucinate what the scene shows 2412.11974 §1. The movement plan is the displacement to the first frame of the next segment, written in a fixed text template 2412.11974 §3.2 app. E. The model also predicts the gripper position at the start of the next segment before it picks the action for the current frame, which the authors call look-ahead 2412.11974 §3.2. Card: Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning.

The author-reported result on 12 real WidowX tasks, 10 trials each, is 57.5% success and 71.66% half-success. OpenVLA scores 33.33% and 45.41%, and ECoT 15.00% success 2412.11974 tab. 1 §4.2. The ablation separates the parts. On six prompts, removing the grounded reasoning drops spatial-relation success from 70% to 10%, and removing the movement plan drops it to 37% 2412.11974 tab. 2. Each prompt has only 10 rollouts and no confidence intervals are given 2412.11974 §4.5. The ablation also has no text-only labelling arm, so the benefit of image-based labels alone is not isolated 2412.11974 tab. 2.

The cost is output length. The model writes about ten times as many tokens per step as OpenVLA, which slows each control step 2412.11974 Limitations.

Length-capped reasoning on an edge GPU

EdgeReasoning measures a DeepSeek-R1 distilled family (1.5B, 8B and 14B) on a Jetson AGX Orin with vLLM, using 3,000 MMLU-Redux questions for accuracy 2511.01866 §4.1 §5. For these models decoding takes over 99% of latency 2511.01866 §4.1. The paper compares four ways to cap output 2511.01866 §5:

• Hard limit (nT): the prompt asks for a length and generation stops at n tokens.
• Soft limit (n-NC): the same instruction, with no enforcement.
• No reasoning (NR): a fixed short thinking block is injected so the model skips its own chain.
• L1-Max: a 1.5B model trained with reinforcement learning to respect a token budget 2511.01866 §2.3.

On the full set, the hard limit cuts accuracy sharply. The 8B model drops from 61.7% at 811 average tokens to 37.9% at a 128-token hard cap, with latency falling from 87.2 s to 7.9 s. The 14B model with a 256-token hard cap reaches 58.6% in 21.5 s, against 77.2% in 70.9 s with the soft limit and 80.6% in 259.0 s uncapped 2511.01866 app. B. A separate 150-question motivation table gives different latencies for the same models, so the two sample sizes should not be mixed 2511.01866 tab. II app. B.

Parallel samples with majority vote

The same paper runs several decoders on one prompt and takes the majority answer 2511.01866 §5.5. At a 128-token budget, accuracy rises by about 1.5 to 1.8 times when the scaling factor goes from 1 to 32. Decode latency rises about 2 times from factor 1 to 64. For the 14B model, energy per question grows by less than 1.5 times up to factor 4 and doubles at factor 16 2511.01866 §5.5 fig. 9 fig. 10. The authors' takeaway is that parallel scaling stays cheap only up to a factor of about 8 [arxiv:2511.01866 §5.5]. The paper does not state the sampling temperature or how ties are broken.

Card: EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs.

Pruning low-entropy steps from a trace

Making Slow Thinking Faster splits a finished reasoning trace into steps at blank lines, scores each step by the mean entropy of its tokens, and replaces the lowest-scoring 80% of steps with [SKIP] 2508.03346 §3.2 §4.1 eq. 3. The method rests on the assumption that the model's own per-token uncertainty is a usable sign of redundancy 2508.03346 §3.1. The score uses full-vocabulary entropy eq. 1, so a serving stack that returns only the probability of the sampled token cannot compute it as written.

In the static setting, the pruned trace is placed after the query and a closing </think> delimiter, and the model writes only the final answer [arxiv:2508.03346 §3.2]. On DeepSeek-R1- Distill-Qwen-7B, thinking tokens fall by 29.7% on MATH500 and 36.3% on AIME 2024, with MATH500 accuracy unchanged at 88.17% 2508.03346 tab. 1. The same cut costs AIME 2024 accuracy: 63.33% falls to 56.67%, and the 14B model drops about 7 points on AIME 2024 and AIME 2025 2508.03346 tab. 1. On the full DeepScaleR and OpenR1-Math sets, the 7B model loses 1.6 and 1.1 points 2508.03346 tab. 2. The paper's random-pruning and high-entropy-pruning controls show the direction matters: removing high-entropy steps hurts at once 2508.03346 §4.1 fig. 1.

The trained variant teaches the model to write [SKIP] itself. An SFT stage on pruned traces, then GRPO with a composite reward, cuts thinking tokens by 35% to 57% on four benchmarks 2508.03346 tab. 3. For the 7B model, the change in accuracy is +0.6 on GSM8K, −3.2 on MATH500, −6.2 on AIME 2024 and −2.4 on AIME 2025 2508.03346 tab. 3. The reward terms are not interchangeable. Correctness and skip ratio alone drop MATH500 accuracy to 51.00%; adding the skip-count penalty brings it back to 85.00% 2508.03346 tab. 5. In the paper's comparison with other compression methods, on AIME 2024 the trained method loses 6.2 points and cuts 57.0% of tokens, against 15.0 points and 34.6% for CoT-Valve. On MATH500, CoT-Valve cuts more tokens (48.4%) but loses more accuracy (10.6 points) [arxiv:2508.03346 tab. 4]. These figures are the authors' and were not re-run.

Card: Making Slow Thinking Faster.

When to use which

Implementation sketch

The training target for grounded chain of thought keeps the paper's order: the segment's reasoning and subtask, then the gripper target, then the movement plan, then the action tokens 2412.11974 §3.3.

# Target text for one frame, in the order the paper uses  [arxiv:2412.11974 §3.3]
def training_target(frame):
    # The subtask and reasoning come from the frame's segment  [§3.2]
    seg = segment_of(frame)
    text = seg.reasoning + seg.subtask
    # 2D gripper position at the start of the next segment  [§3.2]
    text += next_gripper_2d(frame)
    # Displacement to that point, written as template text  [§3.2] [app. E]
    text += movement_plan(frame)
    # The 7-D action, each dimension as one of 256 bins  [arxiv:2412.11974 §4.1]
    text += tokenize(frame.action)
    return text

For a reasoning model with a fixed latency target, the decode latency is a closed form in the input and output lengths, with coefficients fitted per model on the target device 2511.01866 §4.1 eq. 2. The budget can then be chosen before the request is sent.

# Decode time after an input of I tokens and an output of O tokens  [arxiv:2511.01866 §4.1] [eq. 2]
def decode_latency(I, O, m, n):
    # Per-token time n + m * context, summed over the output steps
    return n * O + m * (I * O + O * (O - 1) / 2)

# Largest output budget that fits the target, for a model fitted on the device  [§5.1]
def pick_budget(I, target_s, m, n, prefill_s, max_tokens):
    best = 0
    for O in range(1, max_tokens + 1):
        # Prefill is small next to decode  [arxiv:2511.01866 tab. VII]
        if prefill_s + decode_latency(I, O, m, n) <= target_s:
            best = O
    return best

The paper's own coefficients are for one Orin and are not portable. Its published sign for the 1.5B decode slope disagrees with the released code, so fit your own before relying on the numbers 2511.01866 Paper vs code.

For static step pruning, the score and the cut are both simple. The paper splits at blank lines and averages the entropy of each step's tokens 2508.03346 §3.2 eq. 3.

# Static step pruning of one finished trace  [arxiv:2508.03346 §3.2]
# Each token carries its full-vocabulary entropy in bits  [eq. 1]
def prune_trace(tokens, kappa=0.8):
    # Split at blank lines; each step keeps its tokens  [§3.2]
    steps = split_on_blank_lines(tokens)
    # Score each step by its mean token entropy  [eq. 3]
    scores = [mean([t.entropy for t in s]) for s in steps]
    # Replace the kappa * N lowest-scoring steps with [SKIP]  [§3.2] [§4.1]
    # The paper does not say how kappa * N is rounded  [§3.2]
    n_cut = int(kappa * len(steps))
    order = sorted(range(len(steps)), key=lambda i: scores[i])
    cut = set(order[:n_cut])
    # Keep the other steps verbatim  [§3.2]
    return ["[SKIP]" if i in cut else text_of(s) for i, s in enumerate(steps)]

Contradictions

None known between the three papers: they measure different things, a robot action and two kinds of text answer, and do not test the same claim. The two text-reasoning cards test different interventions, a hard token cap and entropy-based step removal, at budgets that do not match, so their accuracy figures cannot be compared directly.

Within the sources, some reported figures do not reconcile:

• Emma-X prints 28.75% as the ECoT half-success average, but the mean of its 12 rows is 29.17% 2412.11974 tab. 1. Its text gives a 29% gain over ECoT on spatial tasks, while Figure 4 and Table 1 imply about 43 points 2412.11974 §4.4 fig. 4. Use the table rows.
• EdgeReasoning gives 91.5 tokens for the 14B model at a 128-token hard cap in its text, but 78.2 in its appendix. The 91.5 figure belongs to the 1.5B row 2511.01866 §5.2 app. B.
• The Making Slow Thinking Faster paper labels its Table 4 as percentage changes, but the accuracy column matches absolute point differences from its Table 3 2508.03346 tab. 4 tab. 3. Read those accuracy values as points.

Papers

• Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning: grounded chain of thought as the target for a robot action model, with segment labels from images.
• Making Slow Thinking Faster: entropy-scored pruning of reasoning steps, applied to a finished trace and learned by SFT and GRPO.
• EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs: latency and energy of reasoning models on an edge GPU, with length caps, parallel sampling and quantization.