Multi-Step Reasoning with Large Language Models, a Survey

In plain words

A language model answers a simple question well when the answer is one step away. It often fails when the answer needs several small steps, such as a multi-step word problem. Asking the model to write out its steps before the answer helped a lot, and many follow-up methods came from that idea.

This paper reviews those methods. It sorts them by three jobs. The first is writing the steps. The second is checking them. The third is deciding how far to explore, for example by trying one path or several. The authors also list which benchmarks were used and which approach suits which kind of problem. The paper gives no new method and no code of its own. Its numbers come from the original studies, which used different models and setups.

TL;DR

• Survey of prompt-based multi-step reasoning in LLMs, centred on chain-of-thought prompting §1 §2.1.
• Taxonomy: step generation, step evaluation, step control §3. Table 2 places 31 approaches in these columns tab. 2.
• No method, no experiments, no code. Benchmark numbers are the cited papers' own, and setups differ §2.4 tab. 2.

Problem

Language models are trained to predict the next token. That works for many language tasks, but multi-step problems such as grade-school math word problems (GSM8K, 8,500 problems) were hard for them before chain-of-thought §1 §2.4. The survey asks how prompting alone, without training, can make models solve such problems step by step, and then where that approach works and where it fails §1 §4.1.

Method

This is a survey. It proposes a taxonomy, not an algorithm. The taxonomy assumes that every in-context reasoning approach has three stages: generate steps, evaluate them, and control how many steps are made and how far ahead the process looks §3. Figure 1 draws the categories fig. 1. Each approach appears in one main column, and some appear in two tab. 2 (footnote in §3).

Generation §3.1:

• Hand-written prompt: the researchers write step-by-step exemplars. Chain-of-thought uses few-shot examples that show the steps §3.1.1 fig. 3. Zero-shot chain-of-thought adds "Let's think step by step" without exemplars §3.1.1 fig. 3.
• Scratchpad is the earlier supervised version: the model is trained to emit intermediate computation steps §3.1 fig. 2.
• External knowledge: Self-ask lets the model ask follow-up questions, which can be answered by a search engine §3.1.2.
• Model-generated prompt: Auto-CoT has the model generate questions, cluster them and sample demonstrations §3.1.3. Complexity-based prompting favours demonstrations with more steps §3.1.3.

Evaluation §3.2:

• Self-assessment: the same model checks its steps. Self-verification rebuilds the question from the conclusion and picks the closest match §3.2.1. Self-consistency samples several reasoning paths and takes a majority vote over the answers §3.2.1 fig. 4. See the pseudocode below.
• Tool-based validation: the steps are written in a formal language, usually Python, and an interpreter or compiler checks them. Program-of-thoughts, Program-aided language models (PAL), Self-debugging, MathPrompter and Codex are in this group §3.2.2 tab. 2.
• External model validation: a critic or corrector model gives feedback (Refiner, Self-correction) or the model is finetuned on its own rationales (Self-improvement, STaR) §3.2.3. Say-Can and Inner Monologue ground steps in robot affordances or language feedback §3.2.3.

Control §3.3:

• Greedy selection: one prompt and one chain of steps, followed in order. Least-to-most decomposes the problem and solves sub-problems in sequence §3.3.1.
• Ensemble strategy: several chains are generated and combined, as in self-consistency and PAL §3.3.2.
• Search or reinforcement-learning control, run by an external algorithm that calls the LLM as a subroutine: Tree-of-Thoughts (breadth-first or depth-first search with a generation prompt and an evaluation prompt) §3.3.3 fig. 5; Buffer-of-Thoughts (a meta-buffer of thought templates) §3.3.3 fig. 5; beam search with self-evaluation §3.3.3; Progressive Hint Prompting (repeat with the previous answer as a hint until the answer is stable) §3.3.3; Self-Refine (generate, get feedback, refine) §3.3.3; ReAct and Reflexion (actor, evaluator and reflector) §3.3.3 fig. 6; Voyager, which writes Minecraft code with a skill library §3.3.3 fig. 7. The survey labels all of these as reinforcement learning in Table 2 tab. 2.

Self-consistency, as the survey describes it §3.2.1 fig. 4:

# Self-consistency: majority vote over sampled reasoning paths  [§3.2.1]
# Schematic: the survey gives no code. Input: a question and an LLM that
# samples one reasoning chain and its final answer.
# Output: the most frequent final answer.

def self_consistency(question, llm, num_paths):
    # Sample several chains, not the single greedy chain  [§3.2.1]
    # num_paths is not stated in the survey.
    answers = []
    for path in range(num_paths):
        chain, answer = sample_chain(llm, question)
        answers.append(answer)
    # Majority vote over the final answers  [§3.2.1]
    return most_common(answers)

Code-generation mode §4.4 and the research agenda §4.5 are discussed in the survey but are not method steps.

Hyperparameters

Not applicable. The survey runs no experiments and states no training, decoding or search settings. Where it gives sizes, they describe benchmarks or cited work: GSM8K has 8,500 problems §2.4. The number of sampled paths for self-consistency is not stated §3.2.1. Settings for each approach are in the original papers.

Results

Author-reported, not re-run. The survey copies numbers from the cited papers. Each row lists its own baseline and setup, so do not compare rows directly.

Table 1 is reproduced in full from the survey tab. 1. The baseline is GPT-3 175B, as reported in the chain-of-thought paper tab. 1:

Selected headline numbers from the taxonomy table and text. Table 2 gives the full list tab. 2; its "+" values are relative to a baseline that is "typically GPT-3.5" tab. 2.

Paper vs code

No official code. The paper has no project page, and its only URL in the text is a footnote to a list of multilingual LLM benchmarks, which is not code §4.5. A GitHub repository search for "multi-step reasoning survey plaat" returned no results. The survey is a literature review, so there is no implementation to compare.

Pitfalls

• Survey, not a method. Nothing here is an algorithm an agent can implement as written. Use the cited papers for implementation details, and the survey for choosing an approach §4.1.
• Numbers are not comparable. Models, prompts, baselines and metrics differ across the cited papers. Table 2 says its "+" values are relative to a baseline that is "typically GPT-3.5" tab. 2, but it does not state the baseline per row.
• Internal disagreements, not resolved by the survey:
  • Self-consistency gains are "10–20 percentage points" in the text §3.2.1, but Table 2 gives +4 (ARC-c) to +18 (GSM8K) tab. 2.
  • Scratchpad Python execution is "30% to 42%" in the text §3.1, which is 12 points, but Table 2 says +21% tab. 2.
  • Codex on HumanEval: "up to 77%" in the text §3.2.2, "70%" in Table 2 tab. 2. The metric variant is not stated.
  • Chain-of-thought on GSM8K: "16% to 47%" in the text §3.1.1, "15.6 → 46.9" in Table 1 tab. 1, and "+39%" in Table 2 tab. 2. The +39% matches neither pair.
• Classification is the survey's choice. It places ReAct, Reflexion and Voyager under reinforcement-learning control tab. 2 §3.3.3. The text describes them as external algorithms that manage state for a prompt-improvement loop, so the label is not a training method.
• Selection is keyword-based and not exhaustive. The search used "reasoning" and "large language model" in titles or abstracts, papers that cite chain-of-thought, and a two-year window before writing. No counts or search dates are given §2.1. Table 2 lists "the main papers" §3.
• Scope limits. Formal logic solvers are mentioned only in a footnote §3.2.2. Finetuning-heavy work is included only where it contributed to in-context methods §2.1.
• One study, broad wording. The survey reads one compositionality-gap study as showing that reasoning does not improve with model size §3.1.2. That is one study's finding, not a general result.
• Date sensitivity. The text cites 2025 work and says "we are unaware of any LLM that has been made to reflect on, or even control, its reasoning" §4.3.2. That is the authors' view at writing. The card follows arXiv v3, updated 2025-11-02 [raw metadata]; published is the first version date.
• Figure and table numbers are counted from the TeX source in order: Figures 1–7 and Tables 1–2. If the rendered PDF numbers differ, check against it.

Lineage

• Builds on: Chain-of-thought prompting, the starting point the survey assumes §2.1. Scratchpads (Nye et al., supervised) §3.1. Zero-shot chain-of-thought (Kojima et al.) §3.1.1.
• Method cards that the survey covers: Self-consistency §3.2.1; PAL §3.2.2; ReAct and Reflexion §3.3.3; Self-Refine §3.3.3; Tree-of-Thoughts §3.3.3.
• Compared against: not a comparison paper. It reports the cited papers' own baselines tab. 2.
• Superseded by: not stated.