---
kind: concept
slug: self-refinement
title: Self-refinement (draft, critique, rewrite loop)
summary: >-
  A model drafts an answer, critiques it, rewrites it, and repeats for a few rounds. The critique can come from the model alone, from running the output, or from a ranking among candidates. At test time no weights change. Gains are largest when the critique is grounded in a run or a test. Unaided self-checks on math and multiple-choice reasoning are mixed. Needs a strong instruction follower.
tags: [prompting, inference, llm, self-feedback, code-generation, agents]
papers: ["2303.17651", "2303.11366", "2305.10601", "2305.12295", "2304.05128", "2308.10144", "2308.00352", "2308.11432", "2410.07076", "2502.18864", "2604.15034", "2302.12813", "2212.08073", "2408.06292", "2411.16723", "2407.11511", "2401.01313"]
related: [llm-self-evaluation, in-context-learning, chain-of-thought-prompting, self-consistency, llm-tree-search, program-repair, multi-agent-systems, llm-agent, tool-use, retrieval-augmented-generation, hallucination-mitigation, neural-symbolic-integration, semantic-parsing, rlaif, rlhf, red-teaming]
status: draft
updated_at: 2026-11-01
---

## Overview

Self-refinement is a test-time loop around a model `M`: produce an output, ask
`M` for feedback on that output, ask `M` to rewrite the output using the
feedback, and stop on a fixed count or a signal in the feedback
[arxiv:2303.17651 §2] [arxiv:2303.17651 alg. 1]. The variants below differ in
where the feedback comes from. It can be the model's own judgement, a run of
the output, a test suite, a solver's error message, a ranking among candidates,
or a gate on stored state. The invariant most variants share is that no weights change at test time: the
only things that change are the prompt, the output, or stored notes
[arxiv:2303.17651 §1] [arxiv:2308.10144 §4.3] [arxiv:2502.18864 p. 31]. A 2023
survey files this loop under model feedback in planning, next to Reflexion's
verbal feedback [arxiv:2308.11432 §2.1.3]. A 2024 survey of prompt-based
reasoning puts Self-Refine in its step-control column, next to Reflexion and
Tree-of-Thoughts, and labels that column reinforcement learning. That label is
the survey's own classification; the loop itself trains nothing
[arxiv:2407.11511 §3.3.3] [arxiv:2407.11511 tab. 2]. One variant breaks the
invariant: Constitutional AI runs the same critique-and-revise steps offline,
to build finetuning data [arxiv:2212.08073 §3.1].

The feedback is meant to be *actionable* (it names a concrete change) and
*specific* (it points at the phrase or code span to change) [arxiv:2303.17651 §2].
Where the rewrite sees the whole history of earlier drafts and critiques,
the loop is a history-conditioned refinement [arxiv:2303.17651 §2] (eq. 4).

## Variants

The corpus has seventeen papers that use or describe this loop, each for a different purpose.
The first four variants come from the Self-Refine paper, which defines the
loop with a single model and few-shot prompts. The rest come from papers that
use a run, a memory, a tournament or a versioned state as the feedback source,
or that apply the critique loop inside a larger pipeline.
Cards are linked in each entry.

### Self-feedback with few-shot prompts (the default)

The same model, with three few-shot prompts (generate, feedback, refine).
Feedback can carry numeric aspect scores; the output with the best total
score across rounds is returned instead of the last one
[arxiv:2303.17651 app. M] [arxiv:2303.17651 app. H]. This is the form
used for dialogue and acronym tasks. Card: [Self-Refine: Iterative Refinement with Self-Feedback](/papers/2303.17651).

### Stop on a feedback signal

The loop ends when the feedback contains a task-specific phrase, such as a
stop score or a "correct" marker, rather than after a fixed count
[arxiv:2303.17651 §2]. The paper leaves the signal to each task; its
concrete phrases are listed in the card's Paper vs code section.

### Oracle feedback (external correctness signal)

Refine only when an external check says the answer is wrong. On GSM8K this
lifts GPT-3.5 from 64.1 to 68.9 and ChatGPT from 74.8 to 76.2, where
self-feedback alone gave 64.1 and 75.0 [arxiv:2303.17651 app. H]. This
variant uses ground truth, so it is not self-contained. The code-focused
variant in the next section is the same idea with unit tests as the check.

### Retrieved evidence with template or self-critique feedback (LLM-Augmenter)

A frozen, black-box LLM answers from evidence that a retriever and linker
gathered first. A utility step then scores each candidate against that
evidence and returns feedback, and the model is asked again until the answer
passes. The paper states no maximum number of rounds and no threshold value
[arxiv:2302.12813 §3.2] [arxiv:2302.12813 §1]. Feedback comes either from a
template that fires on a low score, or from ChatGPT asked to criticise its own
answer [arxiv:2302.12813 §3.2]. The score is knowledge F1, the token overlap
between answer and evidence, so the loop checks grounding in the evidence, not
correctness [arxiv:2302.12813 §3.2]. On customer-service dialog with BM25
retrieval, KF1 goes from 34.07 without feedback to 37.41 with the template and
to 37.10 with self-criticism [arxiv:2302.12813 tab. 4]. ChatGPT alone reaches
31.33 on the same task [arxiv:2302.12813 tab. 2]. On OTT-QA, the paper reports
Wiki QA F1 rising from 0.59 for ChatGPT alone to 11.80 with CORE evidence and
feedback [arxiv:2302.12813 tab. 5]. Card: [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](/papers/2302.12813).

### Latest-output-only refinement

Refine sees only the latest output and its feedback, not the history
(eq. 3 instead of eq. 4) [arxiv:2303.17651 §2]. The paper's code does this
for code optimization; see the card's Paper vs code section for the
details.

### Execution feedback from tests or a run

The program is executed, and the result (failed unit tests, a runtime error,
an output mismatch) goes into the feedback. Self-Debugging generates a program,
runs it, has the model explain it line by line, judges it, and asks for a fix.
It stops on a correct verdict or after 10 turns [arxiv:2304.05128 §3]
[arxiv:2304.05128 §5]. Feedback can be a bare verdict, a verdict with the failed
tests, a model-written explanation, or a model-written trace. A trace is generated
text, not real intermediate state [arxiv:2304.05128 §3]. On TransCoder, where all
10 unit tests are in the prompt, Codex goes from 80.4 to 91.6 with test feedback
and to 92.5 with test feedback plus explanation [arxiv:2304.05128 tab. 2]. On MBPP,
where two of three tests are hidden, GPT-4 goes from 72.8 to 80.6 with test
feedback [arxiv:2304.05128 tab. 2] [arxiv:2304.05128 §4.3]. Card: [Teaching Large Language Models to Self-Debug](/papers/2304.05128).

MetaGPT uses the same pattern inside a multi-agent software team. The engineer
writes unit tests, runs them, and debugs on failure, with a retry limit of three
in the text [arxiv:2308.00352 §3.3]. The authors report +4.2 Pass@1 points on
HumanEval and +5.4 on MBPP from executable feedback [arxiv:2308.00352 §4.4].
Card: [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](/papers/2308.00352).

### Explanation feedback when no tests exist

When the output cannot be tested, the check is the model's own reading of a run.
For text-to-SQL, the loop executes the query, passes the first two rows of the
result to the model, and asks it to explain each clause and judge the query
against the question [arxiv:2304.05128 §4.1]. This lifts Codex on Spider from
81.3 to 84.1. Simple feedback without an explanation left Codex at its baseline
[arxiv:2304.05128 tab. 2]. On MBPP without tests, simple feedback lowered Codex
from 61.4 to 57.6, below its baseline, while explanation feedback reached 64.4
[arxiv:2304.05128 tab. 3].

### Solver error message in a formal program (Logic-LM)

The model translates a logic problem into a formal program, and a deterministic
solver runs it. Only a program that fails to execute goes back to the model,
with the solver's error message and a few demonstrations of common errors. The
model then rewrites the program [arxiv:2305.12295 §3.3] [arxiv:2305.12295 fig. 3].
The paper gives no revision limit in its text; the figure shows zero to three
rounds [arxiv:2305.12295 §4.4]. The paper names this module Self-Refiner and builds
it on self-debugging and self-refine [arxiv:2305.12295 §3.3]. Refinement raises the
share of programs that execute: the paper reports a mean gain of 5.01 points
[arxiv:2305.12295 §4.4]. Accuracy on the programs that execute moves in both directions. For GPT-3.5
on FOLIO it rises from 61.8 to 64.3, while for GPT-4 on FOLIO it falls from 80.4 to 79.9,
even though that model's execution rate rises from 79.9 to 85.8 percent
[arxiv:2305.12295 tab. 3]. The main results table
reports Logic-LM without refinement, so the end-to-end effect of this loop on accuracy
is not shown there [arxiv:2305.12295 §4] [arxiv:2305.12295 tab. 2]. Card:
[Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](/papers/2305.12295).

### Verbal reflection in a bounded memory (Reflexion)

After each failed trial on the same task, the model writes a verbal lesson.
The lesson goes into a memory that is read before the next trial, and only the
last one to three lessons are kept [arxiv:2303.11366 §3] [arxiv:2303.11366 §4.1].
The evaluator can be exact-match grading, a heuristic, an LLM classifier, or
unit tests that the LLM writes itself [arxiv:2303.11366 §3] [arxiv:2303.11366 §4.3].
On HumanEval (Python) the paper reports 91.0 pass@1 against 80.1 for GPT-4
[arxiv:2303.11366 tab. 2]. It reports +22% absolute over ReAct on ALFWorld and
+20% on HotPotQA [arxiv:2303.11366 §1]. On MBPP (Python) it reports 77.1, below
the 80.1 baseline [arxiv:2303.11366 tab. 2]. Its numbers are author-reported.
Card: [Reflexion: Language Agents with Verbal Reinforcement Learning](/papers/2303.11366).

### Retries with cross-task insight memory (ExpeL)

Each training task gets up to three reflection retries, with every trial stored
[arxiv:2308.10144 §4.1] [arxiv:2308.10144 tab. 4]. The model then compares
successful and failed attempts on the same task, and summarises successes across
tasks, into a list of written insights [arxiv:2308.10144 §4.2.2]. Each insight
carries a count that goes up or down as later comparisons agree or disagree.
At test time the full insight list and the most similar past successes go into
the prompt [arxiv:2308.10144 §4.3]. With gpt-3.5-turbo as the acting model, the
authors report success rates of 28.0 to 39.0 on HotpotQA and 40.0 to 59.0 on
ALFWorld, against plain ReAct [arxiv:2308.10144 tab. 3]. Tokens per HotpotQA
trajectory rise from about 1,320 to 4,310 [arxiv:2308.10144 tab. 6]. In the
authors' ablation, feeding reflections into insight extraction lowered HotpotQA
to 29.0, which they attribute partly to hallucinated reflections
[arxiv:2308.10144 tab. 3] [arxiv:2308.10144 §5.6]. Card: [ExpeL: LLM Agents Are Experiential Learners](/papers/2308.10144).

### Reviewer agent in a group chat (coder and reviewer)

A second agent reviews the code for correctness, task fit and safety. It
either sends the code back to the coder with feedback, or hands it to an
executor [arxiv:2411.16723 §3.1]. The paper's text gives no round limit and no
stop rule. The repository uses a maximum of 12 rounds and ends the chat when the
reviewer writes the word "terminate"; the card's Paper vs code section covers
this and other gaps. On a Boston Dynamics Spot, with 7 spoken prompts and 3
runs, the coder-and-reviewer setup had the fewest first-attempt code errors:
9 points below a single agent and 14 points below a planner-coder-reviewer
team [arxiv:2411.16723 §4] [arxiv:2411.16723 fig. 6]. Observer scores on task
success, safety and sociability did not differ significantly across the three
configurations [arxiv:2411.16723 §4] [arxiv:2411.16723 fig. 8]. The paper
reports no statistical test. Card: [Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction](/papers/2411.16723).

### Tournament critique and evolution (Co-Scientist)

A supervisor coordinates generation, review, ranking, proximity, evolution and
meta-review agents over a task queue [arxiv:2502.18864 p. 29–31]. Hypotheses
start at an Elo rating of 1200 and are compared pairwise. Top-ranked pairs get a
multi-turn debate, and others a single comparison [arxiv:2502.18864 p. 34]. The
meta-review turns the reviews and debates into a critique, which is added to every
agent's prompt in the next iteration, with no fine-tuning [arxiv:2502.18864 p. 31]
[arxiv:2502.18864 p. 35–36]. Across 203 research goals, the authors report the
best hypothesis's Elo rising from about 1,350 to about 1,610 over ten time buckets
[arxiv:2502.18864 p. 7] [arxiv:2502.18864 fig. 2a]; these are values read from
a figure. The ratings come from the system's own LLM judgements, not from ground
truth [arxiv:2502.18864 p. 10]. No public code is available [arxiv:2502.18864 p. 40].
Card: [Accelerating scientific discovery with Co-Scientist](/papers/2502.18864).

### Critique-and-revise inside a hypothesis generator (MOOSE-Chem)

Each candidate hypothesis is drafted, then critiqued on validness, novelty,
clarity and significance, then revised. The paper builds on Self-Refine for
this critique loop [arxiv:2410.07076 §6.1]. Each mutation line drafts once and
revises twice, and only the final draft is scored; the mean of the four
self-scores sets the beam and the final order [arxiv:2410.07076 §4.2.3]
[arxiv:2410.07076 §4.2.4]. The loop sits inside a larger pipeline of literature
screening and recombination, so the paper does not isolate its effect. Its
ablations remove the multi-step and evolutionary parts, not the critique:
the top Matched Score falls from 4.020 to 3.765 without multi-step and to
2.863 without multi-step and the evolutionary unit [arxiv:2410.07076 tab. 10].
The Matched Score is judged by GPT-4o, not by experts [arxiv:2410.07076 app. A.14].
Card: [MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses](/papers/2410.07076).

### Fixed reflection passes in a research pipeline (AI Scientist)

The pipeline has no external checker, so each stage uses a fixed number of
self-reflection rounds. Idea generation runs three rounds on each idea and stops
early when the model signals it is finished [arxiv:2408.06292 §3]
[arxiv:2408.06292 tab. 6]. The write-up gives each section one reflection pass,
then a final pass that removes repetition [arxiv:2408.06292 §3]. The reviewer
averages several independent reviews, merges them, and runs five reflection
rounds on the merged review [arxiv:2408.06292 §4] [arxiv:2408.06292 tab. 6].
The paper does not remove these passes in an ablation, so their effect on
output quality is not measured. The authors report about $10 to $15 of API
cost per generated paper [arxiv:2408.06292 §6]. The reviewer and the generator
are both language models, so the score is a model judgement, not a reference.
Card: [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](/papers/2408.06292).

### Gated self-modification of versioned resources (Autogenesis)

Prompts, agents, tools, environments and memory are registered as versioned
resources. A reflection loop proposes an edit, assesses the candidate, and
commits it only if the assessment is accepted. A rejected candidate leaves the
active version as it was [arxiv:2604.15034 §3.2] [arxiv:2604.15034 app. G]. In
one complete run on GPQA-Diamond with gpt-4o, a prompt edit moved accuracy from
47.98 to 53.54 percent [arxiv:2604.15034 tab. 1]. In three repeated gpt-4o runs,
a simple reflection baseline scored 54.04 on GPQA-Diamond against 47.98 for the
vanilla model, and 10.00 against 13.33 on AIME24 [arxiv:2604.15034 app. B]. The
acceptance test is the same model's judgement, not a reference answer
[arxiv:2604.15034 §3.2]. The paper does not show recovery after a faulty version
goes live [arxiv:2604.15034 §6]. Joint edits used about 52 to 61 percent more
output tokens than matched controls [arxiv:2604.15034 app. B]. The card records
that the paper's linked repository, at the commit checked, did not contain the
reflection loop under the paper's names, so these numbers are not reproduced
here. Card: [Autogenesis: A Self-Evolving Agent Protocol](/papers/2604.15034).

### Survey-listed self-checks for hallucination (nine works)

A 2024 survey of hallucination mitigation files nine works under
self-refinement: prompting GPT-3 for reliability, ChatProtect, Mind's Mirror, a
self-reflection method for medical QA, structured comparative reasoning, DRESS
for vision-language models, MixAlign, Chain-of-Verification (CoVe) and CoNLI
[arxiv:2401.01313 tab. 1] [arxiv:2401.01313 §2.1.2]. CoVe drafts an answer,
plans checks, answers them independently and rewrites the answer. CoNLI judges
sentences and entities with a chain of natural language inference steps, then
post-edits the text [arxiv:2401.01313 §2.1.2]. The table gives tasks and models
for each work but no result values, and it marks all nine as covering
detection, a term the survey does not define [arxiv:2401.01313 tab. 1]. The
survey describes CoNLI as detection followed by post-editing, yet files it under
self-refinement [arxiv:2401.01313 §2.1.2], so the family label does not tell you
the mechanism. Card: [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](/papers/2401.01313).

### Offline critique-revision for training data (Constitutional AI)

This variant runs the loop before deployment, not at test time. For each
red-team prompt, the helpful model samples a response at temperature 1. Then,
for four rounds, it draws one of 16 written principles, writes a critique, and
writes a revision [arxiv:2212.08073 §3.1] [arxiv:2212.08073 §3.2]. The
training pair is the prompt and the final revision; the critique text is not
kept [arxiv:2212.08073 §3.1]. The paper uses 182,831 red-team prompts and
finetunes a pretrained model for one epoch on all revisions and the helpfulness
samples [arxiv:2212.08073 §3.2]. Against helpful RLHF, the result is more
harmless and less helpful [arxiv:2212.08073 §3.3] [arxiv:2212.08073 fig. 3].
Harmlessness scores rise with each revision and helpfulness falls
[arxiv:2212.08073 fig. 5]. Critiqued revisions beat direct revisions for smaller
models, and the two are similar at 52B [arxiv:2212.08073 §3.5]
[arxiv:2212.08073 fig. 7]. These results are author-reported and read from
plots. The RL stage that follows uses AI preference labels, not this loop; the
[RLAIF concept page](/concepts/rlaif) covers that stage. Card: [Constitutional AI: Harmlessness from AI Feedback](/papers/2212.08073).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Output quality is open-ended or has several criteria (style, tone, readability) | Self-feedback with aspect scores, keep the best round | Largest author-reported gains: GPT-4 dialogue preference 25.4 to 74.6, readability 27.4 to 56.2 [arxiv:2303.17651 tab. 1] |
| Output is a single number or a short proof, and the model cannot check its own steps | Expect little from self-feedback alone; add an external check | GSM8K moved by 0.2 points with GPT-4, and by 0 for GPT-3.5 [arxiv:2303.17651 tab. 1]; oracle feedback is needed for gains [arxiv:2303.17651 app. H] |
| You have a checker (tests, a verifier, a reference) you can run after each round | Execution feedback, rewrite only on failure | Test feedback gave the largest code gains: TransCoder, Codex 80.4 to 91.6 [arxiv:2304.05128 tab. 2]. Stop signal is objective |
| Answers must be grounded in a searchable source, and the LLM is a hosted model you cannot fine-tune | Retrieve and link evidence first, then a feedback loop on the answer | Customer service, BM25: 31.33 KF1 for ChatGPT alone, 37.41 with template feedback [arxiv:2302.12813 tab. 2] [arxiv:2302.12813 tab. 4]. The paper often queries the LLM twice per response and calls the cost high [arxiv:2302.12813 §6] |
| Code or a query cannot be tested, but can be run and read back | Explanation feedback on the run result | Spider, Codex 81.3 to 84.1 [arxiv:2304.05128 tab. 2]; simple feedback alone left Codex at 81.3 [arxiv:2304.05128 tab. 2] |
| The model is small or weak at instruction following | Do not use self-refinement with it | A 13B open model could not produce parseable feedback or follow the refine prompt [arxiv:2303.17651 §4]. Reflexion reports no change for Starchat-beta on HumanEval, 0.26 for both baseline and Reflexion [arxiv:2303.11366 app. A] |
| Repeated attempts at one task, and the only evaluator is the model's own tests | Reflection memory with self-written tests, and treat a pass as weak evidence | Rust, 50 hardest HumanEval problems, GPT-4: 0.60 to 0.68 [arxiv:2303.11366 tab. 4]; MBPP Python passes 77.1 against 80.1 for the baseline, with 16% false positives [arxiv:2303.11366 tab. 2] [arxiv:2303.11366 tab. 3] |
| The output is a formal program (logic, a query, a constraint model) that a deterministic solver can run | Run the solver and rewrite only on an execution error, feeding back the error message | Execution rate rose or held in every row of the paper's table, and the largest rise is GPT-3.5 on FOLIO (66.7 to 84.3 percent) [arxiv:2305.12295 tab. 3]; the headline accuracy results exclude refinement [arxiv:2305.12295 tab. 2] |
| Latency or cost per request is tight | Keep rounds at one or two; skip history | Each round is a feedback call plus a refine call, and the refine prompt grows with every round [arxiv:2303.17651 §2] (eq. 4). The Self-Debugging paper reports no token or latency cost [arxiv:2304.05128 §5] |
| You can judge your output with a different, stronger model | Consider an external judge | The paper's preference scores come from GPT-4, which is also one of the models it refines [arxiv:2303.17651 §3.2] |
| Code for a physical robot, with a human watching the run, and you want a second agent to catch errors before the first run | Coder plus reviewer in a chat loop; a planner in front did not help in this study | First-attempt error rate 9 points below a single agent and 14 below the planner setup [arxiv:2411.16723 §4] [arxiv:2411.16723 fig. 6]; the study has 7 prompts and 3 runs [arxiv:2411.16723 §3] |
| Many candidates are affordable and you can build a pairwise judge | Tournament ranking with critique | Best-hypothesis Elo rises with run time [arxiv:2502.18864 fig. 2a], but the judge is the system itself [arxiv:2502.18864 p. 10] |
| Many candidate ideas from a literature pool, no ground truth to check against, and a judge you can trust only partly | Critique-and-revise per candidate, then rank by self-score | Top Matched Score 4.020 vs 2.882 for MOOSE, judged by GPT-4o [arxiv:2410.07076 tab. 10]; the self-scores also set the beam, and the paper's own rank check is weak [arxiv:2410.07076 tab. 8] |
| You need the agent to improve across tasks without weight updates, and can afford a training phase | Insight memory built from successes and failures | HotpotQA 28.0 to 39.0 with gpt-3.5-turbo [arxiv:2308.10144 tab. 3]; prompts grow [arxiv:2308.10144 tab. 6] |
| A long document written in sections, with no checker, and you can afford a few extra model calls per section | One reflection pass per section, then a final pass that removes repetition | The AI Scientist write-up uses this order [arxiv:2408.06292 §3], but the paper does not measure what the passes add [arxiv:2408.06292 §3] |
| You want to see which self-check methods have been tried for hallucination before choosing one | Start from the survey's self-refinement rows, then read each primary paper | The survey lists nine such works, runs no experiments of its own, and gives no result values [arxiv:2401.01313 tab. 1] [arxiv:2401.01313 §2.1.2] |
| You must version and roll back prompts or tools | Gated commit with a stored history | The gate keeps the active version safe from a rejected edit [arxiv:2604.15034 §3.2]; the paper does not test rollback [arxiv:2604.15034 §6] |
| You have a training phase, want a model that follows written principles, and have no human harmlessness labels | Offline critique-revision of your own prompts, then finetune on the revisions | Revisions raised the harmlessness score and lowered helpfulness [arxiv:2212.08073 fig. 5]. Helpfulness labels were still human [arxiv:2212.08073 §6] |

Practical default: start with one or two rounds, a parseable feedback
format and a stop rule that is a plain string match. If the task has a
checker, make the checker the stop rule rather than the model's verdict
[arxiv:2304.05128 §3]. If the tests are the model's own, treat a pass as weak
evidence: the Reflexion paper measures false passes at 16% on MBPP Python
[arxiv:2303.11366 tab. 3]. Measure against the single-pass output on your own
tasks before adding rounds. Gains in the Self-Refine paper grow with the number
of rounds, but the curve is flattening [arxiv:2303.17651 fig. 4].

## Implementation sketch

```pseudocode
# Self-refinement loop  [arxiv:2303.17651 alg. 1]
# Input:  input x, model M (text in, text out), few-shot prompts
#         p_gen, p_fb, p_refine, stop(feedback, t) -> bool
# Output: rewritten output

def self_refine(x, M, p_gen, p_fb, p_refine, stop, max_rounds):
    # First draft from input-output examples  [eq. 1]
    output = M(p_gen + x)
    history = []
    for t in range(max_rounds):
        # Same model critiques its latest output  [eq. 2]
        feedback = M(p_fb + x + output)
        # Stop on a fixed count or a signal parsed from feedback  [§2]
        if stop(feedback, t):
            break
        history.append((output, feedback))
        # Rewrite using all earlier outputs and feedback  [eq. 4]
        prompt = p_refine + x
        for prev_output, prev_feedback in history:
            prompt = prompt + prev_output + prev_feedback
        output = M(prompt)
    # Return the last rewrite, or the best-scored output  [app. H]
    return output
```

Where variants differ: the source of the feedback (the model's own reading,
a run of the output, a tournament, or a judge that is not the same model), the
stop test (fixed count, parsed score, string match, or a test pass), whether the
rewrite gets the full history (eq. 4), only the last pair (eq. 3), or a window
of the last one to three reflections [arxiv:2303.11366 §3], whether only an execution error triggers a rewrite, with the solver's message as feedback [arxiv:2305.12295 §3.3], whether the feedback is checked against retrieved evidence
[arxiv:2302.12813 §2.4], whether the loop runs at inference or offline, where
only the final revision is kept as a training pair and the critique is dropped
[arxiv:2212.08073 §3.1], whether a candidate must pass a gate before it replaces the active output
[arxiv:2604.15034 §3.2], and the selection rule at the end. Also keep the
feedback format strict: a parse failure should be counted and handled, not
silently dropped.

## Contradictions

Within [arxiv:2303.17651](/papers/2303.17651) the reported numbers do not agree
everywhere. The GSM8K self-refine result is 64.1, 75.0 and 93.1 in the main
table [arxiv:2303.17651 tab. 1] but 62.4, 75.1 and 94.5 in the comparison table
[arxiv:2303.17651 app. F]. The stated decoding temperature is also
self-contradictory [arxiv:2303.17651 §3.1]. Treat small differences as noise and
check the card's Pitfalls section before reproducing a number.

Across papers, the evidence on unaided self-judgement points in different
directions. We do not resolve these; the tasks differ:

- **Self-judgement without a check.** On MBPP without tests, simple feedback
  lowered Codex from 61.4 to 57.6 [arxiv:2304.05128 tab. 3]. The Self-Refine
  paper reports gains from self-feedback on its own tasks [arxiv:2303.17651 tab. 1].
  MOOSE-Chem uses the same model's self-scores to rank hypotheses and reports
  a rank-ratio gain for drafts with matched inspirations, but its check is
  weak and its judge is a language model [arxiv:2410.07076 tab. 8] [arxiv:2410.07076 app. A.14].
  LLM-Augmenter finds self-criticism close to its template feedback on KF1
  (37.10 against 37.41) [arxiv:2302.12813 tab. 4]. KF1 measures overlap with
  retrieved evidence, not correctness, so this does not show that the answers
  are right.
- **Critiques that are often wrong.** Constitutional AI reports that its
  critiques were frequently inaccurate or overstated, yet the revisions still
  scored as more harmless [arxiv:2212.08073 §3.5] [arxiv:2212.08073 app. A].
  The score comes from a preference model trained on AI labels, not from ground
  truth, so the revisions are not shown to be correct.
- **Math and multiple-choice reasoning.** Self-Refine reports almost no change
  on GSM8K [arxiv:2303.17651 tab. 1]. The Autogenesis paper reports a rise on
  GPQA-Diamond, a multiple-choice science set, from 47.98 to 53.54 for gpt-4o
  and from 65.15 to 68.69 for gpt-4.1, with a single run each
  [arxiv:2604.15034 tab. 1]. Its repeated runs show a mixed picture on AIME24
  [arxiv:2604.15034 app. B]. GPQA is not grade-school arithmetic, so the gap may
  reflect the task rather than a contradiction.
- **Where the gain comes from on code.** On the 50 hardest HumanEval problems in
  Rust with GPT-4, Reflexion's ablation scores 0.60 without the reflection text,
  0.52 without generated tests, and 0.68 with both [arxiv:2303.11366 tab. 4].
  Read that as the reflection text carrying the gain. Self-Debugging's TransCoder
  result points the other way: test feedback carries most of the gain, and the
  explanation adds 0.9 points (80.4 to 91.6, then 92.5) [arxiv:2304.05128 tab. 2].
  The tasks, models and benchmarks differ, so we do not resolve this.
- **Execution rate and accuracy.** In the Logic-LM paper, refinement raises how many
  programs run, but accuracy on the programs that run moves both ways: up for GPT-3.5 on
  FOLIO, down for GPT-4 on FOLIO and AR-LSAT (60.0 to 58.8) [arxiv:2305.12295 tab. 3].
  The released code refines only FOLIO and AR-LSAT, so the refinement rows for the other
  three datasets in that table cannot come from it; see the card's Paper vs code section
  [arxiv:2305.12295 tab. 3].
- **Two agents versus three.** The paper's Discussion concludes that the
  two-agent setup performs better than the three-agent one [arxiv:2411.16723 §5],
  and calls the effect significant. The Results text says observer scores do
  not differ significantly across configurations [arxiv:2411.16723 §4]. The
  paper reports no test, and each configuration was scored by one observer per
  run. Treat the conclusion as tentative.
- **Typical size of the gain.** The 2024 survey gives a typical Self-Refine gain
  of about 30 points [arxiv:2407.11511 §3.3.3] [arxiv:2407.11511 tab. 2]. The
  Self-Refine table cited above does not support a single typical figure: the
  GPT-4 gain is 0.2 points on GSM8K [arxiv:2303.17651 tab. 1] and 49 points on
  dialogue preference [arxiv:2303.17651 tab. 1]. The survey does not say which
  tasks its figure averages over, so treat it as a loose summary.
- **Size of the rounds.** Self-Debugging reports that most successful runs end
  within three turns [arxiv:2304.05128 §5], while the Self-Refine paper reports
  gains that grow with more rounds [arxiv:2303.17651 fig. 4].

## Papers

- [Self-Refine: Iterative Refinement with Self-Feedback](/papers/2303.17651): defines the draft, critique, rewrite loop with a single model and few-shot prompts, and tests it on seven tasks.
- [Reflexion: Language Agents with Verbal Reinforcement Learning](/papers/2303.11366): writes a verbal lesson after each failed trial and keeps the last one to three in memory; the evaluator is exact match, a heuristic, a classifier or self-written tests.
- [A Survey on Large Language Model based Autonomous Agents](/papers/2308.11432): a 2023 survey whose taxonomy places Self-Refine and Reflexion under model feedback in planning; it reports no experiments of its own.
- [Tree of Thoughts](/papers/2305.10601): describes self-refinement as a special case of its tree search [arxiv:2305.10601 §3], and reports an iterative refine baseline on Game of 24 that uses ground-truth feedback [arxiv:2305.10601 §4.1].
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](/papers/2302.12813): wraps a frozen ChatGPT with evidence consolidation and a utility step whose feedback re-prompts the model until the answer passes.
- [Teaching Large Language Models to Self-Debug](/papers/2304.05128): runs generated code or queries, has the model explain and judge them, and fixes them for up to 10 turns; gains depend on tests or on the run result.
- [ExpeL: LLM Agents Are Experiential Learners](/papers/2308.10144): per-task reflection retries, then written cross-task insights that go into the prompt of a frozen agent.
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](/papers/2308.00352): a software-team agent system whose engineer tests and debugs its code within a retry limit.
- [Accelerating scientific discovery with Co-Scientist](/papers/2502.18864): a multi-agent hypothesis generator where a meta-review feeds critique back to every agent and an Elo tournament ranks the output.
- [MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses](/papers/2410.07076): a three-step chemistry hypothesis pipeline whose candidates are critiqued and revised, then ranked by self-scores.
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](/papers/2305.12295): an LLM writes a formal program, a solver runs it, and the program is rewritten only after an execution error, with the error message as feedback.
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](/papers/2408.06292): an LLM pipeline that proposes ideas, runs experiments with a coding agent, writes a paper, and has it reviewed; each stage uses a fixed number of reflection passes.
- [Autogenesis: A Self-Evolving Agent Protocol](/papers/2604.15034): versioned prompts, agents, tools and memory that change through a propose, assess, commit cycle.
- [Constitutional AI: Harmlessness from AI Feedback](/papers/2212.08073): runs critique and revision offline on red-team prompts under 16 written principles, then finetunes on the revisions; its RL stage uses AI preference labels.
- [Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction](/papers/2411.16723): a coder and a reviewer agent write robot code in a group chat, compared with one agent and with a planner-coder-reviewer team.
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](/papers/2401.01313): a survey that files nine self-refinement works under prompt engineering; it runs no experiments and its table gives no result values.
- [Multi-Step Reasoning with Large Language Models, a Survey](/papers/2407.11511): a taxonomy of prompt-based reasoning that files Self-Refine under step control and reports the cited papers' own numbers; it runs no experiments.