--- kind: concept slug: llm-self-evaluation title: LLM self-evaluation summary: >- A language model judges its own output, or a partial one, and the judgement sets the next step: a rewrite, a re-run, a retry with a written lesson, a search choice or a rank. Mostly no training, but every judgement is an extra model call and is only as good as the judge. Author-reported gains are largest where the judge sees concrete errors, such as failing tests, and small on math. tags: [prompting, inference, evaluation, search] papers: ["2303.17651", "2305.10601", "2304.05128", "2502.18864", "2410.07076", "2603.17839", "2207.05221", "2303.11366"] related: [chain-of-thought-prompting, in-context-learning, self-refinement, llm-as-a-judge, llm-tree-search, calibration, uncertainty-estimation, hallucination-detection, multi-agent-systems, llm-agent] status: draft updated_at: 2026-10-19 --- ## Overview Self-evaluation means the model that produced an answer is also the one that scores it, critiques it or compares it with alternatives. The judgement is then used inside the same program: to rewrite a draft, to re-run and fix a program, to decide which partial solution to extend, or to rank candidate hypotheses [arxiv:2303.17651 §2] [arxiv:2305.10601 §3] [arxiv:2304.05128 §3] [arxiv:2502.18864 p. 34] [arxiv:2410.07076 §4.2.4]. Reflexion writes a verbal lesson after each failed attempt and keeps it for the next one [arxiv:2303.11366 §3]. Kadavath et al. ask a model whether its own sampled answer is True [arxiv:2207.05221 §4.1]. Most of these methods train nothing. The exception is the P(IK) value head, which is trained on labelled samples [arxiv:2207.05221 §5]. Self-Refine drives every step with few-shot prompts [arxiv:2303.17651 §2], and the tree search likewise uses prompts only [arxiv:2305.10601 §3]. Self-Debugging is also prompt-only [arxiv:2304.05128 §3]. The problem these papers address is that a single left-to-right pass can miss goals that are hard to specify or that depend on an early choice. Self-Refine asks whether one off-the-shelf model can critique and improve its own output without extra data [arxiv:2303.17651 §1]. Tree of Thoughts asks whether the same model can serve as the heuristic for a search over intermediate steps [arxiv:2305.10601 §1] [arxiv:2305.10601 §3]. Self-Debugging asks whether a frozen model can find its own bugs from its own explanation or from running its code [arxiv:2304.05128 §1] [§3]. Reflexion asks whether an agent can improve across retries from words alone, without changing weights [arxiv:2303.11366 §1]. Kadavath et al. ask whether models can tell when they are right, and whether that can be trained [arxiv:2207.05221 §1]. The invariant across the decision variants: the judge sees a candidate, produces a signal (feedback text, a score, a class, a vote or a rating), and that signal decides the next step. The signal is the weak point, though the papers locate it differently. Self-Refine traces most failures to bad feedback [arxiv:2303.17651 §4], while the tree-search paper finds that generation is the bottleneck when models are mixed [arxiv:2305.10601 app. B.2]. Where the candidate can be run, the run gives the judge a signal it does not have to invent [arxiv:2304.05128 §3]. Reflexion's coding gains rest on tests the model writes itself, which can be wrong [arxiv:2303.11366 tab. 3]. The confidence variants are different: the rating is the output, and nothing is re-run. This page covers the eight papers on wikillm under a reuse license that study self-evaluation. Other self-evaluation work, such as Self-Correct (Welleck et al., [arXiv:2211.00053](https://arxiv.org/abs/2211.00053)), which trains a separate corrector, has no card here and is not summarized. ## Variants ### Iterative self-feedback and rewrite The same model writes a draft, then writes feedback on its own draft, then rewrites the draft using that feedback and all earlier drafts and feedback. The feedback is meant to be actionable (it names a change) and specific (it names the phrase or code to change) [arxiv:2303.17651 §2]. The loop stops after at most 4 rounds or when a task-specific signal appears in the feedback [arxiv:2303.17651 §3.1] [arxiv:2303.17651 alg. 1]. Author-reported GPT-4 results over one-pass output include dialogue responses from 25.4% to 74.6% and constrained generation from 15.0% to 45.0%, while GSM8K moved from 92.9% to 93.1% [arxiv:2303.17651 tab. 1]. Card: [Self-Refine](/papers/2303.17651). The feedback carries most of the effect. On sentiment reversal with ChatGPT, feedback written in the Self-Refine style scored 43.2%, generic feedback such as "improve it" scored 31.2%, and no feedback scored 0 [arxiv:2303.17651 tab. 2]. In the authors' manual analysis of 70 samples, 33% of failures came from feedback that pointed at the wrong location and 61% from feedback that proposed a wrong fix [arxiv:2303.17651 §4]. Output selection is a per-task choice. The main algorithm returns the last rewrite, but for multi-aspect tasks the appendix keeps every output with its feedback score and returns the highest-scoring one [arxiv:2303.17651 alg. 1] [arxiv:2303.17651 app. H] [arxiv:2303.17651 app. M]. ### Execution-grounded self-debugging The model writes a program, the program is run on unit tests or on sample inputs, and the model explains the code, judges it from the run or from its explanation, and asks for a fixed program. The loop runs for up to 10 turns, and the paper's successful runs mostly end within 3 [arxiv:2304.05128 §3] [§5]. The feedback comes in four forms: a bare verdict, a verdict with the failed unit-test results, a line-by-line explanation, and a generated execution trace [arxiv:2304.05128 §3]. With unit-test feedback plus explanation, Codex on TransCoder rose from 80.4 to 92.5, and GPT-4 on MBPP rose from 72.8 to 80.6 with unit-test feedback [arxiv:2304.05128 tab. 2]. Card: [Teaching Large Language Models to Self-Debug](/papers/2304.05128). Where the tests are visible, most of the gain comes from running the program [arxiv:2304.05128 §5.1] [§5.2.2]. On text-to-SQL, which has no unit tests, the explanation step is what helps. Codex rose from 81.3 to 84.1 with an explanation, while a bare verdict left it at 81.3 [arxiv:2304.05128 tab. 2] [§5.1]. The SQL is still executed, and its first two result rows go into the explanation prompt [arxiv:2304.05128 §4.1]. Without tests, the effect depends on the model. On MBPP, a bare verdict lowered Codex from 61.4 to 57.6, while GPT-4 rose from 72.8 to 76.0 [arxiv:2304.05128 tab. 3]. ### Verbal self-reflection across retries The agent runs a trial, an evaluator scores it, and on failure the model writes a short reflection on what went wrong. The reflection goes into a bounded memory that is read before the next trial. The evaluator can be an environment or exact-match grader, a heuristic or LLM classifier, or unit tests the model wrote itself [arxiv:2303.11366 §3] [§4.1] [§4.2] [§4.3]. The memory window is 1 to 3 reflections [arxiv:2303.11366 §3] [§4.1]. Nothing is trained. Author-reported GPT-4 pass@1 on HumanEval Python rose from 80.1 to 91.0 [arxiv:2303.11366 tab. 2]. On ALFWorld the authors report +22% absolute over ReAct within 12 trials [arxiv:2303.11366 §1] [§4.1]. Each trial is a full episode and each failure adds a reflection call, so cost grows with trials [arxiv:2303.11366 §4.1]. The gain depends on the evaluator. In the 50-hardest-Rust ablation, removing the self-written tests drops the score from 0.68 to 0.52, and removing the reflection text leaves it at 0.60, the base level [arxiv:2303.11366 tab. 4]. A weak test suite passes wrong code: the false-positive rate is 0.16 on MBPP Python [arxiv:2303.11366 tab. 3]. Card: [Reflexion](/papers/2303.11366). ### Value scoring inside a tree search Here the model is not rewriting a finished answer. It proposes next steps for a partial solution, then scores each new partial solution on its own. The score is a class such as sure, maybe or impossible, mapped to a number, and it is the heuristic for either breadth-first search (keep the top b states) or depth-first search (prune states below a threshold and backtrack) [arxiv:2305.10601 §3] [arxiv:2305.10601 alg. 1] [arxiv:2305.10601 alg. 2]. On Game of 24 with GPT-4, the breadth-first version with b = 5 reaches 74% against 4.0% for chain-of-thought and 9.0% for 100-sample chain-of-thought voting [arxiv:2305.10601 tab. 2]. On Mini Crosswords, depth-first search with a prune rule that rejects a state when any remaining clue looks impossible raises the word-level success from 15.6% (chain-of-thought) to 60% [arxiv:2305.10601 tab. 3]. Card: [Tree of Thoughts](/papers/2305.10601). The value scorer is also the weak point. On crosswords, the model sometimes marks valid but rare words as impossible and prunes a correct branch [arxiv:2305.10601 §4.3]. The output heuristic is weak too: an oracle that picks the best final state reaches 67.5% word-level success, against 60% for the model's own choice [arxiv:2305.10601 tab. 3]. ### Comparative vote over candidates When an absolute score is hard to pin down, the model can instead be shown several candidate states and asked which one is best, with the vote sampled several times. The authors use this for Creative Writing, where they sample 5 plans, vote on them 5 times with a zero-shot prompt and keep one plan [arxiv:2305.10601 §4.2]. Author-reported GPT-4 coherency scores were 6.19 for one-pass output, 6.93 for chain-of-thought and 7.56 for the tree search [arxiv:2305.10601 §4.2]. The authors chose the vote for properties such as coherence that are hard to score directly [arxiv:2305.10601 §3]. Card: [Tree of Thoughts](/papers/2305.10601). ### Pairwise tournament with debate Co-Scientist does not score each hypothesis on an absolute scale. New hypotheses enter a rating tournament at 1200 [arxiv:2502.18864 p. 34]. Pairs are chosen by similarity to the goal and by rank, and newer or top-ranked hypotheses come up more often [arxiv:2502.18864 p. 35]. Top-ranked pairs get a multi-turn scientific debate, and lower-ranked pairs a single comparison turn [arxiv:2502.18864 p. 34]. The Elo update rule and K-factor are not stated [arxiv:2502.18864 p. 34]. The ratings feed back into the next round: a meta-review writes a critique that is appended to every agent's prompt, so the system improves without fine-tuning [arxiv:2502.18864 p. 31] [p. 35–36]. Author-reported, over 203 research goals, the Elo of the best hypothesis rose from about 1,350 to about 1,610 across ten time buckets [arxiv:2502.18864 p. 7] [fig. 2a]. Card: [Accelerating scientific discovery with Co-Scientist](/papers/2502.18864). ### Multi-aspect rubric scores for a final ranking Here the model scores a finished set of candidates on several named aspects, then ranks by the average. MOOSE-Chem scores each final hypothesis on validness, novelty, significance and potential, and the rank is the mean of the four [arxiv:2410.07076 §4.2.4] [eq. 3]. The same aspects, minus potential and with clarity added, also drive the critique inside each refinement line [arxiv:2410.07076 §4.2.3]. Unlike the pairwise tournament, every candidate gets an absolute score, so the ranking needs no opponent pool. Author-reported, on a 51-paper benchmark with GPT-4o as generator and judge: the top Matched Score is 4.020, against 2.882 for the MOOSE baseline. The average is 2.564, against 2.464 [arxiv:2410.07076 tab. 10]. Removing the multi-step search drops the top score to 3.765 [arxiv:2410.07076 tab. 10]. That ablation removes the search as a whole, so it does not isolate the scoring. The Matched Score is itself judged by GPT-4o. The authors report that experts usually rate one to two points lower [arxiv:2410.07076 app. A.14], and expert-only counts are in [arxiv:2410.07076 tab. 7]. The ranking itself gets only a modest check. Drafts with two matched inspirations have an average rank ratio of 0.411, against 0.521 for drafts with none, where lower is better [arxiv:2410.07076 tab. 8]. The GPT-4o Matched Score trend across scores is not monotone, and it rests on small groups [arxiv:2410.07076 tab. 9]. Card: [MOOSE-Chem](/papers/2410.07076). ### Probability that one sampled answer is correct The model samples an answer at temperature 1, then is asked whether that answer is "(A) True" or "(B) False". The probability on True is the score [arxiv:2207.05221 §3.2] [§4.1]. Zero-shot, the score is poorly calibrated and sits near 50% for typical samples [arxiv:2207.05221 §4.1] [app. B]. Showing five samples and judging one of them, with 20-shot prompting, is the best setting the paper reports [arxiv:2207.05221 §4.2] [fig. 11]. Its numbers appear only in figures. Accuracy on the answers scored above 0.5 is higher than overall accuracy, and the gap grows with model size [arxiv:2207.05221 §4.2] [fig. 11]. Card: [Language Models (Mostly) Know What They Know](/papers/2207.05221). A second variant trains a value head to predict P(IK), the chance that the model knows the answer. The head is trained on hard labels: 30 samples per question at temperature 1, each marked right or wrong [arxiv:2207.05221 §5] [app. A.6]. On TriviaQA with a 52B model, AUROC is 0.864 [arxiv:2207.05221 tab. 1]. Trained on TriviaQA only, it transfers partly: AUROC is 0.606 on LAMBADA and 0.624 on GSM8k [arxiv:2207.05221 tab. 1]. With the other tasks added to training, AUROC is 0.853 on LAMBADA, and 0.752 on GSM8k, which is held out [arxiv:2207.05221 tab. 1]. Calibration on those other tasks is poor, and models are often underconfident there [arxiv:2207.05221 fig. 13]. Prepending a relevant Wikipedia article raised P(IK) on an obscure question from 18% to 78% at 52B [arxiv:2207.05221 §5.3]. The models are the authors' own, and no code or weights are released [arxiv:2207.05221 §1.2] [§7]. ### Verbal confidence on request Some models, asked how sure they are, answer with a class such as "Likely" or with a number. Here the rating is the output: nothing is rewritten or searched. The paper asks where that rating comes from. On Gemma 3 27B, on TriviaQA with no chain of thought, the authors find that the model forms a judgement of its answer while writing the answer, keeps it at the newline after the answer, and reads it out at the confidence token [arxiv:2603.17839 §1] [§2.2] [§2.3]. Probes at that newline explain variance in the rating beyond six summaries of answer log-probabilities [arxiv:2603.17839 §2.6] [app. D.3]. Author-reported behaviour on the same setup: expected calibration error 0.12, AUROC 0.71, accuracy 77.4% [arxiv:2603.17839 §2] [fig. 9]. Card: [How do LLMs Compute Verbal Confidence](/papers/2603.17839). This is an interpretability study. It does not propose a better way to get a confidence score, and its main evidence is a single open model. ## When to use which | Situation | Use | Why | |---|---|---| | One artifact to polish, and a critic can name concrete fixes | Iterative self-feedback and rewrite | Gains on dialogue, code readability and constrained generation [arxiv:2303.17651 tab. 1]. Feedback quality is the bottleneck [arxiv:2303.17651 §4]. | | Output is code that can be run against tests or sample inputs | Execution-grounded self-debugging | Codex on TransCoder rose from 80.4 to 92.5 with unit-test feedback and explanation [arxiv:2304.05128 tab. 2]. Each turn adds an explanation call and a feedback call, and the paper gives no token or latency figures. | | Output is code or SQL that cannot be tested, but can still be executed | Explanation of the executed result | Codex on Spider rose from 81.3 to 84.1 with an explanation, and a bare verdict did not move it [arxiv:2304.05128 tab. 2]. Without tests a bare verdict can hurt: Codex on MBPP fell from 61.4 to 57.6 [arxiv:2304.05128 tab. 3]. | | Task splits into steps, and a partial state can be judged cheaply, with early mistakes fatal | Value scoring in a tree search | Game of 24 at 74% vs 4.0% for chain-of-thought [arxiv:2305.10601 tab. 2]. Costs more: $0.74 vs $0.47 per problem against best-of-100 chain-of-thought [arxiv:2305.10601 tab. 7]. | | Candidates are hard to score on an absolute scale but easy to compare | Comparative vote | Used for coherence in creative writing [arxiv:2305.10601 §4.2]. | | Many candidates, each pair can be compared, and the ranking has to improve over rounds | Pairwise tournament with debate | Elo of the best hypothesis rose across the run [arxiv:2502.18864 fig. 2a]. Each pair costs a debate or a comparison, and the authors report no compute figures [arxiv:2502.18864 p. 8]. | | Many candidates from one generator, and a single ranked list is needed | Multi-aspect rubric scores for a final ranking | Top Matched Score 4.020 against 2.882 for MOOSE [arxiv:2410.07076 tab. 10]. The judge is the generating model family, and experts rate lower [arxiv:2410.07076 app. A.14]. Pair the ranking with an outside check before relying on it [arxiv:2410.07076 tab. 8]. | | You need a confidence number next to a black-box answer | Verbal confidence, with a check on the distribution | Gemma 3 27B: ECE 0.12, AUROC 0.71 [arxiv:2603.17839 fig. 9]. A reasoning model put about 92% of its ratings in the top class, "Almost certain" [arxiv:2603.17839 app. D.2], so look at the spread before you rely on it. | | Agent gets repeated tries in an environment or against tests, and failed attempts can be described in words | Verbal self-reflection across retries | Reflexion, GPT-4 on HumanEval Python: 91.0 vs 80.1 [arxiv:2303.11366 tab. 2]. Needs a reliable evaluator, since weak self-written tests give 0.16 false positives on MBPP Python [arxiv:2303.11366 tab. 3]. Each trial is a full episode [arxiv:2303.11366 §4.1]. | | One sampled answer needs a score, and you can sample several answers from the model | Probability that the sampled answer is correct | P(True) with five comparison samples is the reported best setting, with numbers in figures only [arxiv:2207.05221 fig. 11]. Zero-shot P(True) is poorly calibrated [arxiv:2207.05221 §4.1]. Training a P(IK) head needs labelled samples and fine-tuning, and transfers only partly to other tasks [arxiv:2207.05221 tab. 1]. | | The answer can be checked by something outside the model | An external check first; self-judgement alone adds little | Self-Refine moved GPT-4 GSM8K by 0.2 points [arxiv:2303.17651 tab. 1]. With oracle feedback, GPT-3.5 rose from 64.1% to 68.9% [arxiv:2303.17651 app. H]. | | Small or weak model, or one that does not follow output formats | Avoid self-judgement, or use a stronger model as the judge | Vicuna-13B could not produce the required feedback format or follow the refine prompts [arxiv:2303.17651 §4]. | | Tight latency or cost budget | One pass, or one refine round | Each round adds a feedback call and a refine call, and the refine prompt grows with history [arxiv:2303.17651 eq. 4]. The tree search generates 5 to 100 times more tokens than chain-of-thought, depending on prompts and search [arxiv:2305.10601 app. B.3]. | Three cautions apply to the judging variants. First, when the judge and the generator are the same model family, the judge may favour its own style. Self-Refine's preference scores come from GPT-4, which is also one of the refined models [arxiv:2303.17651 §3.2]. Co-Scientist runs its generation and judging on Gemini 2.0 variants [arxiv:2502.18864 p. 29]. MOOSE-Chem uses GPT-4o for every step, including drafting, critique, scoring and the judge [arxiv:2410.07076 §5] [§6]. Second, the generator matters more than the judge in the tree search. Mixing models showed that GPT-4 generation with GPT-3.5 evaluation reached 64% on Game of 24, while GPT-3.5 generation with GPT-4 evaluation reached 31% [arxiv:2305.10601 app. B.2]. A stronger model is better spent on generation first. Third, a self-judgement with no outside grounding can flatter the candidate. In Co-Scientist, removing the search step from Reflection let a known non-novel idea score 6.14 out of 10 for novelty, against 2.38 with search [arxiv:2502.18864 p. 38]. Reflexion's self-written tests carry the same risk in code, where they pass wrong solutions [arxiv:2303.11366 tab. 3]. The Elo ratings come from the system's own judgements, not from ground truth [arxiv:2502.18864 p. 10]. ## Implementation sketch Both decision variants share a skeleton: produce candidates with the model, let the model judge them, then keep, prune or rewrite. The first block is the Self-Refine loop, the second is one breadth-first step of the tree search. Self-Debugging is close to the first block, with a run of the program inserted before the feedback call [arxiv:2304.05128 §3]. Reflexion's retry loop [arxiv:2303.11366 alg. 1] has the same shape, but it keeps a window of past reflections instead of the full history. Pseudocode is a sketch in the style of the cards, not a copy of either paper's code. ```pseudocode # Self-feedback loop [arxiv:2303.17651 alg. 1] # Input: task input x, model M (text in, text out), few-shot prompts, # stop(feedback, t) -> bool # Output: rewritten output def refine_loop(x, M, p_gen, p_fb, p_refine, stop, max_iters=4): # Draft from input-output examples [eq. 1] output = M(p_gen + x) history = [] for t in range(max_iters): # Same model critiques its latest output [eq. 2] feedback = M(p_fb + x + output) # Stop on a fixed count or a signal in the feedback [§3.1] if stop(feedback, t): break history.append((output, feedback)) # Rewrite using every earlier output and feedback [eq. 4] refine_prompt = p_refine + x for old_output, old_feedback in history: refine_prompt += old_output + old_feedback output = M(refine_prompt) return output ``` ```pseudocode # One breadth-first step of tree search [arxiv:2305.10601 alg. 1] [§3] # Input: kept states (lists of thoughts), model M, proposal count k, # value function value(M, state) -> number, breadth b # Output: the b states to keep for the next step def bfs_step(states, M, propose, value, k, b): # Propose k candidate next thoughts for each kept state [§3] candidates = [] for state in states: for thought in propose(M, state, k): candidates.append(state + [thought]) # Score every candidate; value() may sample several times [§4.1] scored = [(value(M, c), c) for c in candidates] scored.sort(key=lambda pair: pair[0], reverse=True) # Keep the b best states [alg. 1] return [c for _, c in scored[:b]] ``` ## Contradictions Self-Refine and Tree of Thoughts report self-evaluation results on math, and the effects are small and not directly comparable. Self-Refine moves GPT-4 GSM8K from 92.9% to 93.1% and ChatGPT from 74.8% to 75.0% [arxiv:2303.17651 tab. 1]. The tree-search paper reports zero-shot tree search on GSM8K at 90, against 86 for chain-of-thought, on 100 random test questions [arxiv:2305.10601 tab. 4]. The baselines differ: Self-Refine's math starts from program-style prompts [arxiv:2303.17651 app. S], while the tree-search baseline is a text chain-of-thought. The papers do not resolve whether the gap is the method or the baseline. Whether self-ranking tracks quality is also reported differently. Co-Scientist reports that the Elo of its best hypothesis rose over a run [arxiv:2502.18864 fig. 2a]. MOOSE-Chem's own check finds that its GPT-4o Matched Score trend is not monotone [arxiv:2410.07076 tab. 9]. The two measures differ, a tournament rating and an absolute score, and the two papers use different benchmarks, so the results do not settle whether self-ranking tracks quality. Whether self-judgement adds anything without an outside signal is also reported differently. Self-Refine finds that its feedback adds little on GSM8K, and that the gain comes back only with an external correctness signal [arxiv:2303.17651 app. H]. Self-Debugging reports gains from explanation alone on text-to-SQL, where the model judges its own explanation of the executed query, with Codex at +2.8 points [arxiv:2304.05128 tab. 2] [§5.1]. On TransCoder without unit-test execution, Codex rose from 80.4 to 83.9 with an explanation [arxiv:2304.05128 tab. 3]. These are code and SQL tasks, not math, and the papers do not compare them. Whether self-judgement works on smaller models is also reported differently. Kadavath et al. report that the accuracy gap on answers scored above 0.5 grows with model size [arxiv:2207.05221 §4.2] [fig. 11]. Reflexion reports no change for Starchat-beta on HumanEval Python, at 0.26 for both baseline and Reflexion, and concludes that self-correction is an ability of larger models [arxiv:2303.11366 app. A]. These use different tasks and models, and neither paper tests the other's setup. Where the bottleneck lies is also reported differently. Self-Refine attributes most failures to feedback [arxiv:2303.17651 §4], whereas the tree-search model-mixing experiment points at generation [arxiv:2305.10601 app. B.2]. These are different tasks and setups, and both are author-reported. ## Papers - [Self-Refine: Iterative Refinement with Self-Feedback](/papers/2303.17651): one model drafts, critiques and rewrites in a loop, with no training. - [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](/papers/2305.10601): the same model proposes steps, scores or votes on partial solutions, and a classic search keeps the promising ones. - [Teaching Large Language Models to Self-Debug](/papers/2304.05128): a frozen model runs its code, explains it and fixes it, for up to 10 turns. - [Accelerating scientific discovery with Co-Scientist](/papers/2502.18864): a Gemini 2.0 multi-agent system that generates, reviews and ranks hypotheses in an Elo tournament. - [MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses](/papers/2410.07076): a GPT-4o pipeline that screens inspirations, evolves drafts, and ranks them by four self-scores. - [How do LLMs Compute Verbal Confidence](/papers/2603.17839): an interpretability study of where a verbal confidence rating is computed in one open model. - [Language Models (Mostly) Know What They Know](/papers/2207.05221): a model scores its own sampled answers with P(True), and a trained head predicts whether it knows the answer. - [Reflexion: Language Agents with Verbal Reinforcement Learning](/papers/2303.11366): an agent writes a verbal reflection after each failed attempt and keeps a window of them for the next try.