A Survey on LLM-as-a-Judge

In plain words

Using a language model to grade another model's answers, a person's writing or a piece of text is called LLM-as-a-Judge. It is attractive because it is fast, cheap and can be scaled up, and it can often approximate what a human grader would say. The trouble is that the grades are not stable. They can change with the wording of the instructions, the order in which two answers are shown, the length of an answer, and the randomness built into the model.

This survey organises the work on building such judges. It covers how the judge is set up (the prompt, which model to use, how its raw output is turned into a final grade), how to make it more consistent (asking several times and voting, fine-tuning on grading data, refining from feedback), how to test it against human judgments and for biases, and where it is already used. It also runs one small experiment comparing several judge models and simple improvement strategies.

TL;DR

The paper defines LLM-as-a-Judge as an LLM producing an evaluation from an input and a context, and sorts the practice into four parts: in-context learning, model selection, post-processing and pipeline design §2. In its experiment, GPT-4-turbo showed the fewest biases among the six models tested, and majority voting over five rounds plus swapping answer positions were the strategies that helped most §4.4. All numbers are author-reported on a single small setup, so treat them as directional.

Problem

Expert and human evaluation is accurate but costly and slow to scale, and automatic metrics such as BLEU or ROUGE depend on surface overlap §1. An LLM judge can combine scale with context-sensitive reasoning, but the paper argues that its reliability is not yet systematically understood: definitions are informal, the literature is fragmented, and usage is inconsistent §1. The paper's guiding question is how to build reliable judges §1.

The paper assumes a judge is evaluated against human labels, for example by percentage agreement, and against bias tests such as position, length and self-preference effects §4.1–4.2.

Method

This is a survey, so there is no single algorithm. The paper's core is a formal definition and a taxonomy §2, followed by reliability strategies §3 and a meta-evaluation of some of them §4.4.

Definition §2. The final evaluation E is produced from the judge's probability function P over its output, applied to the input x combined with a context C (prompt template, possibly dialogue history). The combination operator ⊕ can place x at the beginning, middle or end of C §2. The formal definition is an unnumbered display equation; cite it as §2.

Components §2.

1. In-context learning: input design (modality, single item, pair or batch, position) and prompt design. Four prompt forms are described: score generation, yes/no questions, pairwise comparison and multiple choice §2.1.
2. Model selection: a general LLM or a fine-tuned judge §2.2.
3. Post-processing: extract specific tokens, normalise output logits, or select sentences from the output §2.3.
4. Evaluation pipeline: judging models, judging data (annotation), judging agents and judging reasoning §2.4.

Yes/No score from logits §2.3, "Normalizing the output logits". The self-consistency score multiplies the probabilities of the tokens in two sets, α and β, over the prompt and the answer; the self-reflection score multiplies the probabilities of the token "Yes" after the prompt; the final score is their product. The paper's sets α and β are not defined in the text. The equations are unnumbered; cite as §2.3.

Improvement strategies §3. Three groups:

• Prompt design: improving task understanding and standardising the output format §3.1.
• Capability enhancement: specialised fine-tuning on meta-evaluation data, built either from evaluation templates or by transforming raw data §3.2; and feedback-driven refinement, which uses stronger-model or human corrections for fine-tuning or as few-shot examples, and online RL with rule-based rewards §3.2.
• Final output optimisation: integrate several rounds or several judges (voting, averaging, cascades, peer review) §3.3; or optimise a single judge's output by score smoothing with token probabilities or by self-verification §3.3.

Multi-round voting, the strategy tested in §4.4. The judge is run five times on each sample; the final label is taken by majority vote, by the mean score, or by the best score [§4.4, "Model Configuration" and "Comparison with Different Strategies"]. The paper runs these at temperature 0.

# Multi-round judging  [§4.4 "Summarize by Multiple Rounds"]
# Input:  item x, judge model, rounds (5 in the paper)
# Output: one final label

def multi_round_judge(x, judge, rounds):
    # 1. Run the same judge several times on the same item  [§4.4]
    verdicts = []
    for round_index in range(rounds):
        verdicts.append(judge(x))
    # 2. Aggregate: majority vote (best), mean or best score [§4.4]
    return majority_vote(verdicts)

Position consistency §4.4, unnumbered formula. Each (question, response 1, response 2) triple is judged in both orders. The metric is the fraction of triples where the two verdicts match. The paper's formula compares the labels r1, r2 or TIE literally, which only works if each verdict is mapped back to the response it names. Our reading, mapping the verdicts back:

# Position consistency  [§4.4]
# Input:  triples (question, response_a, response_b)
# Output: fraction of triples with the same winner in both orders

def position_consistency(triples, judge):
    same = 0
    for question, response_a, response_b in triples:
        # 1. Judge with the original order  [§4.4]
        first = judge(question, response_a, response_b)
        # 2. Judge with the responses swapped  [§4.4]
        swapped = judge(question, response_b, response_a)
        # 3. Map the swapped verdict back to the original responses  [§4.4]
        if first == swap_label(swapped):
            same += 1
    return same / len(triples)

Hyperparameters

Results

Author-reported. Setup: LLMEval² (human preference, pairwise) and EvalBiasBench (six bias types plus position) §4.4. Each cell is a percentage. The "n" in each header is the number of meta-evaluation samples for that column, as printed in the paper's tables.

Judge models §4.4, "Comparison with Different LLMs":

Strategies on GPT-3.5-turbo §4.4, "Comparison with Different Strategies":

Reasoning judges on LLMEval² by human label §4.4, detail table. Accuracy = aligned / total, per column:

The paper's own summary: GPT-4 showed fewer biases than the others; Qwen2.5-7B was the best open model; and LLM choice made little difference to human alignment. Reasoning models were competitive but not consistently better §4.4.

Paper vs code

No official code.

Searched: the paper's project page (awesome-llm-as-a-judge.github.io) links one GitHub repository, IDEA-FinAI/LLM-as-a-Judge. Its top level holds only a README and an images folder: a paper list, with no code, no experiment scripts and no license. The meta-evaluation tool drawn in [§4.4, the experiment pipeline figure] is not released. The paper names no method for a GitHub search, so no search was run beyond the project page.

Because the paper has no official code, the discrepancies below are between paper text and paper tables, not between paper and code.

• Text says the alignment sample is LLMEval², 2,553 samples §4.4; table headers give n = 5106 for alignment §4.4 tables. The paper does not explain the difference. Our check: GPT-4-turbo's 61.54% equals 3142/5106, the sum of aligned counts over the three human-label columns in the detail table (1418 + 1438 + 286), so the 5106 figure seems to count each sample twice (once per swap) — our inference, not stated.
• Text describes EvalBiasBench as 80 samples §4.4; per-type n in the table headers sums to 180 (34 + 28 + 26 + 24 + 24 + 24). Not explained.

Pitfalls

• Small bias columns. Most bias columns have 24–34 samples, so one sample is about 3–4 percentage points (1/24 = 4.17). Differences of a few points between strategies are within noise. Position bias (n = 2633) is the only well-powered bias column.
• Temperature 0 with multiple rounds. The paper sets temperature 0 §4.4, yet majority@5, mean@5 and best-of-5 need diverse samples to differ. How the five rounds vary is not stated.
• The majority@5 position value. In the source table the value is split across two lines ("70" and "11"); we read it as 70.11. Check against the published PDF before relying on it.
• "Improvements across multiple dimensions" overstates majority@5. Relative to base, majority@5 lowers the nested-instruction (58.33 → 54.17) and familiar-knowledge (54.17 → 50.00) columns §4.4 strategies table.
• Pairwise vs pointwise claim. The text says the majority@5 vs mean@5 comparison shows pairwise judging is more reliable than pointwise §4.4. The table compares aggregation rules, not pairwise and pointwise judging, so the claim is not isolated by the data shown.
• Explanations and self-validation. Asking for an explanation at the same time as the score lowered position consistency and alignment in this setup §4.4. The paper suggests this may come from the explanation adding bias §4.4; the prompt text is not given, so the effect cannot be reproduced exactly.
• Position consistency formula. The paper compares the labels r1, r2 and TIE literally §4.4. Unless verdicts are mapped back to the response they name, the metric measures label agreement, not winner agreement. See the pseudocode above.
• Yes/No logit score. The α and β token sets in the self-consistency score are not defined §2.3, so the score cannot be implemented exactly from the text.
• Fine-tuned judges. The survey notes that fine-tuned judges can inherit biases from their training data and generalise poorly beyond it §3.2. The paper gives no fine-tuning hyperparameters for any system it reviews.
• Cost. Multi-round and multi-model voting multiplies calls per item (five rounds, three models per vote set) §4.4. Cost per item is not reported.
• "Novel benchmark" claim. The introduction says the paper proposes a meta-evaluation benchmark §1. The experiment uses existing datasets, LLMEval² and EvalBiasBench §4.4, and the tool is not released.

Lineage

• Builds on: none stated; the paper is a survey of prior work.
• Compared against: the four strategies in §4.4 are compared with each other and with the six base models, not with other published judges.
• Superseded by: not known from content in this wiki.