Can Large Language Models Be an Alternative to Human Evaluations?

In plain words

Checking whether a text is good usually means asking people to rate it on a scale. Human ratings are hard to reproduce and vary between groups of raters. This paper asks whether a large language model can stand in for those raters.

The idea is simple. The model gets the same instructions, text and questions that the human raters got, and it writes a free-text answer. A short rule then pulls the rating number out of that answer. The authors test this on two jobs: judging short stories written by people versus by a GPT-2 model, and judging sentences changed by word-swapping attacks.

Some models give the same overall verdicts as expert raters, and the verdicts hold up when the instructions are reworded or the sampling setting changes. Other models do not clearly separate the two kinds of text. The authors present LLM rating as a cheap, quick first judgment, with human evaluation still used alongside it.

TL;DR

• Task: rate text quality on four story attributes and two adversarial-text attributes, using the same questions and a 5-point Likert scale as human raters §2.2 §3.2 §4.2.
• Method: free-text LLM answers, parsed to a number; teachers as the reference; Welch's t-test and Kendall's tau for comparison §3.3 §3.3.1.
• Headline (author-reported): text-davinci-003 and ChatGPT rate human-written stories above GPT-2 stories on all four attributes, as three certified English teachers do. The paper finds no clear preference from T0 or text-curie-001 tab. 1 §3.3.
• Robustness: changing the instruction (persona, "explain") or sampling temperature shifts absolute scores a little but keeps the human-over-GPT-2 ordering tab. 3.
• Cost: the paper reports US$140 per teacher for 200 stories, against under US$5 for the best InstructGPT model on the same task §5.

Problem

Human evaluation of generated text is central to NLP but hard to reproduce and unstable across rater pools §1. In open-ended story generation, workers on Amazon Mechanical Turk reportedly cannot tell GPT-2 stories from human ones, so the authors use certified English teachers as the reference §3.1 §3.2. Adversarial text generated by synonym-substitution attacks often reads badly and changes meaning, and it has to be checked by people §4.1. The question is whether LLMs, which follow instructions without task-specific training, can serve as the rater §2.1.

Method

Overview. Each rating is one prompt built from three parts: the task instruction, the text being judged, and one question with a 5-point scale. The model answers in free text. A parser takes the first number from that answer as the score §2.2 app. D.2.1. For story generation, the four attributes are grammaticality, cohesiveness, likability and relevance, each with its own question §3.2 app. D.2.2. For adversarial text, the two questions are fluency of one title and meaning preservation between a benign title and an attacked title §4.2 app. D.2.3.

Procedure.

1. Keep the instruction, text and question identical to the human protocol, apart from formatting §2.2 §3.2.
2. Ask one question per attribute. For the story task, T0 and InstructGPT get each attribute in a separate query, while ChatGPT gets all four in one conversation, in the same order app. D.2.2.
3. Sample the answer. T0 and InstructGPT use nucleus sampling with p = 0.9, and three answers are drawn per question to act as three raters §3.2. ChatGPT gives one answer, since its interface offers no sampling control §3.2.
4. Parse each answer to a score app. D.2.1.
5. Summarise per attribute: mean, standard deviation, Krippendorff's α across the three samples, and the share of items where all three agree exactly tab. 1 (caption).
6. Compare writers (human vs GPT-2) with Welch's t-test on the mean ratings §3.3.
7. Compare individual stories: for each story, average the three teacher ratings and the three LLM samples, then compute Kendall's tau across the 200 stories §3.3.1.
8. Probe robustness: prepend a persona sentence, or ask the model to explain its rating §3.3.2; vary temperature T over {1, 0.7, 0.3, 0} with p fixed at 0.9 §3.3.3.

Parsing. Before the first number is taken, the strings "1-5", "out of 5" and "/5" are removed. For the adversarial task, "title 1" and "title 2" are also removed. Half-point scores such as 4.5 are kept app. D.2.1. ChatGPT answers were not parsed by a rule; the authors read each answer and recorded the score by hand app. D.2.1.

Story-generation data. Stories come from the WritingPrompts dataset. A GPT-2 medium model is fine-tuned on its training split, then generates 200 stories from 200 test prompts with nucleus sampling at p = 0.9 §3.1 app. B.2 app. B.3. Each generated story and each matching human story is cut to under 150 words, ending on a full sentence. Some tokenisation artefacts in the human stories are removed by hand app. B.3.

# LLM rating of one text on one question  [§2.2] [app. D.2.2]
# Input:  instruction (str), text (str), question (str), llm (callable)
# Output: list of scores (float), one per sampled answer

def llm_rating(instruction, text, question, llm, n_samples):
    # Same instruction, text and question as the human form  [§2.2]
    prompt = build_prompt(instruction, text, question)
    scores = []
    for _ in range(n_samples):
        # Free-text answer with nucleus sampling p=0.9  [§3.2]
        # Temperature is not stated; T=1.0 is implied by [tab. 3]
        answer = llm.generate(prompt, top_p=0.9, temperature=1.0)
        scores.append(parse_score(answer))
    return scores

# Turn a free-text answer into one Likert score  [app. D.2.1]
# Input:  answer (str), task ("story" or "adversarial")
# Output: score (float), or None when the answer has no number

def parse_score(answer, task):
    # Drop scale phrases the model may repeat  [app. D.2.1]
    text = answer.replace("1-5", "")
    text = text.replace("out of 5", "")
    text = text.replace("/5", "")
    # Adversarial questions name the two titles  [app. D.2.1]
    if task == "adversarial":
        text = text.replace("title 1", "")
        text = text.replace("title 2", "")
    # First number in the text; a 0.5 step is kept  [app. D.2.1]
    return first_number(text)

# Story-level agreement between teachers and one LLM  [§3.3.1]
# Input:  teacher_ratings (200, 3), llm_ratings (200, 3)
# Output: Kendall's tau (float)

def story_agreement(teacher_ratings, llm_ratings):
    # Average the three teachers on each story  [§3.3.1]
    teacher_means = teacher_ratings.mean(axis=1)  # (200,)
    # Average the three LLM samples on each story  [§3.3.1]
    llm_means = llm_ratings.mean(axis=1)  # (200,)
    return kendall_tau(teacher_means, llm_means)

Equations. The paper has no numbered equations. Kendall's tau, Welch's t-test and Krippendorff's α are named and used but not written out. The sections include inline notation for the vectors A and B in §3.3.1.

Adversarial data. The attacked titles come from a released dataset of attacks against a BERT-base-uncased classifier fine-tuned on AG-News app. E §4.2. For each of three attacks (TextFooler, PWWS, BAE), 100 benign/adversarial pairs are drawn at random §4.2.

Hyperparameters

Results

Author-reported. Setup: story generation, 200 human-written and 200 GPT-2-generated stories, each rated by three English teachers and by each LLM §3.2. Adversarial task: 100 pairs per attack, rated by three teachers and by ChatGPT §4.2.

Full table, story generation tab. 1. Cells give mean (SD) and, after the semicolon, Krippendorff's α and the share of items with exact agreement. The source prints one value as "0.64" in two cells; see Pitfalls.

Gap between human-written and GPT-2 stories (mean difference, computed from tab. 1; positive means human-written rated higher):

Paper's significance statements: T0's grammaticality and relevance gaps are not significant by Welch's t-test; text-davinci-003's gaps on all four attributes are significant §3.3.

Story-level agreement (Kendall's tau between teacher averages and text-davinci-003 averages) tab. 2:

The paper reports all of these as positive with p < 0.05 §3.3.1. The test used for the p-values is not stated.

Robustness (text-davinci-003, mean (SD) per cell) tab. 3:

The persona and explain rows test two instruction variants; temperature varies one parameter at a time, with p fixed at 0.9 §3.3.2 §3.3.3. The paper also reports, without a table, that agreement rises as temperature falls §3.3.3.

Teacher-to-teacher agreement (Kendall's tau, averaged over the three pairs) tab. 6:

Human ratings with mixed sources (teachers rated a shuffled set of human and GPT-2 stories) tab. 5:

Adversarial-text task tab. 4. Teachers and ChatGPT, mean Likert score. Fluency is for one title; meaning is for the benign/attacked pair.

Same task with text-davinci-003 tab. 11:

Per-teacher fluency tab. 12:

Tables 7 to 10 show ratings for four example stories and are not reproduced here. Figures are described in words and not included: the illustration of the method fig. 1, the two halves of the story rating form fig. 2 fig. 3, and the two adversarial rating forms fig. 4 fig. 5.

Paper vs code

The official repository is d223302/LLM-Evaluation, owned by the first author, Cheng-Han Chiang, as shown by the GitHub profile name. The paper does not link it, so this is an owner match only §1. Its commit is bcf03aecd619bbae5f2ab8d7f074502e98640cf5. The repository holds result files and a readme, not the prompting, parsing or statistics code readme.md.

Differences and gaps between paper and repository:

• The human file is the mixed-source experiment, not the Table 1 data. The paper's Table 1 uses teachers who rated human and GPT-2 stories separately §3.2 app. C.1. The released human_eval_result.json holds the mixed design, and it has 200 human-written entries human_eval_result.json. The readme says its numbers "may be slightly different" from the paper because some ratings were missing and re-collected readme.md. Compare it to tab. 5, not tab. 1.
• Only text-davinci-003 and text-curie-001 outputs are released. The files are davinci, davinci-explain, davinci-persona and curie, all named temp-1 and top_p-0.9 davinci-temp-1-top_p-0.9.json readme.md. There are no released outputs for T0 or ChatGPT, and the readme says ChatGPT results were not released because the UI was used readme.md. Table 1 rows for T0 and ChatGPT cannot be checked against the repository.
• Parsing is described in the paper but not published. The repository has no parser. Some stored answers still contain raw text, for example "4. The story fragment is grammatically correct, with only minor errors.", and the first-number rule in app. D.2.1 gives 4 for it davinci-temp-1-top_p-0.9.json. Implementers must write the parser themselves.
• Prompt and sampling settings are only in the paper. The prompt templates appear only in app. D.2.2 and app. D.2.3. Their wording is in the Method section above. The repository contains no prompt text.
• Query dates. The readme says the released outputs were queried in January 2023 readme.md. The paper gives the ChatGPT versions as Dec. 15 and Jan. 9 app. D.1 and no date for the API calls.
• Version. The readme cites the ACL 2023 version readme.md. This card follows the arXiv v1 LaTeX source, so section and table numbers may differ from the published version.
• Licence. The readme states Apache-2.0 for the data readme.md, but there is no LICENSE file and GitHub reports no licence. Nothing from the repository is quoted, only cited.
• Not checked. The means and Kendall's tau values were not recomputed from the released JSON files in this session, so the agreement of those numbers with the paper is unverified.

Pitfalls

• Judge choice decides the result. The paper reports a clear human-over-GPT-2 preference only for text-davinci-003 and ChatGPT, on all four attributes. For T0 and text-curie-001 it reports no clear preference, and T0's grammaticality and relevance gaps are not significant tab. 1 §3.3. Validate a judge on your own human-labelled set before relying on it.
• Absolute scores depend on the judge. text-davinci-003 scores higher than teachers, and ChatGPT scores lower. Only compare systems rated by the same judge §3.3.
• Free-text parsing needs care. The rule takes the first number after removing "1-5", "out of 5" and "/5". Answers with no number give no score; the paper does not say how such cases were handled beyond excluding refused items app. D.2.1. Half-point scores are kept, so a scale that is strictly integer will differ app. D.2.1.
• ChatGPT is not reproducible as published. It was queried through the web UI with no seed or sampling control §3.2 app. D.1. Refusals on likability were regenerated "until it gives a rating" §3.3, which selects which answer is kept. Scores were read by hand app. D.2.1. Two stories and three titles were dropped for content-policy refusals app. D.2.1.
• Sampling choices are narrow. Temperature was varied with p fixed at 0.9, and temperature and p were not varied together §3.3.3. The paper says the instruction variants are only two, and that the results may not hold for other changes [Limitations].
• Table 1 typo. Two cells for text-curie-001 print the Krippendorff's α share as "0.64" (likability and relevance). This is probably a typo for 64, but the source is ambiguous and the card reproduces it as printed.
• Text and table disagree in one place. The paper says ChatGPT rates adversarial samples higher than teachers §4.3. That is not true for TextFooler fluency, where ChatGPT gives 2.12 and teachers give 2.17 tab. 4.
• Sanity checks differ by model. Identical benign pairs scored 5.00 with ChatGPT tab. 4 and 4.56 with text-davinci-003 tab. 11. The paper uses the check to argue ChatGPT understands the task §4.3.
• Human baseline is noisy. Teacher agreement is low on likability (α 0.08 for human-written, -0.21 for GPT-2) and on grammaticality between teachers (tau 0.25 and 0.15) tab. 1 tab. 6. A small LLM gap can sit inside this noise.
• Manual steps in data preparation. Stories were truncated and artefacts removed by hand app. B.3, and the adversarial attacks come from a released dataset rather than being rerun app. E. Rebuilding the exact data needs those manual steps.
• Costs are dated. The US$140 and under US$5 figures §5 are 2023 prices and a single run, with the teacher pay stated in app. C.1.
• Numerical sensitivity. Mean differences on the 1-5 scale are small in several cells (for example, T0 cohesiveness -0.04 and text-curie-001 relevance 0.05 tab. 1). Use several samples and a significance test before reading a gap as real.
• Statistical details missing. The tests of Kendall's tau p-values, any multiple-comparison correction, and the Welch's t-test assumptions are not stated §3.3.1.

Lineage

• Builds on: zero-shot instruction following of T0, FLAN, InstructGPT and ChatGPT §2.1. The human-evaluation protocol follows prior work on story generation with English teachers, which the paper cites as Karpinska et al. (2021); no arXiv id is given in the source §3.2.
• Compared against: human ratings by three certified English teachers §3.2 §4.2. The paper cites prior work on crowd raters (Mechanical Turk) as a weaker baseline §3.1.
• Superseded by: not stated in this wiki.