A Survey on Evaluation of Large Language Models

In plain words

Large language models (LLMs) are now used for many kinds of tasks, so knowing how well they work, and where they fail, matters for research and for daily use. This survey gathers the ways researchers have tested them. It organises the work around three questions: what to test (the tasks, such as language understanding, reasoning, ethics, medicine, science and agent use), where to test (the datasets and benchmarks), and how to test (automatic scoring and human judging).

The authors also summarise the tasks where LLMs do well and where they fail, and they list open problems. Their main message is that evaluation should be treated as a discipline of its own. They do not find one benchmark or evaluation method that is clearly the best, so no single score settles the question.

TL;DR

• A survey, not a method: it classifies LLM evaluation along three questions, what to evaluate, where to evaluate and how to evaluate §1 fig. 1.
• It covers tasks from NLP to medicine, science, agents and other applications §3; it compiles 46 benchmarks in three groups: general, downstream and multimodal §4.
• Automatic metrics (accuracy, calibration, fairness, robustness) and human evaluation (six criteria) are described, with formulas for several metrics §5.
• Its conclusions are tied to the cited datasets, and it names seven grand challenges, including dynamic benchmarks against training-data contamination §6 §7.

Problem

Existing evaluation protocols rest mainly on static test sets. The authors argue that they are not enough for LLMs: their capabilities change over time, they are often trained on data that overlaps with public benchmarks, and their interpretability is poor §1 §2.2 §7.4. They also say that evaluation must cover societal risk as well as task accuracy §1 §6.2.

Standard AI model evaluation uses k-fold cross-validation, holdout validation, leave-one-out cross-validation, bootstrap and a reduced-set split §2.2. Deep learning models have usually been checked on static validation sets such as ImageNet, MS COCO, GLUE and SuperGLUE §2.2.

Method

This is a taxonomy, not an algorithm. The survey's structure is:

What to evaluate §3.

• Natural language processing §3.1: understanding, including sentiment, text classification and natural language inference §3.1.1; reasoning §3.1.2; generation §3.1.3; multilingual tasks §3.1.4; factuality §3.1.5. A summary table is given §3.1.
• Robustness, ethics and bias, trustworthiness §3.2 (subsections for robustness §3.2.1, ethics and bias §3.2.2 and trustworthiness §3.2.3).
• Social science §3.3; natural science and engineering, split into mathematics, general science and engineering §3.4.
• Medical applications, split into medical queries, medical examinations and medical assistants §3.5.
• Agent applications §3.6; other applications, including education, search and recommendation, personality testing and specific applications §3.7.

Where to evaluate §4. The paper states that it discusses benchmarks rather than single datasets §4.

• General benchmarks §4.1, such as Chatbot Arena, MT-Bench, HELM, BIG-bench, MMLU, AlpacaEval, AGIEval, the OpenLLM leaderboard, GLUE-X, PromptBench and PandaLM.
• Downstream benchmarks §4.2, covering question answering, dialogue, safety, Chinese-language exams, mathematics, code, legal contracts, tool use and others.
• Multimodal benchmarks §4.3, such as MME, MMBench and SEED-Bench.

How to evaluate §5. The split is whether the criterion can be computed automatically. If it can, it is automatic evaluation; otherwise it is human evaluation §5.

• Automatic evaluation §5.1 uses four metric families: accuracy, calibration, fairness and robustness. The metric definitions as printed are in the table below. They are written in the survey's notation and are not normalised or checked against the original sources.

• Human evaluation §5.2 judges generated text, and the survey lists six criteria: accuracy, relevance, fluency, transparency, safety and human alignment. These extend the 3H rule (helpful, honest, harmless) §5.2. The survey also lists factors to set in a study: number of evaluators, criteria and evaluator expertise §5.2 [table of key factors, §5.2].
• New protocols §5 include human-in-the-loop testing, crowd-sourced dynamic test sets, and challenge sets built from templates or adversarial filters. The survey gives a summary table of these protocols §5.

The survey's split between the two families, as a decision rule:

# Choose the evaluation family for a criterion  [§5]
# Input:  criterion (a property to score in model outputs)
# Output: "automatic" or "human"

def evaluation_family(criterion):
    # Automatic if a standard metric can compute the criterion  [§5]
    if can_compute_automatically(criterion):
        return "automatic"
    # Otherwise people judge the outputs; see the criteria in [§5.2]
    return "human"

Hyperparameters

Not applicable. The survey proposes no method, model or training run, so it has no hyperparameters. The only settings the paper names are the standard protocol choices (k-fold, holdout, leave-one-out, bootstrap, reduced set) §2.2. It does not give their parameter values.

Results

Author-reported. The survey has no experiments of its own. Its headline counts and findings are:

The survey lists seven grand challenges: designing AGI benchmarks §7.1, complete behavioural evaluation §7.2, robustness evaluation §7.3, dynamic and evolving evaluation §7.4, principled and trustworthy evaluation §7.5, unified evaluation across tasks §7.6, and using evaluation to improve LLMs §7.7.

Paper vs code

The paper links an open repository in its abstract and contributions: https://github.com/MLGroupJLU/LLM-eval-survey [Abstract] §1. The repository's README lists the same authors, so it belongs to them README.md. It is a curated list of papers organised by the survey's taxonomy, not an implementation. The commit recorded in the frontmatter is the one cloned for this card.

• The README says it may be updated after the arXiv version, and that it may miss some benchmarks README.md README.md. Its counts and entries can therefore differ from the 46 benchmarks in the arXiv version, so cite the paper for the numbers.
• The repository has no LICENSE file, so its text and lists are not licensed for reuse here. Nothing from it is quoted or copied into this card.
• The arXiv version is v9 (updated 2023-12-29) [raw/arxiv/2307.03109.json]. The README's news list records the arXiv releases as well, so the repository has changed since v1 README.md.

Pitfalls

• Not an implementable method. An agent cannot reproduce anything from this card. To implement a metric, take its definition and normalisation from the original source of that metric, not from this survey (see the ASR note below).
• Snapshot of 2023. The literature window ends in July 2023 fig. 2. The models named in the text, such as ChatGPT, GPT-4 and Claude 1 and 2 §5.1 §5.2, are the ones the cited work used. The authors say online services change and that some limitations may be fixed [Disclaimer].
• Conclusions depend on datasets. The authors say their success and failure conclusions are drawn from existing evaluations and "only depend on specific datasets" §6.1. Treat them as a summary of reported results, not as a general law.
• No single best benchmark. The survey's own conclusion is that no evaluation protocol or benchmark is shown to be best §6. Do not select a benchmark for an agent from this card alone.
• ASR formula is ambiguous as printed. The sum is over samples, but each term is a ratio of two indicators, the denominator being \mathcal{I}[f(x)=y]. For a sample the model already gets wrong, that denominator is zero. The survey does not say how such samples are handled or how the sum is normalised. Use the definition in the attack paper that §5.1 cites.
• PDR depends on the evaluation function. PDR is a ratio of two sums, so the normalisation cancels. But \mathcal{M} must be defined per task, and the survey does not give those definitions §5.1.
• F1 is defined only for binary classification §5.1. The survey uses EM and ROUGE for generation tasks, but gives no rule for choosing between them.
• RLHF is described loosely. Section 2.1 says fine-tuning uses human-generated responses as rewards §2.1. The usual method trains a reward model from human preference comparisons. The survey does not say this. Check the original RLHF papers before implementing.
• Human evaluation has no protocol. The six criteria and the study factors are considerations, not a procedure §5.2. The survey gives no rater count, scale or agreement threshold. It notes that human judgments can vary with culture and individual differences §5.2.
• Claims are inherited. Many findings are the cited works' claims, for example GPT-4 performing near or above human level on some tasks, which §5.2 attributes to the work of Bubeck et al. The survey does not re-run them.
• Coverage is partial. The authors say they may have missed benchmarks and invite corrections §4 footnote [Disclaimer].
• Figures carry the trend. The publication trend fig. 2 and the evaluation process fig. 3 are figures, and the paper gives no numbers for them in the text. Describe them only as the figures show.
• Tables are not reproduced. The survey's large tables (NLP tasks, robustness, medical, science, other applications, benchmarks, protocols) are summaries of cited works. Use the paper's tables and the cited works for any specific number.

Lineage

• Builds on: standard AI evaluation protocols and static benchmarks such as GLUE, SuperGLUE, ImageNet and MS COCO §2.2. These are not compiled here, so none are linked.
• Compared against: the survey positions itself as the first comprehensive survey on LLM evaluation §1. It names no predecessor survey.
• Superseded by: not known from this wiki.