Is ChatGPT a General-Purpose Natural Language Processing Task Solver?

In plain words

Before ChatGPT, a chatbot-style model was judged mostly on conversation. This paper asks a plainer question: if you give ChatGPT a task in plain English and no examples, how well does it do across many kinds of language work? The authors test it on 20 standard datasets. The tasks range from arithmetic word problems and yes/no reading questions to summarizing chats, finding names of people and places in text, and labelling movie reviews.

The method is deliberately simple. Each test item is sent with a one-line instruction that says what to do and what the answer must look like. For harder reasoning questions, the model is first asked to think step by step, and its written reasoning is then fed back with a short final question that pulls out the answer. The same setup is run on the older GPT-3.5 model, so the two can be compared directly.

The results are mixed. ChatGPT is strong at arithmetic word problems, at deciding whether one sentence follows from another, and at dialogue, and it often beats the older model there. It is weaker at summarization, at tagging names and places in text, and at several commonsense tasks, and it remains far behind models that were trained on each task. The paper's value is as a map of where a general-purpose chat model was reliable in early 2023 and where it was not.

TL;DR

• Zero-shot study of ChatGPT (gpt-3.5-turbo) against text-davinci-003 (GPT-3.5) on 20 NLP datasets in 7 task categories §1, tab. 1.
• Each item is one prompt: a task instruction plus the test input, with no demonstrations §3.
• For reasoning tasks, a two-stage zero-shot chain-of-thought prompt is used §3.
• ChatGPT beats GPT-3.5 on most arithmetic, NLI, QA, dialogue and sentiment sets but trails on summarization, NER and several commonsense sets §4.2, tab. 11.
• Author-reported. Most results sit far below the fine-tuned or few-shot baselines. No code is released §4.3.

Problem

Large language models were known to solve many NLP tasks zero-shot, that is without task-specific training data, when given a suitable prompt §1. The paper states that existing models still make many mistakes in zero-shot use, and that prompt wording can change results substantially, for example adding "Let's think step by step" improves reasoning accuracy of InstructGPT §1. The authors' question is whether ChatGPT, which was trained with RLHF, is a generalist that works zero-shot across tasks, and where it fails §1. The paper states no prior systematic zero-shot study of ChatGPT across this range of datasets §1.

Method

The paper is an empirical evaluation, not a new model. The two systems under test are gpt-3.5-turbo, which the paper calls ChatGPT, and text-davinci-003, called GPT-3.5 §3. For a task instruction P and a test input X, the model is asked to produce the output Y = f(P, X) with the two concatenated as input §3. Task instructions and input formats are shown in fig. 2 and fig. 3; the instructions are taken from or inspired by earlier work §3. The exact joining of P and X (separators, newlines) is not stated.

Two-stage zero-shot chain-of-thought is used for reasoning tasks, following Kojima et al. §3. Stage 1 uses the instruction "Let's think step by step." to elicit a rationale. Stage 2 feeds the rationale, the original input and the stage-1 instruction back to the model, followed by a trigger sentence that asks for the answer. The paper gives the trigger "Therefore, among A through E, the answer is" as an example §3. For arithmetic the answer trigger is "The answer (arabic numerals) is" fig. 3.

Evaluation covers 20 datasets in 7 categories: arithmetic, commonsense, symbolic, logical reasoning, natural language inference, question answering, dialogue, summarization, named entity recognition and sentiment §4.1, tab. 1. By default the test split is used where labels are available. COPA and CommonsenseQA use the validation split. StrategyQA uses the open-domain question-only setting from BIG-bench §4.1.

# Zero-shot prompting  [§3]
# Input:  instruction P, test input X, model f
# Output: answer Y = f(P, X)

def zero_shot_answer(instruction, test_input):
    # Concatenate task instruction and test input into one prompt  [§3]
    prompt = instruction + test_input
    # One call, no demonstrations; the joining format is not stated  [§3]
    return model(prompt)

# Two-stage zero-shot chain-of-thought  [§3]
# Input:  test input X, P1 = "Let's think step by step.",
#         P2 = answer trigger (e.g. "Therefore, ... the answer is")
# Output: answer Y

def zero_shot_cot(test_input, first_instruction, trigger):
    # Stage 1: elicit a free-text rationale R with P1  [§3]
    rationale = model(first_instruction + test_input)
    # Stage 2: rationale R, input X and P1, then trigger P2  [§3]
    # Order and separators are not stated; this order is an assumption
    answer = model(first_instruction + test_input + rationale + trigger)
    return answer

Hyperparameters

No training is done, so there are no training hyperparameters. The settings that matter for reproduction are below.

Results

Author-reported. Setup: 20 datasets in 7 categories, zero-shot for ChatGPT and GPT-3.5, with and without zero-shot chain-of-thought for reasoning tasks §4.2, tab. 2, tab. 3. Baselines are numbers reported by earlier papers and not re-run, as stated in [Limitations].

Sources: sample counts tab. 1; the two model columns and the prior column are the final comparison tab. 11. For each reasoning row, the better of zero-shot and zero-shot chain-of-thought is shown for each model tab. 11 caption. The SAMSum value is the mean of ROUGE-1, ROUGE-2 and ROUGE-L, as the table shows tab. 9: ChatGPT gives 42.4, 17.6 and 33.0 tab. 9. The table's "Fine-tuning" label does not match all of its values; see Pitfalls.

Key findings, as the paper states them §1, §4.2:

• Arithmetic: without CoT, ChatGPT beats GPT-3.5 on five of six datasets; AQuA-RAT is tied at 28.0 tab. 2. With CoT it beats GPT-3.5 on all six tab. 2.
• Commonsense, symbolic and logical: ChatGPT is below GPT-3.5 on CommonsenseQA and StrategyQA. CoT lowers ChatGPT's CommonsenseQA score (73.7 to 71.5) and StrategyQA score (61.1 to 55.5) tab. 3.
• NLI: zero-shot ChatGPT beats GPT-3.5, FLAN, T0 and PaLM on RTE and CB tab. 4.
• QA and dialogue: ChatGPT beats GPT-3.5 on BoolQ and MuTual tab. 6, tab. 8.
• Summarization: ChatGPT is below GPT-3.5 on all three ROUGE scores tab. 9.
• NER: CoNLL03 F1 is about equal, but ChatGPT and GPT-3.5 are far below fine-tuned models tab. 10.
• Sentiment: ChatGPT beats GPT-3.5 on SST-2 (93.7 vs 88.8) tab. 12.

Comparison with fine-tuned or few-shot baselines: the paper concludes ChatGPT "underperforms previous fine-tuning methods in most cases" §4.3, tab. 11.

Figures: fig. 1 plots ChatGPT, GPT-3.5 and fine-tuned models for the 20 datasets as a radar chart. fig. 4 shows an arithmetic answer where ChatGPT corrects GPT-3.5. fig. 5 shows a dialogue case where GPT-3.5 adds unsupported content. fig. 6 shows summary-length examples. fig. 7 shows NER failure cases on the Miscellaneous class.

Paper vs code

No official code.

Pitfalls

• Model snapshots. The results depend on the API snapshots used between 06/15/2023 and 06/21/2023 [Limitations]. ChatGPT is named only as gpt-3.5-turbo and GPT-3.5 only as text-davinci-003 §3. Re-running today gives different numbers, and the paper does not say how the models were accessed.
• Decoding and parsing are unstated. Temperature, top-p, maximum output length, number of samples, and how answers are extracted from free text are not reported. Any reimplementation has to choose these, and the reported accuracies depend on them.
• Off-format replies. The paper notes that ChatGPT still outputs answers outside the requested set, such as "It is unclear" for yes/no §4.2.4, or "neutral" and "mixed" for sentiment §4.2.8. It attributes part of the gap to these. How such replies are scored is not stated.
• Mixed baselines. Few-shot and fine-tuned numbers come from earlier papers, not from re-running them. The Limitations say the paper reports the best published result for models that are not public (for example PaLM) and the best prompt found for public models [Limitations]. The comparison is therefore not controlled.
• Table 11 label mismatch. The row labelled "Fine-tuning" contains few-shot prompting numbers for some tasks, not fine-tuned ones. For example, 96.2 on MultiArith is the Codex CoT value in tab. 2, and 99.4 on Last Letter is the PaLM CoT value in tab. 3. Read that row as "best prior result", not as fine-tuning.
• Prompt choice. The paper says it reports the best prompt it found for public models [Limitations], and that more diverse prompt templates could help [Limitations]. Results should be read as one prompt per task, not a tuned optimum.
• Summary length. ChatGPT's summaries average 36.6 words, against 23.3 for GPT-3.5 and 20.0 for the references §4.2.6. Asking for at most 25 words reduces the length to 22.8, but the ROUGE average falls from 31.0 to 30.6 §4.2.6. An explicit length limit in the instruction hurt in this study.
• CoT is not uniformly helpful. Chain-of-thought helps arithmetic and some symbolic tasks, but lowers ChatGPT's accuracy on CommonsenseQA and StrategyQA tab. 3.
• Small test sets. CB has 56 test items and COPA has 100 validation items tab. 1. Differences of a few points there are within noise. The paper gives no confidence intervals.
• Class imbalance. On RTE, ChatGPT is much better on the entailment class (92.5 vs 70.6) and worse on not-entailment (78.6 vs 90.8) tab. 5. On SST-2 the gap is mostly on negative examples §4.2.8. The authors attribute these patterns to RLHF and training data. These are their hypotheses, not measured results.
• NER scope. ChatGPT scores 4.1 F1 on Miscellaneous, against 4.8 for GPT-3.5 tab. 10. The authors argue that the gold labels and the models disagree on what counts as an entity, and that CoNLL03 labels may need cleaning §4.2.7. Their attempt to ask GPT-3.5 for each entity type separately gave F1 34.8 §4.2.7.
• Format conformance. The paper says that naming the exact output words (for example "entail" and "not entail") helped ChatGPT follow the requested format on NLI §4.2.3. For other tasks it reports off-format output. An implementer should parse outputs strictly and log failures.
• Appendix examples. The appendix shows input-output pairs with the answers marked as correct or wrong app. A. The text does not say whether these pairs were used as demonstrations in any experiment. The main experiments are zero-shot §3.
• Instructions in figures. The per-task instruction wording is shown in fig. 2 and fig. 3, and the text quotes only the sentiment instruction §3. The NER example prompts in the appendix name the four entity types in their instruction app. A.

Lineage

• Builds on: Kojima et al., zero-shot chain-of-thought ("Let's think step by step"), arXiv:2205.11916. The two-stage prompt follows it §3.
• Builds on: Wei et al., chain-of-thought prompting, arXiv:2201.11903. Source of the few-shot reasoning baselines §2.3, tab. 2.
• Related: Wang et al., self-consistency, arxiv:2203.11171. Named as a way to reduce commonsense errors §4.2.2.
• Compared against: GPT-3.5 (text-davinci-003) §3; text-davinci-002, UL2, LaMDA, Codex, PaLM tab. 2, tab. 3; FLAN, T0, GPT-3.5 tab. 4; BART and CODA tab. 9; Flair, LUKE and ACE tab. 10; T5-11B tab. 12.
• Superseded by: not stated in this wiki.