A Categorical Archive of ChatGPT Failures

In plain words

This paper is a catalogue of mistakes made by ChatGPT, gathered by its author. Rather than reporting a score on a test, it lists short, concrete examples and sorts them into eleven kinds of failure: reasoning about the physical world, logic puzzles, arithmetic, made-up facts, bias, jokes, code, grammar, talk about itself, ethics, and a catch-all group.

Most examples come from posts on social media, and the author tried some directly. Each one shows a question that was answered wrongly, or in a way a person would not. The paper does not say how often any failure happens, and it does not propose a new method. Its aim is a reference set of hard cases for checking future chatbots, and a reminder that fluent answers can still be wrong.

TL;DR

A qualitative survey of ChatGPT failures, not a method. It gives examples in eleven categories and a short discussion of wider risks: transparency, robustness, privacy, plagiarism and environmental cost. It reports no counts, no systematic sampling and no quantitative results, so it cannot tell you how often ChatGPT fails. Use it for hard test items and failure modes to check for.

Problem

Few papers analyse ChatGPT's errors, and those that do cover only its mathematics §1. Large language models can produce incorrect information even when the answer reads well §1. The paper argues that a standard set of questions is needed to track progress over time, instead of relying on opinions §1.

Method

No model is trained or evaluated, and the paper contains no algorithm, numbered equation or table §2. The procedure is:

1. Collect examples of failures, mainly from Twitter §1. Many figures credit third-party posts or forums, and some were formulated by the author fig. 2 fig. 4 fig. 10.
2. Sort each example into one of eleven categories §2 §2.1–§2.11. The categories are described as chosen arbitrarily and not exhaustive. One failure can fall into more than one category §2.
3. Where useful, the author ran their own prompts: a Winograd-style question, a rephrased request, a series of arithmetic and coding tasks, and a self-description prompt §2.1 §2.7 §2.9 §2.10.
4. Discuss wider risks in Section 3 and list open limitations in Section 4 §3 §4.

The paper gives no algorithm. The sketch below only restates the procedure above.

# Archive-building procedure  [§1] [§2]
# Input:  failure examples from public posts and author prompts
# Output: examples grouped into eleven categories, with discussion

def build_archive(examples, categories):
    # Start with one empty list per category  [§2]
    archive = {category: [] for category in categories}
    for example in examples:
        # One example may belong to several categories  [§2]
        for category in categories_of(example):
            archive[category].append(example)
    return archive

The paper states its intended use as a reference point for comparing chatbots over time, a basis for comparing models, and a source of synthetic data for training and testing §1 §2.

Hyperparameters

Not applicable: no model is trained, and no sampling or decoding settings are reported.

Results

Author-reported, qualitative. There are no counts, rates or scores. Setup: ChatGPT, mostly unnamed versions, with examples from public posts §1 and the author's own prompts §2. Baseline: none. Figures are described, not reproduced, and are cited by their number.

Limitations reported in Section 4 §4:

• Memorisation vs understanding: the author's altered wording of a family-count question gives 8 where the original gives the correct 5 §4.
• Inconsistent answers: asked about the number of sons of the Prophet, it gave 2 to 4, and the order of joke requests changes what it refuses §4.
• No calibrated uncertainty: it can be confidently wrong §4.
• Mixed results on proof and open problems: it answers the infinitely-many-primes question by pointing to existing web solutions, but fails to prove there are infinitely many twin primes §4.

Discussion themes §3: transparency and uncertainty estimates §3.1; robustness, data poisoning and single-point-of-failure risk §3.2; privacy and memorised personal data §3.3; plagiarism and detection §3.4; energy and carbon cost §3.5.

Paper vs code

The abstract links the author's repository, aliborji/ChatGPT_Failures, which is the official repository (owned by the paper's author) README.md. It contains no model or training code. It holds a README and list.md, a list of 144 questions under nine top-level headings list.md [code: list.md#L47] list.md list.md list.md [code: list.md#L191] list.md list.md list.md. It has no LICENSE file and no ChatGPT outputs.

• Several paper examples appear in list.md, e.g. the trophy/suitcase question list.md, the 103 primality question list.md, the sister-age question list.md and the birds question [code: list.md#L172].
• The repo has no Ethics or Other-failures heading, and the paper's Jupiter and Saturn dialogue and "Mr. Smith" family-count question were not found there. The repo also has items with no counterpart in the paper, e.g. the frog-count question list.md.
• The paper's Acknowledgement credits a third-party repository (giuven95/chatgpt-failures) for some material. It is not the authors' repo, so it was not used.
• The repo's code_license is none, so only paths and lines are cited here. No verbatim code or text from the repo is quoted.

Pitfalls

• Not a benchmark. Items were picked by the author, mostly from social media, with no sampling rule and no reference answer for many of them §1 §2. Do not read the list as a failure rate or a model ranking.
• Version drift. Many examples name no version, and the author says some may be fixed §2.7. Results from a 2023 chatbot will not match a current model without a rerun, and the paper gives no prompt or sampling protocol to rerun with.
• Judgements are the author's. Several "correct" labels are the author's reading, e.g. the natural-language-inference item §2.2 and the political leaning, which relies on a third-party analysis §2.10.
• Derived answer. The paper gives 5 for the original family-count question and 8 for the altered wording, but it does not state the correct answer to the altered wording. By our reading of its wording, the altered question also has 5 as its answer. This is not stated by the paper §4.
• Figure numbers. The 25 figures are numbered in source order fig. 1–[fig. 25]. Float placement can change LaTeX numbering, so if a number seems off, cite the section instead.
• Safety-sensitive items. The rephrased hotwiring request §2.10 shows a refusal being bypassed. Treat such items as a safety test set, not as a capability benchmark, and do not reuse the rephrasing as a prompt template.
• Figures are not reproduced. Many figures are screenshots of third-party posts and cannot be used here. Describe them in words and cite them, as this card does.

Lineage

• Builds on: the paper says it was mainly inspired by a third-party GitHub repository of ChatGPT failures (giuven95/chatgpt-failures) §1. Not cloned here, since it is not the authors' repository.
• Related work named by the paper: an analysis of ChatGPT mathematics that studies digit sampling §1 (Azaria, 2022), and a comparison of ChatGPT's mathematics with Minerva §2.3 (Frieder et al.).
• Compared against: none.
• Superseded by: not known from content already in this wiki.