Paper · 2023-07-26 · cs.CL, cs.CY
Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions
Literature review of 109 Scopus-indexed ChatGPT articles published up to 25 March 2023. It groups them as evaluations, predictions and reviews, lists application areas such as healthcare, science writing and software engineering, sorts known limitations into intrinsic and usage issues, and proposes future directions. It runs no experiments; all counts are author-reported.
LLM ethicsAI-generated text detectionHallucination mitigationBibliometric analysissurveychatgptllm-applicationsethics
AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
ChatGPT became popular quickly after its launch, and people in many fields started using it and writing about it. This paper gathers the academic articles about it that existed up to March 2023, sorts them into groups, and summarises what they report. The groups are studies that test ChatGPT, studies that guess how it will affect a field, and earlier reviews.
The authors also list where ChatGPT is used (health, business, software, science writing, education, environment, language processing), what goes wrong with it (made-up facts, bias, out-of-date knowledge, misuse, copying), and what they think could come next. The paper does not run any experiments of its own. Its value is the map it draws, not new measurements.
TL;DR
- A literature review of 109 Scopus-indexed articles on ChatGPT, collected up to 25 March 2023 §2.
- It sorts the articles into evaluations, predictions and reviews §4, and covers seven application areas §5.
- It groups limitations into intrinsic issues (hallucination, bias, not real-time, misinformation, inexplicability) and usage issues (ethics, copyright, over-reliance) §6.
- It proposes future directions (conversational ability, personalization, multimodal design, trustworthiness) without testing them §7.
- Use it as a map of the early literature and as a list of open questions, not as evidence for any performance number.
Problem
The authors argue that ChatGPT research was growing fast across many fields, and that earlier reviews on large language models and AI-generated content were too broad to cover ChatGPT's specific issues §1.3. They set five research questions on architecture and progress, publication trends, applications, multimodal use and challenges §1.3. The paper states no hypothesis to test; it is descriptive and does not test whether ChatGPT works in any setting.
Method
This is a systematic-style literature review. The authors cite the Kitchenham procedure as their basis §2.
- Search. The corpus comes from the Scopus database. The paper says papers were found with a search for "chatgpt" or "chat-gpt" in the title or abstract §2 (IC-3, EC-2). The exact query string is not stated.
- Filter. Inclusion and exclusion criteria are applied, listed in the Hyperparameters section below §2. Screening is done by reading the abstract and judging its fit to the title §2.
- Corpus. After filtering, 109 articles remain. They come from 349 authors in 53 nations §2. Subject areas are reported in fig. 1 and countries in fig. 2 §2.
- Classification. Each article is put into one of three groups: evaluations of ChatGPT, predictions made with ChatGPT, or reviews. The classification uses the abstract or the full text §4.
- Synthesis. The authors describe a common structure in most papers (choose a topic, ask ChatGPT questions, interpret the answers) §4 fig. 3. They build a taxonomy of research areas §4 fig. 4. They then summarise each application area in §5 and tab. 2, and compile the limitations in tab. 3 §6.
The screening steps, reconstructed from the criteria in §2, are below. The paper gives no code; the attribute names are ours.
# Screening as the paper describes it [§2] (IC-1 to IC-5, EC-1 to EC-5)
# Input: Scopus search hits (records)
# Output: included articles
def screen(records):
# Keep English, peer-reviewed works [§2] (IC-1, IC-2)
kept = [r for r in records if r.language == "en"]
kept = [r for r in kept if r.peer_reviewed]
# Keep only works with "chatgpt" or "chat-gpt" in title or abstract
# [§2] (IC-3, EC-2)
kept = [r for r in kept if mentions_chatgpt(r.title, r.abstract)]
# Drop short papers, duplicates and errata versions [§2] (EC-1,4,5)
kept = [r for r in kept if not r.short and not r.duplicate]
kept = [r for r in kept if not r.erratum]
# Keep only works published on or before 25 March 2023 [§2] (IC-5)
kept = [r for r in kept if r.published <= "2023-03-25"]
return keptHyperparameters
This paper trains and runs no models. Its only settings are the search and inclusion choices below, listed here because they decide what the corpus is.
| Name | Value | Where | Notes |
|---|---|---|---|
| Database | Scopus | §2 | The only database named |
| Keyword | "chatgpt" or "chat-gpt" in title or abstract | §2 (IC-3, EC-2) | Exact query string not stated |
| Language | English | §2 (IC-1) | |
| Publication cut-off | on or before 25 March 2023 | §2 (IC-5) | Later work is not covered |
| Peer review | required; workshop, journal, book or conference | §2 (IC-2) | |
| Excluded types | short papers, GPT-only papers, duplicates, errata versions | §2 (EC-1, EC-3, EC-4, EC-5) | |
| Screening | abstract read against the title | §2 | Number of screeners not stated |
| Category rule | evaluation, prediction or review | §4 | Tie-break rule not stated |
Results
Author-reported counts. The paper runs no experiments, so these describe the corpus, not ChatGPT's performance.
| Setting | Metric | Value | Source |
|---|---|---|---|
| Corpus | included articles | 109 | §2 |
| Corpus | authors / nations | 349 / 53 | §2 |
| Subject area | medicine / social sciences / computer science | 23% / 20% / 11% | §2 fig. 1 |
| Country | publications: United States / UK / Australia / China | 33 / 10 / 9 / 9 | §2 fig. 2 |
| Venue | Nature (journal) | 13 articles | §3 |
| Category | evaluations / predictions / reviews | 68 / 39 / 10 | §4 |
| Topic | evaluations of scientific writing | 30 publications | §4 |
| Topic | predictions about education | 12 publications | §4 |
| Reviews | reviews focused on ChatGPT alone | none found; one review covers chatbots more broadly | §4 |
Other reported findings, cited to their sections: the paper names 13 other ChatGPT surveys and compares them across several criteria in tab. 1 §1.3. It finds no NLP conference papers in its corpus §3. It says most papers centre on ChatGPT as a scientific-writing assistant or on predictions about education §4.
Paper vs code
No official code. The paper links no repository or project page. Its only URLs are in the bibliography and in an Overleaf comment. A GitHub search through the API for "decoding chatgpt taxonomy" returned no repositories. The paper implements nothing, so there is no code to compare.
Pitfalls
- Old cut-off. The corpus ends on 25 March 2023 §2. Later evaluations, models and policies are not covered, and the paper's predictions and "current" challenges were written for that date.
- Counts do not add up. The category counts 68, 39 and 10 sum to 117, but the corpus has 109 articles §4 §2. The paper does not say whether categories can overlap, so do not treat the three counts as a partition.
- Roadmap is off by one. The introduction says Section 2 covers research domains, Section 3 trends, Section 4 applications and so on §1.3. The compiled sections are numbered Methodology §2, Diversity §3, Trends §4, Applications §5, Challenges §6, Future §7 and Conclusion §8. Cite the numbered sections above.
- Search is hard to reproduce. The query string, the number of screeners and the way disagreements were settled are not given §2. Another team may get a different corpus.
- Architecture paragraph is generic. It describes ChatGPT as an encoder-decoder transformer with a tokenizer and softmax output §1.1. The paper cites no source that specifies ChatGPT's internals, so use this only as background, not as a spec.
- Self-description is not evidence. The quoted answer in which ChatGPT describes itself §1.1 is the model's own text. It shows nothing about how the model works.
- Wording drift. The research-questions paragraph talks about "recommender systems" §1.3, which the review does not cover.
- Eight areas or seven? The introduction says the review covers 8 applications §1.3. The Applications section and the abstract list 7 areas §5. Cite the section list, not the count.
- Future work is untested. The proposals in §7 (more data, fine-tuning, human feedback, personalization, multimodal inputs, fairness, transparency, explainability) come with no experiments or measured effects. fig. 5 to fig. 8 are diagrams of those ideas, not measured results.
- Ethics and detection. The detection discussion §6.3 describes a two-step idea: first decide if text is ChatGPT's, then decide if it is fake. The paper reports no detection accuracy for ChatGPT text, and it notes that detection of ChatGPT-written text is "relatively new" §6.3.
- No quality appraisal. The method describes screening by abstract and criteria §2, with no quality assessment of the 109 articles. Treat the counts as a corpus description.
Lineage
- Builds on: the Kitchenham systematic-review procedure §2.
- Compared against: 13 earlier ChatGPT surveys, listed in tab. 1 §1.3. It also positions itself against broader large language model and AI-generated content reviews §1.3.
- Superseded by: not known from this wiki.
Source and license
Adapted from “Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions” by Shahab Saquib Sohail, Faiza Farhat, Yassine Himeur, Mohammad Nadeem, Dag Øivind Madsen, Yashbir Singh, Shadi Atalla, Wathiq Mansoor (arXiv:2307.14107), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.