Emergent Abilities in Large Language Models: A Survey

In plain words

Some abilities of large language models seem to appear suddenly once a model is big enough. Before that point the model is close to random on a task, and after it the score jumps. This survey asks whether those jumps are real, what causes them, and what they mean for safety.

The authors first show that people use the word "emergent" in several incompatible ways. They then look at the evidence. Some jumps shrink when the scoring method changes, while others persist. Some researchers argue that a jump only counts as emergence when you can also find the internal change that produces it. The survey then covers related topics: learning from examples in the prompt, models that reason before they answer, groups of agents, and behaviours such as deception that can appear alongside useful skills.

TL;DR

This is a literature review of emergent abilities in LLMs. It finds the definitions inconsistent, the evidence split between "jumps are real" and "jumps are a scoring artifact", and a growing set of mechanistic studies that point to internal structure behind some abilities. It also covers reasoning models, multi-agent systems and emergent misalignment. It reports no experiments of its own, so every number on this card comes from a cited study.

Problem

An ability is called emergent when it is absent in smaller models and present in larger ones, and when a scaling curve fitted to small models would not have predicted it §2. The field has no shared definition, and the authors list this as the central source of confusion §1. Because emergence is hard to predict, it matters for choosing model sizes and for anticipating harmful capabilities §1.

The survey asks three questions. What do the competing definitions mean? Under which conditions do abilities appear, such as scale, pre-training loss, task complexity, quantization and prompting? Are the jumps real or metric artifacts §1, §2, §3?

Method

This is a narrative survey. The authors gathered papers from a Google Scholar query on "Emergent Abilities" and "Large Language model", plus tracing of citations from foundational works §1, §3. They favour work that adds mechanistic, statistical or predictive insight over work that only reports a new benchmark gain §1. The survey is organised as follows.

1. Compare definitions, from Lewes and Anderson through Hopfield to Wei et al. and the complexity-science framing of Krakauer et al. §2.
2. Review evidence on scale and metric choice §3.1, prompting and tuning §3.2, pre-training loss §3.3, quantization §3.4, task difficulty §3.5, internal mechanisms §3.6 and predictability §3.7.
3. Cover in-context learning §4, large reasoning models §5, LLM-powered agents §6, harmful behaviours §7 and governance §7.1.
4. Conclude with the distinction between emergent capability and emergent intelligence §8.

The survey proposes one evaluative test. Ask whether a low-dimensional internal structure tracks the ability, whether changing that structure changes the behaviour, and whether the structure screens off the individual weights §2. Few abilities pass this test in the cited work §2.

The only formula in the survey is a threshold on an emergence score, which the authors take from a BIG-Bench meta-analysis §3.1. The score compares the spread of a metric across model sizes with a typical gap. The text defines the spread but not the typical gap precisely, so the sketch below marks that step.

# Emergence score for one metric-task pair  [§3.1]
# Input:  metric values for models of increasing size, shape (M,)
# Output: scalar score; the cited meta-analysis calls a pair emergent above 100

def emergence_score(scores):
    # Spread between the largest and smallest metric value  [§3.1]
    spread = max(scores) - min(scores)
    # "Median difference": the text does not define it precisely  [§3.1]
    typical_gap = median_gap(scores)
    return spread / typical_gap

The authors argue that a score between 20 and 100 may also indicate a discontinuous jump §3.1.

Hyperparameters

Not applicable. The survey defines its scope as a review and reports no training or evaluation of its own §1. Settings that matter for reproducing the cited studies are listed in the Results table and the Pitfalls, not here.

Results

Author-reported in the cited studies, not by this survey. The survey quotes them with the setup given below. Setups are as stated in the survey's text.

The survey's own tallies of the emergent-misalignment studies are in tab. 2. Three of the six studies characterise onset as abrupt tab. 2.

Paper vs code

No official code.

Searched: the arXiv metadata has no code link, the LaTeX source has no repository or project-page link, and a GitHub repository search on the title words returned one result, owned by a third party (AnshSingh-2024). It is not an author repository, so it was not used.

Pitfalls

• Survey, not a method. Every number is a snapshot of someone else's study. The authors warn that model-specific figures date quickly §1.
• Selection is not systematic. Papers come from one keyword search and citation tracing §1, §3. Coverage of the field is therefore unknown.
• Emergence score is underspecified. The "median difference" in the score is not defined in the text §3.1. The 100 threshold is contested: the authors suggest 20–100 may also count §3.1.
• Metric artifact is contested. Schaeffer et al. argue jumps vanish under linear metrics; Du et al. and Zhao et al. argue they survive §3.1. Gu et al. report transitions under validation loss §3.1. Do not pick a side without reading the metric and the seed count.
• Uncited numbers. The frontier scores in §3.2 (Claude 3.5 Sonnet on GSM8K, o3-mini and GPQA) appear without a citation at that sentence §3.2. Treat them as unverified.
• Figures are third-party. fig. 2 and fig. 3 are reproduced from Schaeffer et al. and fig. 4 from Wu et al. This card describes them only. fig. 1 is an overview diagram, and its contents are not described here.
• Commented-out material. The source contains commented-out blocks, including an earlier emergent-misalignment table and a taxonomy table. They are not in the compiled paper. Cite only tab. 1 and tab. 2.
• Prompting claims are dated. The survey itself says the Lu et al. finding that instruction tuning does not yield reasoning is dated, and it contrasts this with later reasoning models §3.2.
• Correlations in ICL. The symbolic induction heads show r=0.11 with standard induction heads and r=0.86 with function-vector heads §4. These are correlations within a single study.
• Pre-training loss is correlational. The survey notes that the loss threshold story is not causal, and that a common confounder is plausible §3.3.

Lineage

• Builds on: the definition in Wei et al., arXiv:2206.07682 §2, §3, and the complexity-science criterion of Krakauer et al. §2.
• Compared against: Lu et al. (2023) on few-shot, instruction tuning and CoT §3.2; Schaeffer et al. (2024) on metric artifacts §3.1; Du et al. on pre-training loss §3.3.
• Superseded by: not stated.