On the Opportunities and Risks of Foundation Models

In plain words

Large AI models are now trained once on huge amounts of general data, and then reused for many jobs. This report, written by a large group of researchers, gives that kind of model a name, foundation model, and asks what it makes possible and what could go wrong.

Two ideas run through the report. The first is emergence: abilities show up because the model is big and trained broadly, not because someone programmed them in, so nobody can fully predict what a model can do. The second is homogenization: many systems now share the same base model, so one improvement helps everything built on it, but one flaw is passed on to everything too.

The report walks through what these models can do (language, vision, robots, reasoning, interaction), where they are used (healthcare, law, education), how they are built, and what society has to watch for. It does not give a recipe to build one. It is a map of the open questions, and it says openly that not all authors agree with every view in it.

TL;DR

• Defines a foundation model as a model trained on broad data, usually with self-supervision at scale, that can be adapted to many downstream tasks §1.
• The report covers 26 sections in four parts: capabilities, applications, technology and society §1, overview; fig. 4.
• Its two central themes are emergence and homogenization: the same base model gives leverage, and it also passes its defects to every adapted model §1.1; fig. 1.
• It contains no experiments and no benchmark tables; figures and numbers come from other work and are cited as such.

Problem

The report asks how to reason about the opportunities and risks of models that are trained once and then adapted to many tasks, when their capabilities and failure modes are not well understood §1. It also argues that their social impact is easier to judge for a specific deployed system than for the base model, because the base model can be adapted to unforeseen uses §1.2.

Method

The source is a synthesis report, not a method paper. Its framework has four parts that a reader can use to organize the material.

Definition and history. A foundation model is any model trained on broad data, generally with self-supervision at scale, that can be adapted (for example by fine-tuning) to a wide range of downstream tasks. BERT, GPT-3 and CLIP are named as examples §1. The report says the building blocks (deep networks and self-supervised learning) are not new; scale, the Transformer architecture, and more data are what changed §1. It frames the shift as transfer learning plus scale: pretraining on a surrogate task, then adapting to the downstream task §1.

Ecosystem. The report splits the pipeline into five stages. Each stage can create harm, and each can be run by a different organization §1.2; fig. 3:

1. data creation, which is always done by people;
2. data curation, where a dataset is selected and filtered;
3. training of the foundation model;
4. adaptation to a task, which often adds rules, classifiers and other signals around the model;
5. deployment to people.

It says deployment is a separate decision from construction, and that deployed systems should be tested more rigorously than research models §1.2.

Five properties of a foundation model. For the modeling chapter, the report names five properties it considers essential: expressivity, scalability, multimodality, memory capacity and compositionality §4.1.

Adaptation. The report names three factors that a practitioner should weigh when choosing an adaptation procedure §4.3:

1. the compute budget, in memory and storage;
2. how much task-specific data exists;
3. how much access to the model's gradients is available.

Low-storage methods freeze most pretrained parameters and learn a small set of task-specific ones. The report lists examples: tuning only the last layer, bias vectors, low-rank residuals, masks, soft prompts, and inserted modules §4.3. It flags that low-memory methods matter for access, and that some of them trade compute for memory §4.3. It also argues that large models are likely to be offered only through restricted API access rather than full parameter access §4.3.

Formal view of adaptation. The theory section writes adaptation as a constrained optimization: adapted parameters are chosen to minimize the adaptation loss, given the pretrained parameters, under an optional complexity constraint §4.10. It states that current learning theory does not fully explain the gap between the pretraining phase and the adaptation phase §4.10.

Environmental accounting. The environment section gives a simple score for a model's overall impact, with four terms: net social benefit, social cost of carbon, energy cost, and other second-order effects §5.3:

# Overall impact of model M, in dollars  [§5.3]
# S: net social and environmental benefit
# C: social cost of carbon from energy use
# E: energy cost, including grid strain
# O: other second-order environmental effects

def overall_impact(model, S, C, E, O):
    # value = benefit minus three kinds of cost  [§5.3]
    return S(model) - C(model) - E(model) - O(model)

Written as an equation, the score is V(M) = S(M) - C(M) - E(M) - O(M) §5.3.

Other ideas the report proposes. It proposes a "data hub" for managing the data behind foundation models §4.6. It proposes a "one model, many models" view for interpretability: asking how far the base model and its adapted versions share the same decision-making parts §4.11. For fairness, it separates intrinsic biases (properties of the foundation model) from extrinsic harms (arising in a specific application) §5.1. For misuse, it defines misuse as using the model as designed but to cause social harm §5.2.

Hyperparameters

Not applicable. The report proposes no training or inference method and gives no learning rates, batch sizes or architectures of its own §1. Where it describes the parameters of existing models, it cites them to other papers (for example GPT-3 with 175 billion parameters §1.1).

Results

The report has no experiments and no tables. The following numbers appear in the text. They are cited from other sources and were not measured by the authors of this report:

The source also reports qualitative results from its own syntheses, such as the claim that adapting a foundation model improves robustness across a wide variety of distribution shifts, which the report attributes to existing work §4.8.

Paper vs code

No official code. The report has no code repository. Searches done: the paper and its source contain no project or code links (only news, government and product links); a GitHub repository search for the title words returned only third-party copies, which were not used.

Pitfalls

• Nothing to implement. A coding agent cannot take a training or inference algorithm from this card. Use it for context, risks and open questions, and go to the cited method papers for implementations.
• Not a formal definition. The term "foundation model" was chosen partly for its sociological meaning, not as a precise technical class. The report says so §1, naming.
• Views differ between authors. The report says that not all of its views are held by all authors § author contributions.
• Incomplete by design. It states that it omits many applications, for example natural sciences, music, finance and agriculture §1, overview.
• Dated. The arXiv version used here is v3, dated 2022-07-12. The model examples and cost figures reflect the field at that time, not today.
• Cited numbers are second-hand. Every number in Results comes from another work. Check the cited source before reusing a number.
• Equation numbers not verified. Equations are cited by section, not by number, because the numbering of the compiled PDF was not checked.

Lineage

• Builds on: the technical foundations it names, transfer learning and self-supervised learning §1, with BERT, GPT-3 and CLIP as its anchor examples §1.
• Compared against: not applicable; the report makes no comparisons of methods.
• Superseded by: not known from content already in this wiki.