The Vector Grounding Problem

In plain words

Language models learn only from text, with no eyes, hands or direct contact with the world. That raises an old question: can the words and internal numbers of such a system be about real things, or does their meaning exist only in the heads of the people reading them? The authors argue that it can.

What counts is whether a part of the system is connected to the things it is about, and whether that connection was shaped by a process that rewarded it for being right about them. Text written by people already carries traces of the world. Training a model to prefer accurate answers over false ones adds a direct pressure toward being right. The authors also argue that a model can pick up such pressure during pre-training or while it works on a task, though they present these two routes as more tentative. None of this says the model understands anything. It only says its outputs can be about something.

TL;DR

The paper argues that some LLMs have internal states that are referentially grounded, meaning they stand for worldly things, so their outputs have meaning that does not depend on human interpreters. Two conditions must hold: a causal or informational link to the world, and a history of selection that gave those states the function of carrying that information. The argument is strongest for post-trained models with preference tuning. It is a philosophical position with no experiments, so no measured numbers are reported.

Problem

The classical Symbol Grounding Problem asks how symbols that are manipulated by their shape alone can refer to real things rather than only to other symbols §2. The authors ask whether the same problem arises for LLMs, whose internal states are continuous vectors, and call it the Vector Grounding Problem §1, §3. The worry is that LLM outputs may be meaningful only to the human who reads them. The paper's "Octopus Test" pumps this intuition with a thought experiment in which a system that only handles signals from other speakers reproduces their statistics §3.

The paper does not claim that LLMs understand language, have knowledge or beliefs, or have any conception of truth §1. Its sole claim is that the grounding problem is solvable in principle, and in some cases already solved in practice §1.

Method

The paper is an argument rather than an algorithm. Its steps are:

1. Pick the right kind of grounding. Five notions of grounding are distinguished: referential, sensorimotor, relational, communicative and epistemic §4, fig. 1. Only referential grounding, the link between a representation and its worldly referent, is essential to the problem. The others either stay inside a system of representations or do not explain how a representation comes to be about one thing rather than another §4.1–§4.6.

2. Take the conditions from theories of content. A state is referentially grounded when two conditions hold [abstract, §5, §8]:

   • it stands in appropriate causal-informational relations to the world, and it is used by the system because it carries that information §5;
   • it has a history of selection that gave it the function of carrying this information, so that it can be correct or incorrect §5, §6.2. The authors rely on the selected-effects tradition in philosophy of biology §5. They note that other accounts exist and differ in emphasis §5.

3. Show the causal-informational condition holds for LLMs. Training text is produced by humans who are causally connected to the world, so LLMs stand in indirect, mediated causal relations to it, similar to how humans come to represent things they have never touched, such as enriched uranium §6.1.

4. Show the selection-history condition holds, by three routes.

   • Post-training §6.2.1. Preference tuning, such as RLHF or DPO, selects internal states because they help produce outputs that human judges rate as truthful, helpful or harmless. The paper gives a toy DPO example aimed at factual accuracy §6.2.1. This is the paper's most direct route.
   • Pre-training §6.2.2. Next-token prediction can, in principle, select for internal models of the process that generated the data. The argument is framed as implicit meta-learning §6.2.2. It is argued to hold mainly in formally constrained domains, such as games, and the authors call the mechanism speculative §6.2.2. The Othello-GPT studies are the cited empirical support §6.2.2.
   • In-context learning §6.2.3. During a single prompt, an activation pattern could be selected for serving a task defined in the context, so it would acquire a short-lived function. The authors say whether this happens is open §6.2.3.

5. Apply the conditions to variants. A randomly formed duplicate of a trained model, a "Swamp LLM", lacks grounding because it has no selection history §7.1. A model that only went through pre-training and happens to match a post-trained model's parameters has a learning history, so its grounding depends on whether the pre-training route is accepted §7.1.

6. Assess multimodal and embodied systems. Multimodality and embodiment are neither necessary nor sufficient §7.3, §7.4, §8. CLIP is argued not to be grounded because its contrastive objective has no world-involving standard §7.3. A vision-language model with a frozen language model is argued to inherit the language model's grounding §7.3. Embodied systems such as SayCan keep the language model frozen and so add nothing to its grounding. RT-2, RT-X and OpenVLA may inherit grounding, and their embodied training may add more §7.4.

Figure 1 is a five-panel diagram, labelled A to E, that illustrates the five notions of grounding with a DOG example fig. 1.

# Referential-grounding test for one internal state  [§5, §8]
# Input:  a state of a model, its causal history, its selection history
# Output: True if the state is referentially grounded

def is_referentially_grounded(state, causal_history, selection_history):
    # Causal-informational condition: the state carries information about
    # the world, possibly through human-produced data  [§5] [§6.1]
    causal_ok = carries_worldly_information(state, causal_history)

    # The state must also be used by the system because of that information
    # [§5]
    used_for_information = is_used_for_information(state)

    # Selection condition: the state was selected for carrying that
    # information, so it has a function and can misrepresent  [§5] [§6.2]
    selected_for_information = was_selected_for(state, selection_history)

    return causal_ok and used_for_information and selected_for_information

Hyperparameters

Not applicable. The paper trains and evaluates no model and reports no training configuration §1 §8.

Results

No measured results. The paper contains no experiments or tables. Its empirical support is cited from other work: Othello-GPT, a transformer trained only on Othello move sequences, whose linear board-state directions can be intervened on to change its move predictions §6.2.2. The authors also say these studies cover formally constrained games, not open-ended language §6.2.2. Every other claim is argued from the philosophical literature or from described thought experiments §3, §7.

Paper vs code

No official code. The arXiv metadata and the LaTeX source contain no code or project link. A GitHub repository search for "vector grounding problem" returned one third-party repository, a reply to the paper by another author, which is not an official resource. A GitHub user search for the first author's surname found an account named raphael-milliere. A repository search for "grounding" within that account returned nothing, so no author-owned code was found.

Pitfalls

• Position paper, not evidence. Most claims are arguments from theory and thought experiments. The authors call the pre-training and in-context routes speculative, and say whether in-context selection happens in LLMs is open §6.2.2, §6.2.3. Do not read the card as establishing that any deployed model is referentially grounded.
• Preference tuning carries the main argument. The most direct route is post-training with human preference data §6.2.1. The authors agree that fine-tuning is a marginal factor in an LLM's capabilities, and argue that grounding is a different question §7.2. Grindrod's objection, quoted in §7.2, argues that a marginal procedure cannot make the difference between intentionality and its lack. Treat this disagreement as open.
• Depends on one theory of content. The conditions come from teleosemantic, selected-effects theories §5. Other theories, including ones built on structural resemblance or inferential role, are mentioned but not adopted §5, §6.2.2. A reader who rejects selected-effects accounts will reject the conclusion.
• Structural correspondence is not enough. Mapping onto the world's structure is cheap, so a world model alone does not secure representation. The authors require the function and use described above §6.2.2.
• Intrinsic meaning is not understanding. The paper says intrinsic meaning is compatible with externalist views of content and does not imply understanding, knowledge or consciousness §1 fn., §8. Do not cite the paper as support for those.
• Deviant meanings. Post-trained models could have intrinsic meanings that differ from human ones, depending on the norms used in training. The paper leaves this to future work §6.2.1.
• Game domains are the clearest case. The authors say pre-training selection is most plausible in formally constrained domains such as games, where success conditions are fixed by rules §6.2.2. Open-ended text is harder to justify.
• Othello-GPT evidence has limits. The authors also note that Othello-GPT may use many local heuristics rather than one board-state algorithm. They argue this still fits their view §6.2.2 fn..
• Duplicate and in-context cases. Frozen weights with in-context learning do not create grounding by themselves. The authors argue the capacity for in-context learning needs its own selection history §7.1 fn..
• Terms. "Referential grounding" is the one notion the paper says is essential. Other uses of "grounding" in NLP and cognitive science are set aside §4. Read "grounded" in this card as "referentially grounded" unless stated.
• Version. The compiled source is arXiv v3, updated 2025-12-09. Its date line says the paper is forthcoming in Philosophy and the Mind Sciences [source date line]. The published text may differ.
• Citations. The paper has no numbered equations or tables. Citations here point to section numbers, counted from the LaTeX \section and \subsection commands.

Lineage

• Builds on: the Symbol Grounding Problem of Harnad §2; the Chinese Room critique it follows §2 fn.; selected-effects theories of function and representation §5; the meta-learning account of pre-training from Wu et al. §6.2.2; mesa-optimisation in in-context learning §6.2.3.
• Discussed critic: Grindrod's objection to the post-training argument §7.2.
• Discussed thought experiments: the Octopus Test, attributed to Bender (2020) §3; the swampman scenario, adapted as the Swamp LLM §7.1.