Paper · 2024-07-15 · cs.AI, cs.CL, cs.CY, cs.LG
Transforming Agency. On the mode of existence of Large Language Models
Philosophy paper arguing that LLMs such as ChatGPT are not autonomous agents. It explains GPT-3's architecture, training and agent add-ons, then tests three embodied-agency conditions: LLMs fail individuality and normativity and partly meet interactional asymmetry. It proposes "interlocutor automata" and midtended agency. No experiments; no code.
LLM agentReinforcement learning from human feedback (RLHF)Attention mechanismHuman-AI collaborationagencyphilosophy-of-mindllm-analysistransformers
AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
This paper asks what kind of thing a large language model such as ChatGPT is. Some people treat it as close to a thinking person, and others dismiss it as a random word machine. The authors first explain how these systems are built and trained, and how they are wrapped in extra software so they can act like assistants. Then they ask whether they are agents: systems that make themselves, set their own goals and start their own actions. Their answer is no. The model only reacts to the text it is given and holds no goals of its own.
They propose a different description. The system is closer to a huge library that can talk back, one that holds the written traces of many people and can carry on a conversation with you. A chat with it can feel like talking to a real partner, even though no one is on the other side in the sense a person would be. The authors also suggest that when people work closely with such systems, their own agency can become blended with the machine's output. They call this midtended agency.
TL;DR
The paper is a conceptual analysis, not an experimental study. It walks through GPT-3's architecture, training and agent extensions, then tests the model against three embodied-theory conditions for autonomous agency. Its verdict: the model fails individuality and normativity and only partly meets interactional asymmetry. It proposes describing LLMs as interlocutor automata. Relevant if you need a precise framing of what LLM agents lack, or a citable argument on human-LLM agency. It supplies no benchmark numbers and no implementation details.
Problem
The authors ask what the ontological status of LLMs is: whether they are intelligent, whether they are agents, and what else they could be §1. They find the public debate polarised between inflationary views (LLMs as sentient or as genuine authors) and deflationary ones ("stochastic parrots") §1. They argue that neither side specifies what agency is, so the status cannot be settled by benchmark scores alone §2. The paper treats the question as one about the system's mode of existence, which requires understanding its architecture, training and coupling with its environment §1 §3.
Method
The paper is an argument in three parts. It does not define or train a model.
Part A: technical account (§3). The authors describe GPT-3 as the prototype of a transformer LLM §3. They trace one forward pass through seven steps: tokenisation, embedding, positional encoding, 96 blocks of attention and feed-forward processing, output projection with softmax, and auto-regressive feeding back of the sampled token §3.1 fig. 1. They then cover training: pre-training on next-token prediction §3.2.1, fine-tuning and RLHF §3.2.2, and AI-supervised methods such as process reward models, distillation and Constitutional AI §3.2.3. Section 3.4 lists five enhancement strategies that turn a chat model into an "agent": extended memory, planning (chain of thought, tree of thought), reflexive evaluation, tool use, and multi-agent interaction §3.4.
The core of the loop is summarised below. It is a paraphrase of the seven steps in §3.1 and fig. 1, not a published algorithm.
# Auto-regressive generation of a reply [§3.1] [fig. 1]
# Input: prompt (text)
# Output: reply (text)
def generate(prompt):
# Repeat steps 1-6, one token at a time [§3.1 step 7]
tokens = tokenize(prompt) # ids from a 50257 vocab
reply = []
while True:
# Embed ids to 12288-dim vectors, cap input at 2048 [§3.1 step 2]
matrix = embed(tokens[-2048:]) # (<=2048, 12288)
# Add positional encoding for word order [§3.1 step 3]
matrix = matrix + positional_encoding(matrix)
# 96 blocks, each with attention and a feed-forward net [§3.1 steps 4-5]
for block in blocks:
matrix = block(matrix)
# Project to one score per vocabulary token [§3.1 step 6]
# The paper does not say which row supplies the next token.
scores = project(matrix) # (rows, 50257)
probs = softmax(scores)
# Sample the next token; the paper gives no top-k value [§3.1 step 6]
next_token = sample(probs)
if next_token == END_OF_SEQUENCE:
break
# Feed the new token back as input [§3.1 step 7]
tokens.append(next_token)
reply.append(next_token)
return detokenize(reply)Part B: agency test (§4). The authors adopt the three-condition account of agency from Barandiaran et al. (2009) §4: an agent is self-individuating, generates its own norms, and is the asymmetric source of its interaction with the environment §4. They apply it to ChatGPT, one condition at a time.
# Agency test from embodied and enactive theory [§4]
# Input: a system and its coupling with its environment
# Output: whether all three conditions hold
def is_autonomous_agent(system):
# Individuality: the system produces or sustains itself [§4]
individual = self_produces_itself(system)
# Normativity: the system generates its own norms and goals [§4]
normative = generates_own_norms(system)
# Interactional asymmetry: the system is the source of its actions [§4]
asymmetric = is_source_of_interaction(system)
# All three conditions are required [§4]
return individual and normative and asymmetric
# Paper's verdict for an LLM: individuality not met, normativity not met,
# asymmetry partly met; so not autonomous agents [§4] [abstract]Part C: positive characterisation (§5). The authors separate operations (externally defined) from actions (internally defined) and call the shared ground "tasks" §5 tab. 1. They describe transformers as digital language automata, specifically interlocutor automata that act as a "library-that-talks" §5.1. The paper's metaphors are offered as heuristic framings §5.1 footnote. Section 5.2 names two embodiments the authors consider under-studied: the training corpus as a material body, and the human and ecological labour and energy that sustain the system §5.2. Section 5.3 distinguishes assisted, extended and midtended agency. Midtention is their name for a hybrid process in which a generative system intervenes in a person's ongoing writing or design and the result is still taken as the person's own §5.3.
Section 6 argues that the view should avoid deflationary "stochastic parrot" framings and inflationary "agency without intelligence" framings alike, and calls for transparency about training and resource use §6.2 §6.3.
Hyperparameters
The paper reports GPT-3 architecture figures, not hyperparameters of any experiment it runs. Values below are as stated in the paper.
| Name | Value | Where | Notes |
|---|---|---|---|
| transformer blocks | 96 | §3.1 step 3 fig. 1 | processed in sequence |
| attention heads per block | 96 | §3.1 step 4 fig. 1 | run in parallel; outputs combined |
| embedding / model width | 12288 | §3.1 step 2 fig. 1 | dimensions per token |
| vocabulary size | 50257 | §3.1 steps 1-2 | tokens |
| maximum input length | 2048 | §3.1 step 2 | tokens; the paper calls this the context limit |
| feed-forward network | 3 layers | §3.1 step 5 | input, expanding hidden, reducing output |
| parameters (largest GPT-3) | 175 billion | §3.3 | |
| attention share of parameters | over 30% | §3.1 step 5 | citing Huben et al. (2023) |
| feed-forward share of parameters | about 70% | §3.1 step 5 | same source |
| pre-training data | about 300 billion tokens, about 570 GB | §3.3 | author-reported, from Brown et al. (2020) |
| pre-training compute | 3.14 × 10^23 FLOPs | §3.2.1 | from Brown et al. (2020), appendix D |
| tokens per English word | about 0.75 word per token; 4 characters | §3.1 step 1 | average |
| sampling "heat" (temperature) | not stated | §3 | described qualitatively only |
| top-k in output selection | not stated | §3.1 step 6 | paper says "top-k" without a value |
| optimiser, learning rate, batch size, schedule | not stated | §3.2.1 | "an optimization algorithm" only |
| RLHF reward model and PPO-style settings | not stated | §3.2.2 | |
| reasoning-data scale (DeepSeek distillation) | about 800K reasoning chains | §3.2.3 footnote | compared with 75K examples cited for another team |
Results
Author-reported. The paper runs no experiments and reports no new benchmark scores. Its quantitative statements are the GPT-3 figures above and the authors' own estimates in §3.3, which are shown below with the assumptions the authors state. The conclusion is a conceptual verdict, not a measured result.
| Quantity | Value | Basis | Source |
|---|---|---|---|
| FLOPs to process a 2000-token exchange (250 out, 1750 in) on 175B parameters | about 350 trillion | cost roughly proportional to parameter count, per Kaplan et al. (2020) | §3.3 |
| Time for a human to do those FLOPs | about 500 million years | 10 s per 5-digit multiplication, 40 h per week | §3.3 |
| Time for a human to do training FLOPs | 4.19 × 10^17 years | same assumptions as above | §3.3 |
| Human reading time for the training data | "more than 500 years" | about 200 billion words, 200 words per minute, 8 h per day | §3.3 |
| Benchmark claims in §2 | cited from other works | author-reported, several contested | §2 §2 footnote |
| Verdict on agency conditions | fails individuality and normativity; partly meets interactional asymmetry | conceptual analysis | §4 [abstract] |
Paper vs code
No official code. The paper links only arXiv, a Springer article for the journal version, an OpenAI tokenizer page and a benchmark site §1 §2 §3.1. No project page is linked. A GitHub repository search for "transforming agency llm" returned one unrelated repository, and a search for "barandiaran agency" returned none. The paper contains no implementation to compare with.
Pitfalls
- Not an empirical paper. Every conclusion about agency is an interpretation under a chosen theory §4. An implementer should not treat the verdicts as measurements of a particular model. The paper itself calls its agent evaluation "an open empirical issue" §3.4.
- GPT-3 is not the model you use. The architecture figures describe GPT-3. The paper says GPT-4 and later OpenAI models are largely undisclosed §3.1. Its claim that GPT-4 used mixture of experts is labelled as speculation §3.1.
- "Word" and "token" are used as synonyms. The paper says so §3.1 step 1. The vocabulary of 50,257 entries is tokens, not words. The "word count" figures in §3.3 inherit this blur.
- "Embedding" names two different objects. The paper uses it for the vocabulary lookup table and for the 2048 × 12288 matrix that is later processed §3.1 step 2 footnote. It asks readers to call the second one "matrix".
- Which position gives the next token is unstated. §3.1 step 6 says the processed matrix is projected to scores over the vocabulary, but does not say which row is used. Do not infer the convention from §3.1 alone.
- Sampling and decoding are underspecified. The "heat" parameter is described qualitatively §3. The top-k value, temperature default, optimiser, learning rate and batch size are not stated §3.1 §3.2.1.
- The training loss is described informally. The paper compares the predicted distribution to a target that is 1 for the correct next token and 0 elsewhere §3.2.1. It does not name a loss function, so a reimplementation must take the standard next-token objective from elsewhere. The paper also calls next-token prediction "unsupervised" §3.2.1.
- Typos. The paper writes "RFHF" for RLHF §3.2.2 and "GPT'S" §3.1 step 7.
- §3.3 arithmetic depends on stated assumptions. Recomputing from the inputs given, 350 trillion FLOPs at 10 s each over 40-hour weeks is about 4.7 × 10^8 years, consistent with "around 500 million". The reading-time figure is conservative: 200 billion words at 200 words per minute, 8 hours a day, is about 5,700 years, not 500. The FLOP-per-token factor is an approximation.
- Metadata abstract differs from the TeX. The arXiv metadata abstract omits "digital extended interface embodiments" and "machine automatism" that appear in the TeX abstract §abstract. The card cites the arXiv v3 source fetched.
- Journal version not checked. The paper cites a 2025 journal article in Phenomenology and the Cognitive Sciences. This card uses the arXiv v3 TeX only.
- Agent extensions are named, not evaluated. AutoGPT, AutoGen and similar projects are listed §3.4, but the paper gives no details of how they work or how well they perform.
- Figures. Figure 1 is the authors' schematic of GPT-3 processing. It is not reproduced here (figures from papers are not supported). Read the caption and §3.1 for the steps.
Lineage
- Builds on: Barandiaran et al. (2009), the three-condition account of agency applied in §4 §4; Simondon's "associated milieu" and "mode of existence" §1 §6.1; embodied and enactive theory of mind §4 §5.2.
- Compared against: Bender et al. (2021), "stochastic parrots" §1 §6.2; Floridi (2023), "agency without intelligence" §4 §6.2. Vaswani et al. (2017) and Brown et al. (2020) supply the transformer and GPT-3 descriptions §3.
- Superseded by: not stated.
Source and license
Adapted from “Transforming Agency. On the mode of existence of Large Language Models” by Xabier E. Barandiaran, Lola S. Almendros (arXiv:2407.10735), used under CC BY-SA 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. This page is licensed under CC BY-SA 4.0.