A Survey on Large Language Model based Autonomous Agents

In plain words

A language model answers a question and stops. An agent is the same model put inside a loop: it watches what is happening, keeps notes, makes a plan, does something, and then looks at the result again. This paper is a long map of the research on that idea, written in 2023.

The authors suggest that most agent designs share the same parts. A short description says what role the agent plays. A memory stores what happened before. A planner breaks a goal into steps and may revise them after seeing results. An action part turns the decision into something real, such as a search, a tool call or a move in a game. Capabilities come either from adjusting the model's weights or from clever prompts and extra mechanisms around the model.

The paper then walks through where such agents have been used (social simulation, chemistry and other sciences, software and robotics), how people have tried to test them, and which problems remain open. It is a reading guide rather than a new method, and it runs no experiments of its own.

TL;DR

• Survey of LLM-based agents from 2023 §1 fig. 1.
• Unified framework: profile, memory, planning and action modules §2.1 fig. 2.
• Capability acquisition is split into fine-tuning, prompt engineering and mechanism engineering §2.2.
• It reports no experiments and no new benchmark results; use it as a map and taxonomy, not as evidence that one design wins.

Problem

Earlier agent research usually trained agents with limited knowledge in isolated environments, which the authors argue is far from how people learn §1. LLMs carry broad world knowledge and offer a natural-language interface, so they have been used as the central controller of agents §1. The authors note that these agents were proposed one at a time, with little effort to compare them as a whole §1. The paper's goal is to organise that work into a common vocabulary and taxonomy §1 §7.

Method

This is a survey, so the "method" is the taxonomy. It has three parts: how an agent is built (§2.1 architecture and §2.2 capability acquisition), what applications exist (§3), and how agents are evaluated (§4). Challenges are in §6.

Unified architecture §2.1 fig. 2. The framework has four modules. The profiling module sets the agent's role, which is usually written into the prompt §2.1.1. The memory and planning modules place the agent in a dynamic environment, letting it recall past behaviour and plan future actions. The action module turns decisions into outputs §2.1 fig. 2. Profile influences memory and planning, and all three influence action §2.1.

Profiling §2.1.1. Three ways to write profiles: handcrafting, LLM generation from rules and seeds, and dataset alignment from real demographic records §2.1.1. The authors suggest combining them §2.1.1 remark.

Memory §2.1.2.

• Structures: unified memory (short-term only, held in the prompt) or hybrid memory (short-term plus a long-term store that is queried) §2.1.2.
• Formats: natural language, embeddings, databases and structured lists §2.1.2.
• Operations: reading, writing (with duplicate handling and overflow handling) and reflection (summarising memories into higher-level insights) §2.1.2.
• Reading ranks memories by a weighted sum of recency, relevance and importance eq. 1 §2.1.2.

m^* = \arg\max_{m\in M} \left(\alpha s^{rec}(q,m) + \beta s^{rel}(q,m) + \gamma s^{imp}(m)\right) eq. 1

Here q is the query, M the set of memories, and s^{rec}, s^{rel}, s^{imp} the recency, relevance and importance scores. s^{imp} does not depend on q. The weights \alpha, \beta, \gamma are balancing parameters eq. 1. The paper's special cases are \alpha=\gamma=0 (relevance only) and \alpha=\beta=\gamma=1.0 §2.1.2.

# One decision step of the unified framework  [fig. 2] [§2.1]
# Input:  observation (text), profile (text), memory (list of records)
# Output: action (text or tool call)

def agent_step(observation, profile, memory):
    # Write the new observation into memory  [§2.1.2]
    memory.append(observation)
    # Read the memories that score best for this query  [§2.1.2] [eq. 1]
    recalled = top_scoring(memory, query=observation)   # k: not stated
    # Plan the next steps; the profile shapes the prompt  [§2.1.1] [§2.1.3]
    plan = planner.make_plan(profile, recalled, observation)
    # Choose an action from the plan and the action space  [§2.1.4]
    action = choose_action(plan, recalled, action_space)
    return action

The loop above is the framework in code form, not an algorithm from the paper. The paper gives no pseudocode.

Planning §2.1.3. Two families.

• Without feedback: single-path reasoning (Chain of Thought, zero-shot "think step by step", re-prompting, ReWOO, HuggingGPT, SwiftSage); multi-path reasoning (self-consistency, Tree of Thoughts, graph-of-thought, RAP's Monte Carlo tree search); and external planners that translate tasks into formal planning languages §2.1.3 fig. 3.
• With feedback: environment feedback (ReAct's thought–act–observation loop, Voyager's execution and verification signals, LLM-Planner's re-planning), human feedback (Inner Monologue asking for scene descriptions), and model feedback (Self-Refine's output–feedback–refinement loop, Reflexion's verbal feedback) §2.1.3.
• The authors' remark: planning without feedback suits short tasks; planning with feedback handles long-range tasks but needs more design §2.1.3.

Action §2.1.4. Four perspectives: goal (task completion, communication, exploration), production (from memory recall or by following a plan), space (external tools such as APIs, databases and external models, or the LLM's own planning, dialogue and common-sense abilities) and impact (changing the environment, changing the agent's own state, triggering further actions) §2.1.4.

Capability acquisition §2.2 fig. 4. Two classes.

• With fine-tuning: human-annotated datasets, LLM-generated datasets, and real-world datasets. Examples are ToolBench-style tool data generated with ChatGPT and Mind2Web-style web data §2.2.
• Without fine-tuning: prompt engineering, and mechanism engineering. The latter covers trial-and-error with a critic, crowd-sourcing by debate, experience accumulation (skill libraries, stored successful trajectories) and self-driven evolution §2.2.
• Remark: fine-tuning adds task knowledge to the weights but only works with open-source models; prompt and mechanism design spaces are large and hard to search §2.2 remark.

Taxonomy table tab. 1. A table maps about fifty representative systems onto profile, memory, planning, action and capability-acquisition choices, with publication month tab. 1. Cells marked "–" mean "not explicitly discussed in the paper", not "absent" tab. 1 caption.

Applications §3 tab. 2. Social science (psychology, political science and economy, social simulation, jurisprudence, research assistance) §3.1; natural science (documentation and data management, experiment assistance, education) §3.2; engineering (computer science and software, industrial automation, robotics and embodied AI) §3.3. The table lists the systems for each domain tab. 2.

Evaluation §4 tab. 3. Subjective evaluation: human annotation and the Turing test; the authors note cost, inefficiency and population bias, and mention LLMs used as evaluators §4.1. Objective evaluation: metrics (task success, human similarity, efficiency), protocols (real-world simulation, social evaluation, multi-task evaluation, software testing) and benchmarks §4.2. Table 3 lists which protocol and benchmark each system uses tab. 3.

Hyperparameters

The paper has no experiments, so it reports no training hyperparameters. The numeric constants it does state are below.

Results

Not applicable: survey, with no measured results of its own §1. The paper does state the sizes of several resources it describes:

These figures are the authors' descriptions of other papers' resources. They are not results the paper measured.

Paper vs code

No official code. The paper has no implementation: it is a taxonomy and a review. Its abstract points to a curated reference list maintained by the authors at github.com/Paitesanshi/LLM-Agent-Survey. That repository is a list of papers, not agent code, so no code field is set and nothing was cloned.

Pitfalls

• Snapshot date. The growth figure and the tables stop at August 2023 fig. 1 tab. 1 tab. 3. Later agent work is not covered. The fetched source is the journal-format LaTeX of the arXiv version recorded in raw metadata (v7). This card follows that source.
• "–" is not "absent". In tables 1 and 3, a dash means the paper does not discuss that aspect. It does not mean the system lacks it tab. 1 tab. 3.
• Taxonomy, not a method. The four modules and the planning and action categories are the authors' classification. A system placed in a box is described in the paper's own words, and the card does not check those descriptions against the original systems.
• Equation 1 is underspecified. The three scores are not normalised in the paper, and no weights are chosen. An implementer has to pick α, β, γ and the scoring functions; the paper gives only the two special cases eq. 1 §2.1.2.
• Citation key mismatch. ToolBench is cited as qin2023toolllm in the capability section §2.2 and as qin2023bmtools in the evaluation section and table §4.2 tab. 3. Check the bibliography before citing either key.
• Speculation flagged as such. The claim that long-term-only memory is rare is the authors' guess about why it is rarely documented §2.1.2 remark.
• Evaluation caveats. Human evaluation is costly and can carry population bias. LLM-as-judge evaluation is mentioned as a growing alternative, with no reliability numbers in the paper §4.1 remark.
• Figures are described, not reproduced. No images are used on this page. Refer to fig. 2 for the framework diagram and fig. 5 for the applications and evaluation overview.

Lineage

• Builds on: not stated. The paper positions itself against earlier surveys on LLMs and reasoning §5, not against a method it extends.
• Compared against: Zhao et al. (2023) on LLMs in general, Huang and Chang (2023) on reasoning, Mialon et al. (2023) on augmented language models, and Chang et al. (2023) on evaluation §5.
• Superseded by: not stated in the sources used for this card.