From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review

In plain words

Large language models are good at answering questions, but a model that can also call tools, look things up and check its own work can carry out multi-step jobs on its own. This survey looks at that shift. It collects the tests people use to measure these models and agents, sorts them into groups, and lists the software frameworks built to make agents, the real-world fields where agents are being tried, and the message formats that let agents and tools talk to each other.

The authors' main complaint is that this material is scattered. The benchmarks do not share a taxonomy, and no single review covers them all. The card's reading is that the paper is a map rather than a method: it tells you what exists and how the pieces are organised, but it does not test any of them.

TL;DR

• A survey, not a method: no experiments, no training recipe, no code.
• Covers about 60 benchmarks from 2019 to 2025, grouped into a taxonomy fig. 2 tab. 4.
• Reviews agent frameworks (LangChain, LlamaIndex, CrewAI, Swarm, and others) and three protocols: MCP, ACP and A2A tab. 5 tab. 12.
• Every number in the paper comes from a cited work and is author-reported there.

Problem

Benchmarks for LLMs and agents are fragmented, and no unified taxonomy or comprehensive survey existed when the paper was written §I. The authors also note that agent-protocol security is largely unexamined §V.F.

The paper assumes that agents need some form of orchestration, tool access and evaluation. It does not state the conditions under which these assumptions fail, and it gives no selection rule for the benchmarks it includes §I tab. 4.

Method

The paper does not describe a search protocol. It states no databases, search strings, inclusion or exclusion rules, or quality checks §I §III. Its structure is: related work §II, a benchmark comparison §III, agent frameworks, applications, protocols, datasets and human-in-the-loop designs §IV, open challenges §V, and a conclusion §VI fig. 1.

The two procedures a reader can implement are the agent loops the paper describes in prose and in its figures. Both are paraphrased below. The paper gives no iteration bound, retry rule or stop condition for either.

Agentic workflow

The paper describes a cycle of planning, acting with tools, and assessing the outcome as satisfactory or not §IV.A fig. 4.

# Agentic workflow  [§IV.A] [fig. 4]
# Input:  user query; tools (search, retrieval, code, ...)
# Output: system reply

def agentic_workflow(query, tools):
    # Form a strategy: break the goal into actionable steps  [§IV.A]
    plan = make_plan(query)
    while True:
        # Carry out the planned steps with the available tools  [§IV.A]
        result = run_steps(plan, tools)
        # Assess the outcome; stop when it is satisfactory  [§IV.A]
        if is_satisfactory(result):
            return result
        # Refine the plan on failure; the paper does not state the rule  [§IV.A]
        plan = refine(plan, result)

Agent-driven RAG

The paper describes an agent that first checks its own store, then breaks the query into sub-questions, retrieves for each one, and checks whether the retrieved material applies before answering §IV.A fig. 5.

# Agent-driven RAG  [§IV.A] [fig. 5]
# Input:  user query; internal knowledge store; retrieval utilities
# Output: answer grounded in retrieved material

def agentic_rag(query, store, retrievers):
    # Answer from the internal store if it already covers the query  [§IV.A]
    if covers(store, query):
        return answer_from(store, query)
    gathered = []
    # Split the query into sub-questions and route each to a retriever  [§IV.A]
    for sub_question in split(query):
        documents = route(sub_question, retrievers)
        # Keep only documents judged applicable  [§IV.A]
        gathered.extend(filter_applicable(documents, sub_question))
    return answer_from(gathered, query)

Benchmark taxonomy

The taxonomy has eight groups. They cover general knowledge, maths, code, retrieval and grounding, domain tests, multimodal and embodied settings, task selection, and interactive agent tests fig. 2. The comparison table records, for each benchmark, year, multimodality, task scope, diversity, reasoning depth and whether it tests agents tab. 4. Those ratings are the authors' judgements and their criteria are not stated.

Protocols

• MCP (Model Context Protocol): introduced by Anthropic in late 2024. A client-server design in which a host connects to several servers. Messages are JSON-RPC 2.0 requests and responses; the host forwards only the context each server declares it can use; errors use JSON-RPC error objects, and the protocol provides cancellation §IV.C tab. 12.
• ACP (Agent Communication Protocol): proposed by IBM Research in 2025 as part of BeeAI. Messages are JSON objects with a role field and a parts array. A session_id keeps state, and reusing it prepends the full turn history to each request. Errors carry code and message; retry is the client's job §IV.C tab. 12.
• A2A (Agent2Agent): introduced by Google in 2025. Built on HTTP, SSE and JSON-RPC 2.0. Agents advertise capability in "Agent Cards". Tasks share an ID, updates stream over SSE, and tasks/resubscribe lets a client reattach to a dropped stream §IV.C tab. 12.

Hyperparameters

Not applicable. The paper trains and runs no model, so it has no optimiser, learning-rate or inference settings. The figures it does report about datasets and benchmarks are listed below for reference.

Results

Author-reported, and the paper runs no experiments of its own. The figures below are quoted from the works the survey cites. Settings are as stated there.

Paper vs code

No official code. A GitHub repository search (owner and title terms) found no repository from the authors or their lab. The paper's only code-like links are to the BeeAI framework, the MCP servers repository and the A2A site, all protocol or framework references rather than the authors' own code. One awesome-list link is commented out in the source and is not used. No discrepancies to check.

Pitfalls

• No experiments. Every number is copied from a cited work and was run under that work's setup. Compare nothing across rows in the Results table.
• Conclusion overreaches the body. The conclusion §VI describes reasoning-training strategies (inference-time scaling, pure RL, SFT plus RL, distillation) and hybrids applied to Qwen-32B and Llama models, citing DeepSeek-R1, o1, o3 and GPT-4o. The body has no section on these methods. Its only reasoning content is Meta Chain-of-Thought §V.A. Do not rely on the conclusion's claims without a primary source.
• Benchmark count. The abstract says about 60 benchmarks. Table 4 has about 66 rows by a hand count. Its year column is also inconsistent: AppWorld is dated 2025 but sits in the 2024 block.
• Dates for frameworks and protocols. The introduction says frameworks span 2023 to 2025 §I. Section IV's opening says 2024 to 2025 §IV. Protocol dates are given per protocol in §IV.C.
• Duplicated paragraph. The Jin et al. paragraph in the software engineering related-work subsection §II.A appears twice in the source with the same wording. Only one copy is meaningful.
• Acronym collision. This card's ACP is the IBM/BeeAI protocol. Other protocols use the same acronym, so state the full name when citing.
• Section and figure numbers are inferred. No compiled PDF was available. Numbers follow the IEEEtran default (Roman for sections, letters for subsections, figures and tables in source order). Float placement in the typeset paper may shift them. Check against the PDF before linking.
• Ratings are subjective. The High, Medium and Low columns in Table 4 come with no stated criteria.
• Agent loops are paraphrases. The pseudocode above restates the paper's prose and figures. It adds no stop rule, retry budget or refinement method, because the paper gives none.
• Figures are not reproduced. The card describes figures in words and cites them, per the no-images rule.

Lineage

• Builds on: not stated. The paper cites prior surveys in its comparison table tab. 1 and related work §II.
• Compared against: 13 earlier surveys in the paper's own overview table, covering software engineering, multi-agent systems, healthcare, game-theoretic agents, GUI agents, personal agents, scientific discovery and chemistry tab. 1. No arXiv ids are given in the paper.
• Superseded by: not known from content in this wiki.