---
kind: concept
slug: neural-symbolic-integration
title: Neural-symbolic integration
summary: >-
  Neural-symbolic integration joins learned representations with explicit symbols and logic, aiming for systems that learn, reason and explain. Fourteen papers map the options, from surveys and program-writing language models to a solver-free loop, a KAN, a mental-state review, a robot method and a logic tensor network. Method results are author-reported. Pick a coupling pattern by constraint.
tags: [neurosymbolic-ai, knowledge-representation, explainable-ai, logic, planning, program-synthesis]
papers: ["1905.06088", "2012.05208", "2210.02875", "2012.05876", "2404.19756", "2412.10390", "1903.12394", "2211.10435", "2304.11477", "2305.12295", "2205.09712", "2601.02378", "2412.11360", "2012.13635"]
related: [explanation-evaluation, causal-inference, counterfactual-explanation, constrained-conditional-models, differentiable-fuzzy-logic, knowledge-graph-reasoning, knowledge-graph-embedding, knowledge-base-question-answering, commonsense-knowledge-injection, in-context-learning, self-consistency, self-refinement, semantic-parsing, tool-use, knowledge-distillation, learnable-activation-functions, informed-machine-learning, chain-of-thought-prompting, llm-planning, multi-hop-reasoning, task-decomposition, world-model, partially-observable-markov-decision-process, inverse-reinforcement-learning, learning-from-demonstration, motion-retargeting]
status: draft
updated_at: 2026-10-31
---

## Overview

Neural networks learn from data but, by the account of the position paper,
are brittle outside their training distribution, hard to explain and
data-hungry [arxiv:2012.05876 §2]. Symbolic systems reason precisely over
facts and rules but learn poorly from raw perception [arxiv:2012.05876 §2]
[arxiv:2012.05876 §4]. Neural-symbolic integration combines the two so that a
system gets the strengths of both [arxiv:2012.05876 §2]. The survey frames
the same goal as a question of representation: which logic is encoded, and
how learning, reasoning and extraction work on that encoding
[arxiv:1905.06088 §1] [arxiv:1905.06088 §2].

The invariant that the position paper asks for is a principled two-way link
between the two representations. Knowledge can be compiled into a network,
and symbolic descriptions can be extracted from a trained one
[arxiv:2012.05876 §2]. Not every variant has both directions. Binder trains
nothing and uses the language model only through prompts and calls inside a
symbolic program [arxiv:2210.02875 §2.1] [arxiv:2210.02875 §2.2]. The
binding-problem paper takes a different position: object-level structure
should be learned inside the network, not handed to a separate symbolic
system [arxiv:2012.05208 §7].

The position paper calls the choice of representation the first design
decision: whether knowledge is held as discrete named symbols (localist) or
as continuous vectors (distributed) [arxiv:2012.05876 §3]. The survey uses
the same kind of split to organise its methods, by representation, learning,
reasoning and explainability [arxiv:1905.06088 §3–§6].

## Variants

The position paper's own taxonomy is Kautz's six types, which the authors
re-read through the localist/distributed distinction and extend in places
[arxiv:2012.05876 §3]. The variants below group those types, and the survey's
methods, by how the symbolic and neural parts are joined.

### Hybrid network and symbolic solver

A network handles perception and a separate symbolic reasoner handles
inference. The two exchange data through an interface, so the solver can be
exact and checkable [arxiv:2012.05876 §4]. The position paper names AlphaGo,
which pairs a network with Monte Carlo tree search, as a loosely coupled
example (Type 2), and the neuro-symbolic concept learner and deepProbLog as
examples where the two parts work on complementary tasks (Type 3)
[arxiv:2012.05876 §3]. The survey's vertical hybrids follow the same pattern:
a logic layer sits over a perception network, as in DeepProbLog, where network
outputs act as neural predicates [arxiv:1905.06088 §4.3]. The authors see
this pattern as the likely practical choice [arxiv:2012.05876 §4].
Card: [Neurosymbolic AI: The 3rd Wave](/papers/2012.05876).

### Neural calls inside a symbolic program

A language model writes a program in a symbolic language such as SQL or
Python. Where the language has no operator for a sub-task, the program holds
a call back to the same model, asking a small question about a few cells or
columns. The program is executed, and several sampled programs vote on the
answer [arxiv:2210.02875 §2.1] [arxiv:2210.02875 §2.2]. The method needs only
a few annotated program exemplars and no training [arxiv:2210.02875 §3.1].
Card: [Binding Language Models in Symbolic Languages](/papers/2210.02875).

### Language model writes a program or problem for a solver

The model does not produce the answer. It writes a formal artifact, and a
fixed engine computes the result. In PAL the artifact is a short Python
program with each step written as a comment, and the interpreter's output is
the answer. The method uses a few worked examples and no training
[arxiv:2211.10435 §3]. In LLM+P the model writes a PDDL problem file from a
task description, guided by one example pair. The classical planner Fast
Downward then finds a plan with its optimal setting, and the domain file is
written in advance for each domain [arxiv:2304.11477 §3.2]
[arxiv:2304.11477 §3.3]. In both, the answer is only as good as the artifact,
since the interpreter or planner is exact [arxiv:2211.10435 §6]
[arxiv:2304.11477 §5.3]. Cards:
[PAL: Program-aided Language Models](/papers/2211.10435),
[LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](/papers/2304.11477).

Logic-LM applies the same split to four formal languages: logic programs,
first-order logic, constraint satisfaction, and analytical puzzles posed as
constraints. Each has its own solver, from Pyke and Prover9 to
python-constraint and Z3 [arxiv:2305.12295 §3.1] [arxiv:2305.12295 §3.2].
The paper's argument is that the model only has to represent the problem, not
solve it [arxiv:2305.12295 §3], and that the answer is faithful only as far as
the formulation is correct [arxiv:2305.12295 §3]. When the solver reports an
execution error, the program, the message and examples of common errors go
back to the model for a revision, up to a limit [arxiv:2305.12295 §3.3]. Card:
[Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](/papers/2305.12295).

### Select-then-infer loop without a solver

Selection-Inference has no solver at all. The language model is the only
engine, and the neural and symbolic parts are split into two calls per step
[arxiv:2205.09712 §4]. The selection call picks facts from the context. It
scores candidate facts rather than generating them, because the authors found
that free generation invented facts [arxiv:2205.09712 §4.1]. The inference
call then sees only the selected facts, not the question, and writes one new
fact, which is appended to the context [arxiv:2205.09712 §4.2]. The number of
steps is fixed, and the paper leaves a learned stop to future work
[arxiv:2205.09712 §4] [arxiv:2205.09712 §7]. No step is checked by an exact
engine, so the trace is readable but not verified. Card:
[Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](/papers/2205.09712).

### Knowledge compiled into the network

Symbolic knowledge is translated into the training data or into the initial
architecture and weights of a localist network. The position paper extends
this type from if-then rules to general symbolic knowledge
[arxiv:2012.05876 §3]. In the compiler view, the knowledge either sets initial
weights or adds a knowledge-based penalty to the loss [arxiv:2012.05876 §2].
The survey lists early examples such as KBANN, which stacks one perceptron per
implication, and CILP [arxiv:1905.06088 §3.1.1]. It also describes horizontal
hybrid learning, in which rules are combined with data during training, and
self-transfer, where rules extracted from a teacher network regularise a
student [arxiv:1905.06088 §4.2]. The informed-machine-learning survey gives
the same kind of example: rules encoded in a network with KBANN, and a
knowledge term added to the usual loss [arxiv:1903.12394 §5.5]
[arxiv:1903.12394 §5.1]. It reports that logic rules often enter the
hypothesis set, while simulation results most often enter the training data
[arxiv:1903.12394 §3.2.3]. Cards:
[Neurosymbolic AI: The 3rd Wave](/papers/2012.05876),
[Neural-Symbolic Computing: An Effective Methodology](/papers/1905.06088),
[Informed Machine Learning: A Taxonomy and Survey](/papers/1903.12394).

### Logic as differentiable constraints

First-order formulas become differentiable constraints with truth values in
[0, 1], so they shape an embedding during training and the trained network
can be queried for whether a statement holds [arxiv:2012.05876 §2]
[arxiv:2012.05876 §3]. Logic Tensor Networks are the main example, and
tensor product representations are named as a related approach
[arxiv:2012.05876 §3]. The survey places LTNs among its tensorised
first-order methods, as an extension of Neural Tensor Networks to whole
formulas [arxiv:1905.06088 §3.2.2]. In this type a rule acts as a soft
constraint, or regulariser, on the loss [arxiv:2012.05876 §3]. Cards:
[Neurosymbolic AI: The 3rd Wave](/papers/2012.05876),
[Logic Tensor Networks](/papers/2012.13635).

The LTN paper turns the whole knowledge base into one satisfiability score,
which training pushes up by gradient descent. The connectives and quantifiers
use a stable product configuration of fuzzy operators [arxiv:2012.13635 §2.4]
[arxiv:2012.13635 §3.2]. In its digit-addition setup the network sees only
the sum of two images. A rule allows only the digit pairs that add up to that
sum, so the digits are learned from the sums alone [arxiv:2012.13635 §4.4].
The same formalism covers classification, regression and clustering, but only
the digit-addition results have a baseline [arxiv:2012.13635 §4].

### Reasoning inside a neural engine

The aim is symbolic reasoning, ideally combinatorial reasoning, carried out
inside the network itself (Type 6) [arxiv:2012.05876 §3]. The position paper
says that no complete Type 6 system for combinatorial reasoning existed at the
time of writing, and lists goal-directed commonsense and combinatorial
reasoning as an open challenge [arxiv:2012.05876 §3] [arxiv:2012.05876 §6].
The survey describes differentiable forms of this idea: a neural theorem
prover that extends backward chaining, and neural logic machines that take
facts as input tensors and output new facts [arxiv:1905.06088 §4.1]. It also
states that exact maximum satisfiability is NP-hard, without a citation, and
that a neural approximation gives feedforward inference instead
[arxiv:1905.06088 §5.2]. Card:
[Neurosymbolic AI: The 3rd Wave](/papers/2012.05876).

### Rule extraction from a trained network

The decompiler direction: a symbolic description is pulled out of a trained
network [arxiv:2012.05876 §2]. The position paper uses it to support
explanations that show a proof trace, which a raw activation pattern cannot
[arxiv:2012.05876 §5]. Extraction from very large networks, with soundness
guarantees, is listed as the first open challenge [arxiv:2012.05876 §6]. The
survey judges extraction methods on accuracy, fidelity, consistency and
comprehensibility, and reports that global combinatorial search does not
scale to current networks [arxiv:1905.06088 §6.1]. It also reports that
clauses pulled from the first layer of a deep belief network work well, while
higher layers lose accuracy when treated the same way [arxiv:1905.06088 §6.1].

A KAN gives another route to a formula. Each edge is trained with a sparsity
penalty, the network is pruned, and each surviving edge can then be fixed to
a named function such as a sine or an exponential by a grid search and a
linear fit [arxiv:2404.19756 §2.5.1]. The authors apply this by hand to small
examples, for instance a [2,5,1] network that is reduced to [2,1,1] and then
fixed to symbols [arxiv:2404.19756 §2.5.2], and they report recovered formulas
for science problems [arxiv:2404.19756 §4]. The fit is per edge and manual,
and the interpretability claims rest on small examples [arxiv:2404.19756
§2.5.1] [arxiv:2404.19756 §6]. Card: [KAN: Kolmogorov-Arnold Networks](/papers/2404.19756).

### Symbolic beliefs about other agents

An embodied agent that must infer what people believe, want or intend can
use either of two paradigms [arxiv:2601.02378 §4]. In Prompting, a language
model reasons about mental states through prompt design or context
engineering, with no fine-tuning [arxiv:2601.02378 §4.1]. SymbolicToM, which
tracks symbolic beliefs, is one method listed in this group [arxiv:2601.02378
tab. 4]. In Model-Based Inference, an explicit probabilistic model of the
other agent is inverted to infer its state from its observed actions
[arxiv:2601.02378 §4.2]. The review's central trade-off is interpretability
against flexibility [arxiv:2601.02378 §3]. It names a combination of symbolic
beliefs and distributed activation vectors, by decoding symbolic belief units
from LLM hidden states, as the most interesting recent direction
[arxiv:2601.02378 §3]. It also names neural-symbolic integration as the main
direction in two forms: neural generation with symbolic verification, and
symbolic guidance with neural fine-tuning [arxiv:2601.02378 §6.1]. Card:
[Modeling the Mental World for Embodied AI: A Comprehensive Review](/papers/2601.02378).

### Object representations learned end to end

The binding-problem paper argues that networks cannot dynamically bind
distributed information into object-like representations, and that this
limits systematic generalisation [arxiv:2012.05208 §2.3]. It splits the
problem into representation, segregation and composition [arxiv:2012.05208
§3] [arxiv:2012.05208 §4] [arxiv:2012.05208 §5]. It judges slot-based
representations the simplest mechanism, while augmented distributed codes and
tensor product representations are harder to build but support more
sophisticated use [arxiv:2012.05208 §3.4]. It argues that object learning
should be largely unsupervised, fully differentiable and learned end to end,
rather than handed to a separate symbolic system [arxiv:2012.05208 §4.4]
[arxiv:2012.05208 §7]. The paper has no experiments, so these are design
arguments. Card:
[On the Binding Problem in Artificial Neural Networks](/papers/2012.05208).

### Neural-symbolic search over knowledge graphs

Knowledge-graph reasoning mixes the two parts in a different way. For a single
missing fact, the survey lists symbolic rules, embeddings, and neural-symbolic
variants such as neural rule learning, which learns chains of relations with
a recurrent network [arxiv:2412.10390 §3.1] [arxiv:2412.10390 §3.2.2]. For
logical queries, a link predictor gives each triple a fuzzy truth value in
[0, 1], and a solver searches for the entity assignments that satisfy the
query. Beam search covers acyclic query graphs, and a continuous relaxation in
embedding space allows gradient-based search [arxiv:2412.10390 §4.3]. The
survey reports no numbers for these methods. Card:
[Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective](/papers/2412.10390).

### Learned mapping between two embodiments

A robot learns from a person's video. The reward and policy come from
adversarial inverse reinforcement learning, fed with 3D human keypoints and
object positions [arxiv:2412.11360 §IV.A]. A learned model then maps the
human arm pose to initial joint angles on a cobot, using a fixed one-to-one
joint correspondence, and an inverse-kinematics step refines the angles on a
learned forward-kinematics model [arxiv:2412.11360 §IV.B] [fig. 3]. The
authors call this mapping neuro-symbolic, but its symbolic part is the joint
table, which the text describes without equations [arxiv:2412.11360 §IV.B].
Against RRT and RRT-connect, the authors report shorter and smoother motion
on two cobots [arxiv:2412.11360 tab. 5] [arxiv:2412.11360 tab. 6]. Card:
[Visual IRL for Human-Like Robotic Manipulation](/papers/2412.11360).

## When to use which

The position papers and surveys give design arguments, not measured
comparisons. Binder, PAL, LLM+P, Logic-LM, Selection-Inference and KAN report measured
numbers, but only their authors' own. The rows below follow the stated constraints. Test each
one against your own task before relying on it.

| Situation | Use | Why |
|---|---|---|
| Expert rules are precise and exact reasoning is required | Symbolic reasoning over expert knowledge; compare with deep learning [arxiv:2012.05876 §4] | The paper says no neurosymbolic step is needed in this case, and that any approach should be compared with deep learning [arxiv:2012.05876 §4] |
| Perception must be learned from data and the rules are exact | Hybrid network plus solver, through a neurosymbolic interface [arxiv:2012.05876 §4] | The paper names this as the likely practical option [arxiv:2012.05876 §4] |
| Knowledge is commonsense or soft and must shape training | Differentiable constraints on the loss [arxiv:2012.05876 §6] | The paper says differentiable reasoning fits commonsense knowledge, while exact reasoning fits expert knowledge [arxiv:2012.05876 §6] |
| Expert knowledge is available and reasoning is exact | Keep it symbolic; do not back-propagate through it [arxiv:2012.05876 §6] | The paper advises against back-propagating through expert knowledge [arxiv:2012.05876 §6] |
| Training data is scarce, but symbolic knowledge is available | Knowledge compiled into the network, or horizontal hybrid learning [arxiv:1905.06088 §4.2] | The survey says symbolic knowledge can help when data is scarce [arxiv:1905.06088 §4.2] |
| A trained network must be audited or explained | Rule extraction, judged by fidelity to the model [arxiv:2012.05876 §5] | Explanations should be judged on how closely they match the model; the paper argues that many local methods have low fidelity [arxiv:2012.05876 §5] |
| A universal statement must be learned from data | Prefer the symbolic form where it is known; treat a learned rule as a revisable generalisation [arxiv:2012.05876 §5] | The paper says universal statements are easy to reason about but hard to learn, and that a learned one is data-dependent [arxiv:2012.05876 §2] [arxiv:2012.05876 §5] |
| Existential facts can be learned from few examples | Learn them with the network [arxiv:2012.05876 §2] | The paper says existential statements are easy to learn from one example but harder to reason with [arxiv:2012.05876 §2] |
| A language model is available, the question is over a table or database, no training data exists, and the query language lacks an operator the question needs | Neural calls inside a symbolic program [arxiv:2210.02875 §2.1] [arxiv:2210.02875 §3.1] | Author-reported: 65.0 against 60.2 overall on WikiTQ dev for symbolic SQL alone, and 41.3 against 31.2 on the program-unsolvable subset [arxiv:2210.02875 §4.1]. It takes about six hours per 1,000 examples at 200 queries per minute [arxiv:2210.02875 §3.1]. The model used is retired; see Limits of the evidence |
| Tables are large or contain distractor cells | Give each model call only the sub-table it needs [arxiv:2210.02875 §2.3] | Author's test: with 15% distractor cells, Binder drops 1.3 points and end-to-end QA drops 6.7 [arxiv:2210.02875 §4.3]. Expanded-table results appear only in figures [arxiv:2210.02875 §4.3] |
| A small, smooth scientific function must become a readable formula, and a human can name candidate functions | KAN with pruning and manual symbolic fitting [arxiv:2404.19756 §2.5.1] [arxiv:2404.19756 §2.5.2] | Author-reported on small examples [arxiv:2404.19756 §4]. On the Feynman table, KANs do not beat MLPs on average [arxiv:2404.19756 §3.3]. KAN training was about 10 times slower than MLP training in the authors' PDE setup [arxiv:2404.19756 §3.4] |
| Facts sit in an incomplete or noisy knowledge graph, and questions need several hops or logical conditions | Neural-symbolic query solver over a link predictor [arxiv:2412.10390 §4.3] | The survey says symbolic reasoning breaks on missing or noisy facts, while neural reasoning is hard to explain [arxiv:2412.10390 §1]. It reports no numbers, so compare the method papers it cites [arxiv:2412.10390 §1] |
| Premises are in plain text, no formal language or solver is available, and each deduction should be readable | Selection-inference loop over a frozen model, as in Selection-Inference [arxiv:2205.09712 §4] | Author-reported with a 7B model, 5-shot: 58.75 against 41.32 for chain-of-thought, averaged over the 10 tasks in the generative setting [arxiv:2205.09712 §5]. Each step costs one scoring pass per context candidate per selected fact, and one generation; the paper does not state this cost [arxiv:2205.09712 §4.1] |
| An embodied agent must infer a person's beliefs or intentions from observed actions, and the inference must be inspectable | Model-based inference with an explicit model, such as Bayesian inverse planning [arxiv:2601.02378 §4.2] | The review describes model-based inference as explicit and interpretable but costly, and prompting as black-box and easy to apply [arxiv:2601.02378 tab. 5] |
| Object-level structure must generalise to new combinations | Object representations learned end to end [arxiv:2012.05208 §7] | Argued, not measured: the paper runs no experiments [arxiv:2012.05208 §7] |
| Exact arithmetic, counting or state tracking; a code interpreter is available; no training data | Program written by the model and run by an interpreter, as in PAL [arxiv:2211.10435 §3] [arxiv:2211.10435 §5.1] | Author-reported with Codex code-davinci-002: GSM8K 72.0 against 65.6 for chain-of-thought with the same model, and 93.3 against 79.2 on Penguins [arxiv:2211.10435 §5.1] [arxiv:2211.10435 §5.2]. Majority voting over 40 samples means 40 completions per question [arxiv:2211.10435 §5.1] |
| A task has a formal model that an exact solver handles, the domain can be written once, and optimal plans matter | Language model writes a problem file, and an optimal classical planner solves it, as in LLM+P [arxiv:2304.11477 §3.3] [arxiv:2304.11477 §5.2] | Author-reported optimal-alias success: 90 on Blocksworld, 95 on Grippers, 85 on Storage and 0 on Floortile [arxiv:2304.11477 tab. 1]. Direct LLM planning reaches at most 35 on the same tasks [arxiv:2304.11477 tab. 1]. Without the example, the pipeline scores 0 in every domain [arxiv:2304.11477 tab. 1] |
| Premises or constraints can be written in a formal logic, a deterministic solver exists for it, and the model is a capable translator | Language model translates the problem, a solver runs it, and solver errors go back to the model, as in Logic-LM [arxiv:2305.12295 §3] [arxiv:2305.12295 §3.3] | Author-reported with GPT-4: Logic-LM beats standard prompting on all five datasets and chain-of-thought on four [arxiv:2305.12295 tab. 2]. On PrOntoQA, chain-of-thought is higher, 98.79 against 83.20 [arxiv:2305.12295 tab. 2]. Self-refinement adds 5.01 points of execution rate on average, by the paper's count [arxiv:2305.12295 §4.4] |
| Knowledge is prior rules or equations, and the place where it enters training is still open | Choose by the survey's three axes: knowledge source, representation, integration point [arxiv:1903.12394 §3.2.1] | The survey is a map of methods, not a measured comparison; its sources are not listed [arxiv:1903.12394 §3.1.2] |
| A robot must reproduce a person's demonstrated motion on a cobot with different joints, and the motion should look human rather than just reach the goal | Learned pose mapping plus IK refinement, as in Visual IRL [arxiv:2412.11360 §IV.B] | Author-reported: 90.4% learned-behaviour accuracy against 83.3% for a discrete-input baseline, in the text only [arxiv:2412.11360 §V.D]. Motion is 14.75 s against 30.87 s for RRT on onion sorting [arxiv:2412.11360 tab. 5]. Both tasks were run in one lab on one pair of cobots |
| Rules are first-order and soft, and must train jointly with a deep model on classification, regression or clustering | Logic Tensor Networks with the stable product configuration [arxiv:2012.13635 §2.4] [arxiv:2012.13635 §4] | Author-reported: 98.04% against 97.20% for DeepProbLog on single-digit addition with 30,000 training examples, and 93.49% against 92.18% with 3,000 [arxiv:2012.13635 tab. 2]. The other experiments have no reported baseline [arxiv:2012.13635 §4] |

## Implementation sketch

No code is released with the position papers or the surveys. The patterns
below are our own sketches of the papers' descriptions, with the names chosen
by us. The Binder pattern is the hybrid loop with the facts produced on
demand, as calls the program makes back to the model [arxiv:2210.02875 §2.1].
The generate-then-solve pattern covers PAL, LLM+P and Logic-LM. Logic-LM adds
a repair loop, in which a failing program is rewritten from the solver's error
message [arxiv:2305.12295 §3.3].

Hybrid loop: the network perceives, the solver reasons.

```pseudocode
# Hybrid prediction: network perceives, solver reasons  [arxiv:2012.05876 §4]
def hybrid_predict(x, rules, solver):
    # Map the raw input to symbol scores with the trained network  [§4]
    symbols = network.perceive(x)
    # Convert scores to facts through the neurosymbolic interface  [§4]
    facts = to_facts(symbols)
    # Exact reasoning over expert rules; the solver stays symbolic  [§6]
    return solver.query(facts, rules)
```

Soft-constraint training: rules shape the loss.

```pseudocode
# Training with rules as soft constraints  [arxiv:2012.05876 §2] [§6]
def train_step(batch, rules, lam):
    # Standard task loss on the labelled data  [§6]
    loss = task_loss(network(batch.x), batch.y)
    # Penalise rule violations; the paper gives no weighting scheme  [§6]
    loss = loss + lam * rule_violation(network, rules, batch.x)
    return loss
```

Generate-then-solve: the model writes an artifact and an exact engine answers.

```pseudocode
# Generate a formal artifact, then let an exact engine answer
# [arxiv:2211.10435 §3] [arxiv:2304.11477 §3.3]
def generate_then_solve(task, examples, engine):
    # The model writes a program or problem file, not the answer  [§3]
    artifact = llm_generate(build_prompt(examples, task))
    # Reject artifacts the engine cannot parse; malformed problem files
    # are the main failure LLM+P reports  [arxiv:2304.11477 §5.3]
    if not engine.parses(artifact):
        return None
    # The engine's result is the answer, not the model's own sums  [§6]
    return engine.run(artifact)
```

For the Binder and PAL patterns, the model writes code that is then executed.
Run it in a sandbox with a time limit, and treat it as untrusted.

## Contradictions

Disagreements between papers, each side cited. Not resolved here.

- **Where the symbols belong.** The position paper calls a hybrid network and
  solver the likely practical choice [arxiv:2012.05876 §4]. The binding-problem
  paper argues that object learning should be integrated and learned end to
  end, rather than handed to a separate symbolic system [arxiv:2012.05208 §7].
- **Ranking under another evaluator.** Binder reports 64.6 on WikiTQ test
  against 63.3 for OmniTab [arxiv:2210.02875 §3.2]. Under the official
  evaluator, the paper reports 61.9 for Binder against 62.9 for OmniTab
  [arxiv:2210.02875 app. C.2].
- **Stepwise reasoning against chain-of-thought.** Selection-Inference reports
  that its loop beats chain-of-thought with a 7B Gopher model on 10 bAbI and
  ProofWriter tasks [arxiv:2205.09712 §5]. Logic-LM reports that GPT-4 with
  chain-of-thought scores higher than its own method on PrOntoQA, 98.79
  against 83.20 [arxiv:2305.12295 tab. 2]. The models, tasks and prompts
  differ, so the two results do not measure the same setting.
- **Type 6 wording.** The position paper says early neural-symbolic work
  achieved symbolic reasoning inside a neural engine [arxiv:2012.05876 §3],
  and also that no complete Type 6 system for combinatorial reasoning existed
  [arxiv:2012.05876 §3]. The two differ in scope.

## Limits of the evidence

Binder, PAL, LLM+P, Logic-LM, Selection-Inference KAN and Logic Tensor Networks report measured
numbers, all author-reported.
Binder's come from one model, Codex code-davinci-002 [arxiv:2210.02875 §3.1],
which is retired, and the official evaluator changes the ranking against
OmniTab [arxiv:2210.02875 app. C.2]. PAL's come from the same Codex model
[arxiv:2211.10435 §4], and the paper's ablations show the gain depends on the
interpreter [arxiv:2211.10435 §6]. LLM+P's come from GPT-4 with one example per
domain [arxiv:2304.11477 §3.2] [arxiv:2304.11477 §5.1]. Its optimal numbers
hold only where the optimal planner finishes in 200 seconds; timeouts fall
back to a sub-optimal setting shown in parentheses [arxiv:2304.11477 §5.2].
KAN's come from small function-fitting, PDE and science tasks
[arxiv:2404.19756 §3]. Its accuracy and scaling results assume the true
network shape is known [arxiv:2404.19756 §3.1], and for one toy shape the
measured scaling exponent is about G^-3 against the predicted G^-4
[arxiv:2404.19756 §2.4]. Its formula recovery needs a human to name each
candidate function [arxiv:2404.19756 §2.5.1]. Logic-LM's numbers come from
GPT-4, GPT-3.5 and text-davinci-003 on five test sets [arxiv:2305.12295 §4].
Its abstract averages, 39.2% over standard prompting and 18.4% over
chain-of-thought, are attributed to GPT-3.5 in the text [arxiv:2305.12295
§4.1]. The card recomputes them as means over the three base models, and some
§4.1 figures do not follow from the paper's own table; see
[the card](/papers/2305.12295). The released code also differs from the
paper in its fallback when no answer is produced, so the paper's fallback
should not be assumed [arxiv:2305.12295 §4.1]. Selection-Inference's
headline number comes from a 7B Gopher model, 5-shot, on 10 tasks
[arxiv:2205.09712 §5]. Its prompt examples are not specified, and its
baseline figures in §6 differ from those in §5, so the §5 and §6 numbers
should not be mixed [arxiv:2205.09712 §5] [arxiv:2205.09712 §6]. The
fine-tuned variant is trained on single steps only [arxiv:2205.09712 §6]. The
six papers that are position
papers or surveys run no experiments. Their claims about efficiency,
modularity, explanation and fidelity are arguments [arxiv:2012.05876 §2]
[arxiv:2012.05208 §7] [arxiv:1905.06088 §6] [arxiv:2412.10390 §1]
[arxiv:1903.12394 §3.1.2] [arxiv:2601.02378 §1]. The
position paper's fidelity claim about LIME-style methods cites an earlier
measurement [arxiv:2012.05876 §5]. Its statement that networks can represent
propositional and some non-classical logics, but only part of first-order
logic, is the basis of its first challenge. It should be checked against the
cited work before it is relied on [arxiv:2012.05876 §2] [arxiv:2012.05876 §6].
Logic Tensor Networks' MNIST numbers are the mean of the 10 best of 15 runs,
and the optimiser stalls at initialisation in about one run in five
[arxiv:2012.13635 §4.4]. Its other experiments are shown as plots or described
in words [arxiv:2012.13635 §4.5] [arxiv:2012.13635 §4.7]. The released code
differs from the text in several example settings, such as the regression
learning rate and the smokes embedding size; see [the card](/papers/2012.13635).
Visual IRL's motion results are measured against RRT and RRT-connect only,
and its learned-behaviour accuracy is given in the text without a table
[arxiv:2412.11360 §V.D]. Its object detector was trained on 3000 frames and
tested on 500, all from one lab setup [arxiv:2412.11360 §V.D]. Its AIRL and
TRPO settings are not stated [arxiv:2412.11360 §IV.A]. No official code was
found for it.

## Papers

- [Neurosymbolic AI: The 3rd Wave](/papers/2012.05876): a position paper that maps neural-symbolic work onto Kautz's taxonomy, argues that localist versus distributed representation is the central design choice, and lists three open challenges. It has no experiments.
- [Neural-symbolic computing survey](/papers/1905.06088): a survey that organises neural-symbolic methods by representation, learning, reasoning and explainability. It gives no new results.
- [Binding Language Models in Symbolic Languages](/papers/2210.02875): Binder, a training-free method in which a language model writes SQL or Python programs with calls back to the model, and votes over sampled programs. It reports author-reported table-QA results.
- [On the Binding Problem in Artificial Neural Networks](/papers/2012.05208): a position survey that splits the binding problem into representation, segregation and composition, and argues for learned object-level structure. It has no experiments.
- [KAN: Kolmogorov-Arnold Networks](/papers/2404.19756): an architecture with learnable spline functions on edges, which can be pruned and fitted to named functions to give formulas. It reports author-reported results on small problems.
- [Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective](/papers/2412.10390): a survey of knowledge-graph reasoning by query type, grouped into symbolic, neural and neural-symbolic families. It gives no numbers.
- [PAL: Program-aided Language Models](/papers/2211.10435): few-shot prompting in which the model writes Python with comments and an interpreter gives the answer, with majority voting over samples. It reports author-reported gains over chain-of-thought with Codex on math and symbolic tasks.
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](/papers/2304.11477): GPT-4 writes a PDDL problem file from one example, and an optimal classical planner solves it. It reports author-reported planning success on seven domains.
- [Informed Machine Learning: A Taxonomy and Survey of Integrating Knowledge into Learning Systems](/papers/1903.12394): a survey with a three-axis taxonomy of how prior knowledge enters learning, covering logic rules, equations and invariances. It has no experiments of its own.
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](/papers/2305.12295): an LLM translates a logic problem into one of four formal languages, a deterministic solver runs it, and solver errors trigger revisions. It reports author-reported accuracy on five logical-reasoning datasets.
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](/papers/2205.09712): a frozen LLM loop that selects facts from the context and derives one new fact per step, with no training and no solver. It reports author-reported accuracy on 10 logic tasks with a 7B model.
- [Modeling the Mental World for Embodied AI: A Comprehensive Review](/papers/2601.02378): a survey that separates physical from mental world models, compares prompting and model-based theory-of-mind methods, and catalogues benchmarks. It runs no new experiments and quotes numbers from cited works.
- [Visual IRL for Human-Like Robotic Manipulation](/papers/2412.11360): adversarial inverse RL from human keypoints, with a learned mapping to cobot joints and IK refinement. It reports author-reported results on two cobot tasks against RRT variants.
- [Logic Tensor Networks](/papers/2012.13635): first-order logic over real-valued tensors, where a knowledge base's satisfiability is maximised during training. It reports author-reported MNIST addition results against DeepProbLog, and qualitative results elsewhere.