---
kind: concept
slug: learning-from-demonstration
title: Learning from demonstration
summary: >-
  Teaching a policy a task from recorded examples instead of hand-coding it. Cards cover planning, behaviour cloning, nearest-neighbour lookup, reward inference, LLM prompting and teleoperation kits. Evidence is author-reported, mostly small or simulated, plus one 100-task real-robot study.
tags: [robot-learning, imitation, manipulation, inverse-reinforcement-learning]
papers: ["1811.06521", "1910.03135", "2105.01767", "2112.01511", "2112.05124", "2202.02005", "2209.05451", "2307.04577", "2307.04721", "2309.02473", "2312.09245", "2410.21795", "2412.10096", "2412.11360", "2506.04982", "2508.05342", "2610.14004"]
related: [imitation-learning, inverse-reinforcement-learning, policy-gradient, in-context-learning, robot-simulation, nearest-neighbor-search, teleoperation, motion-retargeting, zero-shot-learning, reward-modeling, rlhf, vision-language-action-model, scene-graph-generation, low-rank-adaptation, optimal-transport, reward-design, actor-critic, deep-q-learning, neural-symbolic-integration, inverse-kinematics, dense-correspondence, euclidean-equivariant-network]
status: draft
updated_at: 2026-11-01
---

## Overview

Learning from demonstration (LfD) means a learner picks up a task from examples
recorded while someone, usually an expert, performs it. The survey describes it
as a way to avoid explicitly planning the motion, because the robot's trajectory
is learned from the demonstrations [arxiv:2105.01767 §5.1]. Demonstrations also
appear in the sensing and learning part of the survey, where human examples are
said to make end-to-end learning more efficient [arxiv:2105.01767 §3.2].

The other cards use demonstrations in other ways. Ibarz et al. use them to
pretrain a game agent, and fit a reward to pairwise preferences
[arxiv:1811.06521 §2.2]. GF-VLA turns one human RGB-D video into a scene graph
that a language model plans from [arxiv:2508.05342 §4.1]. PerAct trains one
language-conditioned policy by behaviour cloning on keyframe actions
[arxiv:2209.05451 §3.2] [arxiv:2209.05451 §3.4]. BC-Z trains one image-to-action
policy on a real robot across 100 tasks, with demonstrations and human
interventions during rollouts, and is told the task by a language command or a
human video [arxiv:2202.02005 §4] [arxiv:2202.02005 §5.1]. AnyTeleop focuses on how the
demonstrations are recorded, using a camera-based teleoperation system, and then
imitates them [arxiv:2307.04577 §7.1]. A frozen large language model can be
given demonstrations in its prompt and asked to continue them, with no training
[arxiv:2307.04721 §5]. "Showing versus Doing" treats a demonstration as a message
from a teacher, and reports that reward recovery works better from trajectories
chosen to show the task [arxiv:2610.14004 §4.3]. VINN keeps the demonstration
frames and answers each query by averaging the actions of the closest frames,
so no policy is trained after the encoder [arxiv:2112.01511 §3.2].
DriveMLM is trained on recorded expert driving with rule-derived labels
[arxiv:2312.09245 §3.4]. The paper does not describe its method as imitation,
so it is a looser fit than the others. Three more cards use demonstrations to
build a reward or a task structure: TemporalOT compares agent frames with expert
frames under an order-aware optimal-transport cost [arxiv:2410.21795 eq. 5],
the reward-machine paper infers a task graph from unlabelled videos
[arxiv:2412.10096 §4.3], and Visual IRL learns a reward from human keypoints
[arxiv:2412.11360 §IV.A].

The cards share the idea of a policy or trajectory taken from recorded behaviour,
but none gives a formal definition of LfD, and the survey names no standard
algorithm [arxiv:2105.01767 §5.1]. A 2023 survey of imitation learning groups the field
into behavioural cloning, inverse reinforcement learning, adversarial imitation
and imitation from observation [arxiv:2309.02473 §I]. The
[imitation-learning](/concepts/imitation-learning) page covers the supervised
form of this idea in more settings.

## Variants

### Encoding demonstrated trajectories and forces for planning

The robot learns from human demonstrations, and the learned encoding covers
both the robot trajectory and the interaction force. The survey
attributes this approach to Lee et al. (2015) and presents it as one of the
planning methods for deformable objects [arxiv:2105.01767 §5.1]. Lee et al.
is cited in the survey but has no card here, so nothing about its method is
taken from this page. The survey reports no success rate or error figure for
this approach [arxiv:2105.01767 §5.1]. Card:
[Challenges and Outlook in Robotic Manipulation of Deformable Objects](/papers/2105.01767).

### Behaviour cloning from keyframe demonstrations

PerAct learns one multi-task policy for 18 simulated RLBench tasks
[arxiv:2209.05451 tab. 1]. Its demonstrations are reduced to keyframes: an
action is kept when joint velocities are near zero and the gripper state does
not change [arxiv:2209.05451 §3.2]. The paper does not state the velocity
threshold. The policy predicts a voxel for the gripper translation, a rotation
bin per axis, the gripper state and a collision flag, and each head is trained
with cross-entropy [arxiv:2209.05451 §3.4].

With 10 demonstrations per task, the average gain over the 3D U-Net baseline is
1.33x; with 100 it is 2.83x [arxiv:2209.05451 §4.2] [arxiv:2209.05451 tab. 1].
Without the language goal the agent performs at chance [arxiv:2209.05451 §4.2].
Random or fixed-interval keyframes give zero success, so the keyframe filter is
part of the method [arxiv:2209.05451 §4.2]. The real-robot results use 5 to 10
test episodes per task, so they are only indicative [arxiv:2209.05451 tab. 2].
Card: [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](/papers/2209.05451).

### Many-task behaviour cloning with a task embedding

BC-Z conditions one image-to-action policy on a 512-dimensional sentence
embedding of the task, injected through FiLM layers in a ResNet-18 torso
[arxiv:2202.02005 §5.1] [arxiv:2202.02005 §5.3]. The training set has 25,877
demonstrations: 11,108 expert-only and 14,769 collected with HG-DAgger, where a
human takes over when the learned policy goes wrong and the corrections are
added to the data [arxiv:2202.02005 §4]. The policy was redeployed over 16
iterations [arxiv:2202.02005 §4]. The survey files this kind of human takeover
under interactive imitation, which targets the drift of a learner into states
the expert never showed [arxiv:2309.02473 §II].

On held-out tasks, the average success rate is 32% with a language command and
4% with a video command [arxiv:2202.02005 tab. 3]. The authors read the close
training results for one-hot (42%) and language (40%) conditioning as a sign
that the language space is adequate, so the held-out gap sits in the policy
[arxiv:2202.02005 §6.2] [arxiv:2202.02005 tab. 3]. On one task with 1,000
demonstrations, a single-task policy scores 5% and the multi-task language
policy scores 52% [arxiv:2202.02005 tab. 4]. Training the same task with 50%
manual and 50% HG-DAgger episodes reaches 53%, against 27% for 100% manual
demonstrations at the same episode count [arxiv:2202.02005 tab. 4].

The paper states 29 held-out tasks in the text and its table has 28 rows
[arxiv:2202.02005 §4] [arxiv:2202.02005 tab. 2]. The released code covers only
the language-conditioned model. Its learning rate and batch size differ from the
paper's, and it has no video encoder [code: research/bcz/configs/run_train_bc_gtcond_trajectory.gin#L35]
[code: research/bcz/model.py#L63-L65]. Card: [BC-Z](/papers/2202.02005).

### Nearest-neighbour lookup over demonstration frames

VINN has no policy head. Its encoder is a ResNet-50 fine-tuned with BYOL on the
demonstration frames, without action labels [arxiv:2112.01511 §3.1]
[arxiv:2112.01511 §4.3]. At test time the query image is embedded, its k
nearest demonstration frames are found under Euclidean distance, and their
actions are averaged with softmin weights [arxiv:2112.01511 §3.2]. The paper
uses k around 10 and reports no gain beyond k = 20 [arxiv:2112.01511 §4.7].
The weights are exp(-distance), with no temperature stated
[arxiv:2112.01511 §3.2].

On a real Hello Robot Stretch opening three cabinets, 30 trials in total, VINN
opens the door 80% of the time. Behaviour cloning on frozen representations
reaches 53.3%, and end-to-end behaviour cloning reaches 0%
[arxiv:2112.01511 tab. I]. When the cabinet is partly occluded, VINN keeps 70%
with the sign and handle hidden, where the frozen-representation baseline falls
to 10%. This comparison uses 10 trials on one cabinet [arxiv:2112.01511 tab. II].
In the unmodified scene the baseline is better, at 90% against 80%
[arxiv:2112.01511 tab. II].

With few demonstrations, the paper says nearest-neighbour methods are
competitive or better, and that VINN is clearly better on Stacking and Door
Opening below 20 demonstrations. The comparison is shown only as a plot
[arxiv:2112.01511 §4.4] [arxiv:2112.01511 fig. 4]. Each query is compared with
every demonstration embedding, so cost grows with the number of frames. The
paper reports about 0.038 s per query on hardware it does not state, and it
names FAISS as a faster search it did not test [arxiv:2112.01511 §4.8].
Card: [The Surprising Effectiveness of Representation Learning for Visual Imitation](/papers/2112.01511).

### Pose matching to demonstrations with neural descriptor fields

This variant trains no policy. A network trained to reconstruction, without
keypoint labels, gives each 3D point a descriptor, and the descriptor field is
made SE(3)-equivariant with Vector Neurons [arxiv:2112.05124 §II-A]
[arxiv:2112.05124 eq. 5]. A rigid body is represented by a set of query points,
and its pose descriptor is the concatenation of the descriptors of those points
[arxiv:2112.05124 §II-B] [arxiv:2112.05124 eq. 10]. For a new object, a pose is
found by minimising the L1 distance between the candidate's pose descriptor and
the average demonstration descriptor, from random starts, with Adam
[arxiv:2112.05124 eq. 11] [arxiv:2112.05124 §II-C].

In simulation with 10 demonstrations per task, overall success is 0.87 to 0.91
on upright mugs, bowls and bottles, and 0.58 to 0.78 when the objects are in
arbitrary poses. The Dense Object Nets baseline reaches 0.00 to 0.45
[arxiv:2112.05124 tab. I]. Taking the activations of all network layers, rather
than the last or first layer alone, is the largest factor in the ablation: 0.88
against 0.65 for the last layer on upright mugs [arxiv:2112.05124 tab. II]. On
upright mugs, overall success is 0.46 with one demonstration and 0.70 with five
[arxiv:2112.05124 tab. IV].

The method has limits the card records. The real-robot section gives only
qualitative examples [arxiv:2112.05124 §IV-C]. The pose search is non-convex,
and the paper does not state the number of starts or steps
[arxiv:2112.05124 §II-B]. It transfers poses and points, not whole trajectories,
and it assumes rigid objects and a static scene
[arxiv:2112.05124 §III] [arxiv:2112.05124 §VI]. Card:
[Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](/papers/2112.05124).

### Collecting demonstrations by camera teleoperation

This variant is a source of demonstrations, not a learner. AnyTeleop uses a
camera to watch the operator's hand, detects the hand pose, retargets it to the
robot hand joints by optimization, and moves the arm with a GPU motion generator
[arxiv:2307.04577 §4]. Its stated aim is to make demonstration collection cheap
across many robots and settings [arxiv:2307.04577 §1]. RGB-only input works, with
a wrist-position error that the paper does not quantify [arxiv:2307.04577 §4.1].

In simulation, the imitation learner (DAPG) is trained on 50 demonstrations per
task, with three seeds and 100 evaluation trials [arxiv:2307.04577 §7.1]
[arxiv:2307.04577 tab. 3]. AnyTeleop's policy scores higher than the baseline
built from the earlier system's demonstrations in 5 of 6 settings, and higher
than pure RL in all 6 [arxiv:2307.04577 tab. 3]. Several gaps are smaller than
the standard deviations over three seeds [arxiv:2307.04577 tab. 3].

On a real XArm6 with an Allegro hand, one camera and ten runs per task, the
paper reports a higher success rate than the Telekinesis system on 8 tasks and an
equal one on 2 [arxiv:2307.04577 §6.2] [arxiv:2307.04577 tab. 4]. The Telekinesis
numbers are copied from its own paper, not re-run [arxiv:2307.04577 tab. 4].
Mapping human finger keypoints to robot joints is set by hand for each robot
[arxiv:2307.04577 §4.3]. Card:
[AnyTeleop](/papers/2307.04577).

DexPilot is another camera teleoperation system in the cards. Four RGB-D cameras
track the operator's bare hand, and a fingertip-vector cost maps it onto a robot
hand with different joints [arxiv:1910.03135 §7.1]. The card describes no policy
learned from the recorded operator motion. Its learned networks are perception
models, trained on point clouds labelled by a tracker that was seeded with glove
data [arxiv:1910.03135 §6]. The reported results are task completion times on 15
tasks, with 9 failed trials out of 150 [arxiv:1910.03135 app. completion times].
This is therefore a source of control, not evidence about imitation. Card:
[DexPilot](/papers/1910.03135).

### Low-cost glove hardware for recording demonstrations

GEX is also a source of demonstrations, not a learner. It is a teleoperation
kit: a three-finger hand with 11 motor-driven joints, and an exoskeleton glove
with 12. The glove's fingertip keypoints are retargeted to the hand's joints by
optimization [arxiv:2506.04982 §3.2]. The paper says teleoperation for
collecting demonstrations is hard with many-DoF hands, and that its aim is a
cheap kit for more labs [arxiv:2506.04982 §1]. The author-stated cost is USD 600
per device, against above USD 16,000 for the Allegro hand [arxiv:2506.04982 §1].
The paper reports no success rates, latency or baselines. Its results are one
real paper-cup grasp and three simulated tasks, shown qualitatively
[arxiv:2506.04982 §4] [arxiv:2506.04982 §4.1]. The paper gives no learning
result, so this variant has no measured effect on a policy. The retargeting
method in the released code is DexPilot from an external library, not the
optimization written in the paper [code: retargeting/configs/teleop/gx10_right_dexpilot.yml#L2].
The force-feedback part described in the paper is not in the released code
[arxiv:2506.04982 §3.3]. Card: [GEX](/papers/2506.04982).

### Expert driving logs for a language-model planner

DriveMLM fine-tunes a multimodal language model to output one speed decision and
one path decision per step, plus an explanation. A motion planner turns the
decision into controls [arxiv:2312.09245 §3.1]. Its training data are 280 hours
of CARLA driving on 50k routes [arxiv:2312.09245 §4.1]. Experts, human drivers
or agents, drive the challenging scenarios. The decision labels come from
hand-written rules applied to the expert trajectories, and the explanations are
written from scene elements and refined by annotators [arxiv:2312.09245 §3.4].

In closed loop on CARLA Town05 Long, the paper reports a driving score of 76.1
against 71.4 for Apollo, and 0.96 miles per intervention against 0.76
[arxiv:2312.09245 tab. 4]. These are author-reported results from one route set.
The paper does not say whether the baselines were re-run or copied
[arxiv:2312.09245 §4.4]. The data are available only on request, and no code
is released [arxiv:2312.09245 Declarations]. The paper does not state how the
decision definitions are written, so the rule set cannot be rebuilt from it
[arxiv:2312.09245 §3.2] [arxiv:2312.09245 fig. 4]. Card:
[DriveMLM](/papers/2312.09245).

### Scene graphs from one human video, planned by a language model

GF-VLA is a dual-arm system that learns a block-assembly task from one human
RGB-D demonstration. It finds hand and object contacts with entropy and mutual
information over positions, and writes them as a scene graph with keyframes
[arxiv:2508.05342 §4.1] [arxiv:2508.05342 §3.1]. The language-model head
writes a plan of subtasks, each with a reason and a check, and the action
head emits Cartesian poses and gripper states at 5 Hz for the two arms
[arxiv:2508.05342 §4.2] [arxiv:2508.05342 §4.3]. The two heads share one 7B
LLaMA 2 backbone with a separate LoRA adapter each [arxiv:2508.05342 §4.4].

Reported results are for the paper's own tasks. Block-assembly grasp success
is 0.94 and placement success 0.89 on average over 120 trials
[arxiv:2508.05342 tab. 1]. Transfer from one demonstration reaches 0.90 task
success over six variants [arxiv:2508.05342 tab. 3], against 0.79 and 0.76 for
GraspVLA on letter and tower tasks [arxiv:2508.05342 tab. 4]. Removing the
scene graph cuts letter and tower success to 0.48 and 0.35 [arxiv:2508.05342 tab. 2].
Each transfer variant has 10 trials, but the table reports values such as
0.87 and 0.93, which cannot be counts out of 10, and the baseline runs are not
described [arxiv:2508.05342 §5.4] [arxiv:2508.05342 tab. 3]. No code is
released. Card: [Information-Theoretic Graph Fusion with Vision-Language-Action Model for Policy Reasoning and Dual Robotic Control](/papers/2508.05342).

### Demonstrations to bootstrap end-to-end reinforcement learning

End-to-end reinforcement learning maps sensor data straight to robot actions.
The survey says human demonstrations can make this more efficient, and gives
one example: Matas et al. (2018) trained an improved deep deterministic policy
gradient method with 20 demonstrations to manipulate cloth [arxiv:2105.01767
§3.2]. The survey notes that such settings often need a hand-designed cost or
reward, which is the reason for the next variant [arxiv:2105.01767 §3.2]. The
number of 20 demonstrations and the task come from the survey's description of
that work, not from the original paper.

### Demonstrations with a reward learned from preferences

Here the demonstrations are one of two feedback channels. A game agent is first
pretrained on a few expert episodes, and a reward model is then fitted to
pairwise human or synthetic preferences over 25-step clips, and the agent
learns from that reward [arxiv:1811.06521 §2.1] [arxiv:1811.06521 §2.4]. The
demonstrations stay in the replay buffer for the whole run
[arxiv:1811.06521 §2.3]. Most labels are synthetic, generated from the game
score with a 10% annotator-noise model; human labels are reported separately,
with a 5 to 10% mislabelling rate [arxiv:1811.06521 §3.1] [arxiv:1811.06521 app. A.3].
The authors report that the agent beats imitation in 7 of 9 Atari games, with
about 1000 comparisons [arxiv:1811.06521 §4] [arxiv:1811.06521 abstract]. The
feedback has to keep arriving: with a frozen reward model, scores fell in three
games as the agent found loopholes [arxiv:1811.06521 §3.2] [arxiv:1811.06521 fig. 4].
Card: [Reward learning from human preferences and demonstrations in Atari](/papers/1811.06521).

### Recovering the reward from demonstrations

When the reward is hand-designed and hard to get right, the survey suggests
using the demonstrations to recover the reward with
[inverse reinforcement learning](/concepts/inverse-reinforcement-learning)
[arxiv:2105.01767 §3.2]. The survey gives no example result for this variant.
The 2023 survey sorts these methods by how they handle the fact that many
rewards fit the same behaviour: maximum margin, maximum entropy and Bayesian
approaches [arxiv:2309.02473 §III]. It notes that IRL needs repeated
interaction with the environment to estimate the reward, which can be costly
or unsafe in high-risk settings [arxiv:2309.02473 §III]. It reports no
experiments of its own.

"Showing versus Doing" models the case where the demonstrator is teaching. A
teacher chooses trajectories that make the true reward easiest for an observer to
infer [arxiv:2610.14004 §2.2] [arxiv:2610.14004 eq. 4]. The paper applies this
choice to whole trajectories, and sets the observer to assume a demonstrator
who is doing the task with a softmax policy [arxiv:2610.14004 §2.3]
[arxiv:2610.14004 eq. 5]. In two small grid tasks on Mechanical Turk, with 59
and 58 analysed participants, people asked to show the task behaved differently
from people asked to do it. In the first, show-instructed people chose
disambiguating final approaches more often (mean 1.82 against 1.05)
[arxiv:2610.14004 §3.2]. In the second, they looped over the coloured tiles more
(mean 1.38 revisits per trajectory against 0.10) [arxiv:2610.14004 §4.2.2].

Maximum-likelihood IRL recovered the reward better from the showing trajectories
than from the doing trajectories [arxiv:2610.14004 §4.3]
[arxiv:2610.14004 fig. 2]. The distances from the true reward are plotted only,
so the size of the gap is not in the text. Card:
[Showing versus Doing: Teaching by Demonstration](/papers/2610.14004).

### Adversarial IRL on human keypoints, then cobot transfer

Visual IRL takes RGB-D video of a person and finds 3D arm keypoints and object
positions in each frame. The state is the end-effector position, the object
position and the object label, and the action is the end-effector displacement
between two steps [arxiv:2412.11360 §IV.A] [arxiv:2412.11360 §V.A]. Adversarial
IRL (AIRL) fits a discriminator to expert and generated pairs, and the reward is
its log-odds [arxiv:2412.11360 eq. 2] [arxiv:2412.11360 eq. 4]. The policy is
trained with TRPO [arxiv:2412.11360 §IV.A]. The discriminator architecture,
learning rate and discount are not stated [arxiv:2412.11360 §IV.A].

The reward is only half of the method. A separate step maps the human arm pose
to initial cobot joint angles with a learned model, then refines the angles with
inverse kinematics so the gripper reaches the target while the joint changes stay
small [arxiv:2412.11360 §IV.B] [arxiv:2412.11360 eq. 6]. The paper's aim is
motion that looks human, not only motion that reaches the goal
[arxiv:2412.11360 §I].

The paper reports 90.4% learned-behaviour accuracy against 83.3% for a
discrete-input baseline, in the text only [arxiv:2412.11360 §V.D]. On one-object
onion sorting, the neuro-symbolic model takes 14.75 s against 30.87 s for RRT,
with jerkiness of 264.92 against 914.13 degrees [arxiv:2412.11360 tab. 5]. On
bottle pouring it takes 33.80 s against 44.12 s for RRT-connect
[arxiv:2412.11360 tab. 6]. The pouring trigger is a simple tilt, so the task
measures motion, not pouring quality [arxiv:2412.11360 §V.A]. The data come
from one lab with 10 trials per task, and the dataset is not released
[arxiv:2412.11360 §V.D]. Card:
[Visual IRL for Human-Like Robotic Manipulation](/papers/2412.11360).

### Optimal-transport reward from one or two expert videos

TemporalOT learns a policy from one or two expert videos, with no task reward.
Each agent step is relabelled with the negative cost it carries under an
[optimal-transport](/concepts/optimal-transport) plan between agent and expert
frames [arxiv:2410.21795 eq. 5]. Two changes make the reward aware of order.
The cost averages the cosine distance over a short window of consecutive frame
pairs [arxiv:2410.21795 eq. 6], and the plan is restricted to a band around the
diagonal [arxiv:2410.21795 eq. 9]. The encoder is a fixed ImageNet ResNet50
[arxiv:2410.21795 §4.1].

On nine Meta-World tasks with two demonstrations and five seeds, the average
success rate is 61.1, against 35.9 for ADS and 20.8 for the sparse task reward
[arxiv:2410.21795 tab. 1]. These are author-reported numbers, and some standard
deviations are large, for example 42.2 (44.5) for ADS on Basketball
[arxiv:2410.21795 tab. 1]. When the expert is sampled 3x or 4x faster, success
falls on Basketball and Door-open [arxiv:2410.21795 tab. 3]. The method assumes
that the agent and the expert move at a similar speed [arxiv:2410.21795 §3.2.3].

Each trajectory needs one Sinkhorn solve per demonstration, so cost grows with
the number of demonstrations [arxiv:2410.21795 §6]. Wall-clock time is one hour
for state runs and three for pixel runs on one GPU [arxiv:2410.21795 app. B.4].
The repository linked from the paper has no license file, so this page cites
only the paper [Card: Robot Policy Learning with Temporal Optimal Transport
Reward](/papers/2410.21795).

### Reward machine inferred from unlabelled demonstrations

This variant infers the task structure as well as the reward. Frames from
several demonstrations are embedded and clustered with DBSCAN. The frame nearest
each cluster mean becomes a prototype, which gives one state of a reward machine
and one proposition, true when a frame is within a distance threshold of the
prototype [arxiv:2412.10096 §4.3] [arxiv:2412.10096 §4.4]. Transitions are read
from the order of the propositions in each demonstration [arxiv:2412.10096
alg. 1]. A DQN then learns on the inferred machine, with potential-based
shaping [arxiv:2412.10096 §5] [arxiv:2412.10096 eq. 6].

The paper publishes no numeric table. On five simulated Franka Panda block
tasks, learning curves of the inferred-machine agent are similar to those of an
agent given the ground-truth machine, except on Stack-3, where the baseline
plateaus higher [arxiv:2412.10096 fig. 5]. The thresholds and clustering
parameters are tuned per task, and the tuning method is not stated
[arxiv:2412.10096 tab. 1]. The demonstrations are generated by a scripted expert,
and the paper describes them in two different ways [arxiv:2412.10096 §6]. The
method is tested only in simulation [arxiv:2412.10096 §7]. Card:
[Reward Machine Inference for Robotic Manipulation](/papers/2412.10096).

### Demonstrations in the prompt of a frozen LLM

A frozen language model can continue a sequence of numbers, given a few examples
in its prompt, with no training step. "Large Language Models as General Pattern
Machines" applies this to robot motion. In the table-sweeping task, 30
demonstrations are binned into numbers, the first two thirds are given as
context, and the model predicts the last third [arxiv:2307.04721 §5]
[arxiv:2307.04721 app. B.2]. The paper reports that text-davinci-003 has the
lowest variance over 30 trials [arxiv:2307.04721 app. B.2]. For the marker task,
the prompt holds reward-labelled trajectories, and the model writes new ones
expected to score higher [arxiv:2307.04721 §6].

This route has real limits. Each action is one model query, and the paper names
latency, context length and monetary cost as blockers for real systems
[arxiv:2307.04721 §7] [arxiv:2307.04721 §1]. The grid and CartPole experiments
are capped at 1024 tokens of context [arxiv:2307.04721 app. C.2]
[arxiv:2307.04721 app. C.3]. The main results use hosted OpenAI and PaLM models
[arxiv:2307.04721 tab. 1] [arxiv:2307.04721 tab. 2]. Most robot results are
single runs or small samples [arxiv:2307.04721 §5] [arxiv:2307.04721 §6].
Card: [Large Language Models as General Pattern Machines](/papers/2307.04721).

### Reinforcement learning without demonstrations (for contrast)

The survey also describes a reinforcement learning scheme for deformable
objects that does not need initial demonstrations, attributing it to Wu et
al. (2019) [arxiv:2105.01767 §5.1]. It is listed here because a reader choosing
between LfD and RL needs both options. No comparison of the two is reported
in the survey. AnyTeleop's pure RL baseline, which uses no demonstrations, scores
below its demonstration-based policy in all six simulated settings
[arxiv:2307.04577 §7.1] [arxiv:2307.04577 tab. 3].

## When to use which

| Situation | Use | Why |
|---|---|---|
| An expert can perform the task, and a planner over the motion is not wanted | Encoding demonstrated trajectories and forces | LfD avoids explicit planning of the motion [arxiv:2105.01767 §5.1]. No success figure is reported for this approach [arxiv:2105.01767 §5.1] |
| Many tasks described in language, a real robot, and a human who can take over during rollouts | Many-task behaviour cloning with a language embedding | 32% on held-out tasks with language, against 4% with video [arxiv:2202.02005 tab. 3]. Language conditioning on one task with 1,000 demonstrations scores 52% against 5% single-task [arxiv:2202.02005 tab. 4]. The authors report the held-out results on 28 table rows, while the text says 29 tasks |
| Few demonstrations, a camera, and a policy wanted with no training beyond the encoder | Nearest-neighbour lookup over demonstration frames | 80% against 53.3% for frozen-representation behaviour cloning on a real door task [arxiv:2112.01511 tab. I]. Clearly better than end-to-end behaviour cloning below 20 demonstrations on two tasks [arxiv:2112.01511 §4.4]. Lookup cost grows with the frames [arxiv:2112.01511 §4.8] |
| Many demonstrations per task are affordable, with RGB-D cameras and a large GPU budget | Behaviour cloning from keyframes | 1.33x at 10 demonstrations and 2.83x at 100, over a 3D U-Net, in simulation [arxiv:2209.05451 §4.2] [arxiv:2209.05451 tab. 1]. Training took 16 days on 8 V100 GPUs [arxiv:2209.05451 §3.4] |
| Collecting demonstrations is the bottleneck, and one camera is available | Camera teleoperation | RGB-only input works, with depth optional [arxiv:2307.04577 §4.1]. The simulated study uses 50 demonstrations per task [arxiv:2307.04577 §7.1] |
| Collecting demonstrations on a many-DoF hand, with a hardware budget near USD 600 per device and no camera-based tracking | Glove teleoperation kit | USD 600 per device, author-stated [arxiv:2506.04982 §1]. No success rates or baselines are reported, so the kit's effect on a learned policy is not measured [arxiv:2506.04982 §4] |
| A language-model planner is wanted, its decisions must match a rule-based planner's states, and a driving simulator is available | Expert driving logs with rule-derived labels | Driving score 76.1 against 71.4 for Apollo on one CARLA route set, author-reported [arxiv:2312.09245 tab. 4]. Data are available only on request [arxiv:2312.09245 Declarations] |
| End-to-end RL is the chosen learner, and some demonstrations exist | Start RL from demonstrations | The survey says this makes learning more efficient, citing one case with 20 demonstrations [arxiv:2105.01767 §3.2] |
| The reward is hard to write down, and the demonstrator can be asked to show the task rather than just do it | Recover the reward with IRL from showing trajectories | Better reward recovery from showing trajectories in two small grid tasks [arxiv:2610.14004 §4.3]. The survey proposes IRL without a result [arxiv:2105.01767 §3.2] |
| Demonstrations are few, the model is hosted, and a slow control rate is acceptable | Demonstrations in the prompt of a frozen LLM | One model query per action, and latency and cost block real systems [arxiv:2307.04721 §7] |
| The reward is hard to write but a person can compare two clips at a time, and a few expert episodes exist | Demonstrations with a reward learned from preferences | Beats imitation in 7 of 9 Atari games with about 1000 comparisons, author-reported [arxiv:1811.06521 §4]. The reward model must keep getting new comparisons, since a frozen one was exploited in three games [arxiv:1811.06521 §3.2] |
| Multi-step two-arm assembly, one human video available, and a large language model can be hosted | Scene graphs from one human video, planned by a language model | 0.90 task success over six transfer variants, each with 10 trials [arxiv:2508.05342 tab. 3] [arxiv:2508.05342 §5.4]. No official code was found |
| Only one or two expert videos exist, no task reward is written, and expert and agent move at similar speed | Optimal-transport reward from expert videos | 61.1 against 35.9 for ADS in average success on nine Meta-World tasks, author-reported [arxiv:2410.21795 tab. 1]. Success drops when the expert is 3x or 4x faster [arxiv:2410.21795 tab. 3] |
| Human video of a manipulation task exists, the robot's joints and limb lengths differ from the human's, and the motion should look human | Visual IRL with a neuro-symbolic arm mapping | 90.4% learned-behaviour accuracy against 83.3%, in text only [arxiv:2412.11360 §V.D]. Shorter motions than RRT on two tasks [arxiv:2412.11360 tab. 5] [arxiv:2412.11360 tab. 6]. One lab and no released code |
| The task has ordered sub-goals visible in several demonstrations, and a simulator is available for checking | Reward machine inferred from unlabelled demonstrations | Learning curves similar to a ground-truth machine on five simulated tasks, except Stack-3 [arxiv:2412.10096 fig. 5]. No numeric table is published [arxiv:2412.10096 §6] |
| Objects appear in arbitrary poses, depth cameras give point clouds, and a few demonstrations exist; no keypoint labels are wanted | Pose matching with neural descriptor fields | Simulated overall success 0.58 to 0.78 on arbitrary poses, against 0.00 to 0.45 for Dense Object Nets, with 10 demonstrations [arxiv:2112.05124 tab. I]. Real-robot results are qualitative only [arxiv:2112.05124 §IV-C] |
| No demonstrations exist | RL without demonstrations | Pure RL trails the demonstration-based policy in all six settings [arxiv:2307.04577 §7.1] [arxiv:2307.04577 tab. 3] |

The survey gives no thresholds for these choices. The cards test different
tasks, robots, simulators and metrics, so their numbers do not rank the variants
against each other. A preference between them needs a comparison on your own task.

## Implementation sketch

The sketch shows the common core of the trained variants, not any one paper's
code. The LLM-prompt variant has no training step and is left out. The
pose-matching variant has no policy, so it appears only as a search. Where the
variants differ, the comment says so.

```pseudocode
# Collect expert demonstrations; teleoperation is one way to record them
# [arxiv:2307.04577 §4] [arxiv:2105.01767 §5.1]
demos = record_expert_runs(task, n=num_demos)

# Keep keyframes only: joint velocity near zero, gripper unchanged
# [arxiv:2209.05451 §3.2]; the threshold is not stated
keyframes = filter_keyframes(demos)

# Planning variant: fit a trajectory and force model, then roll it out
# [arxiv:2105.01767 §5.1]
model = fit_trajectory_and_force(demos)
plan = model.rollout(initial_state)

# Behaviour cloning variant: cross-entropy on keyframe actions
# [arxiv:2209.05451 §3.4]
policy = train_cross_entropy(policy, keyframes)

# Many-task variant: a task embedding conditions the policy, and human
# interventions during rollouts join the data [arxiv:2202.02005 §5.3] [§4]
policy = train_conditioned(policy, demos, task=sentence_encoder(command))

# Lookup variant: fine-tune the encoder on the frames, no action labels,
# then average the actions of the k nearest frames [arxiv:2112.01511 §3.2]
encoder = finetune_byol(encoder, demos.frames)
neighbours = k_nearest(encoder(demos.frames), encoder(query), k=10)
action = softmax(-neighbours.dist) @ demos.actions[neighbours.idx]

# Pose-matching variant: no policy; search for the pose whose query
# descriptors match the averaged demonstration descriptor [arxiv:2112.05124 eq. 11]
target = mean([pose_descriptor(d.pick, d.cloud, Q) for d in demos])
pose = solve_pose(cloud, target, Q)

# RL bootstrap variant: start RL from the demonstrations
# [arxiv:2105.01767 §3.2]
policy = pretrain_on(demos)
policy = reinforce(policy, env, reward)

# Preference variant: fit the reward to pairwise labels on clips, then RL
# on it; the labels must keep coming during training [arxiv:1811.06521 §2.4]
reward = fit_reward_from_preferences(clip_pairs, labels)
policy = reinforce(policy, env, reward)

# Reward recovery variant: infer a reward, then run RL on it
# [arxiv:2105.01767 §3.2] [arxiv:2610.14004 §4.3]
reward = inverse_rl(demos)
policy = reinforce(policy, env, reward)
```

## Contradictions

None known between the cards on any reported number. The cards differ in what a
demonstration is. The survey assumes an expert performing the task
[arxiv:2105.01767 §5.1]. "Showing versus Doing" argues that a demonstration given
to teach carries different information from one given to do, and its
experiments support that for reward recovery [arxiv:2610.14004 §4.3].

Within one card, the reported count of games differs. The Ibarz et al. abstract
says the method beats imitation in 7 of 9 Atari games [arxiv:1811.06521 abstract],
while its results section says 8 of 9, excluding only Private Eye
[arxiv:1811.06521 §3.1]. The card treats 7 as the conservative figure.

Several results are not comparable across cards. VINN and PerAct test different
robots, tasks and metrics [arxiv:2112.01511 tab. I] [arxiv:2209.05451 tab. 1].
Neural Descriptor Fields reports simulated pick-and-place of mugs, bowls and
bottles against Dense Object Nets, so its numbers cannot be ranked against
either [arxiv:2112.05124 tab. I].
BC-Z's 32% is a held-out success rate on real-robot tasks, and PerAct's 1.33x and
2.83x are gains over a baseline in simulation, so the two cannot be ranked
[arxiv:2202.02005 tab. 3] [arxiv:2209.05451 tab. 1]. The VINN paper also reports that frozen-representation behaviour cloning beats
VINN in the unmodified scene [arxiv:2112.01511 tab. II]. The newer cards are
not comparable with these either: TemporalOT reports success rates in
simulation, Visual IRL reports time and jerkiness on real cobots, and the
reward-machine paper shows curves only [arxiv:2410.21795 tab. 1]
[arxiv:2412.11360 tab. 5] [arxiv:2412.10096 fig. 5].

TemporalOT's own tables disagree on one number. Table 3 gives 89.0 for
Button-press at 1x, while Table 1 gives 92.4 for TemporalOT there and 89.0 for
ADS [arxiv:2410.21795 tab. 3] [arxiv:2410.21795 tab. 1]. The card treats the
Table 1 value as the TemporalOT result.

## Papers

- [Reward learning from human preferences and demonstrations in Atari](/papers/1811.06521): a reward model fitted to pairwise preferences, combined with expert demonstrations for a DQfD agent on nine Atari games.
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](/papers/2105.01767): a survey and position paper on deformable-object manipulation. It describes learning from demonstration as a planning method and as a way to start end-to-end learning, and reports no experiments of its own.
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](/papers/2202.02005): one image-to-action policy over 100 real-robot tasks, conditioned on language or video, with human interventions in the training data.
- [The Surprising Effectiveness of Representation Learning for Visual Imitation](/papers/2112.01511): VINN, a fine-tuned encoder with a nearest-neighbour action lookup over demonstration frames, with real-robot door-opening results.
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](/papers/2209.05451): keyframe behaviour cloning with a voxel Transformer and language goal, for 18 simulated tasks and 7 real-robot tasks.
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](/papers/2307.04577): camera-based teleoperation for collecting demonstrations, with an imitation-learning comparison in simulation.
- [DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System](/papers/1910.03135): vision-only teleoperation with four RGB-D cameras and a fingertip-vector retargeting step; no learned policy.
- [GEX: Democratizing Dexterity with Fully-Actuated Dexterous Hand and Exoskeleton Glove](/papers/2506.04982): a low-cost three-finger hand and exoskeleton glove for teleoperation, with qualitative results only.
- [Large Language Models as General Pattern Machines](/papers/2307.04721): a frozen LLM continues few-shot sequences, including robot trajectories, with no training.
- [DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving](/papers/2312.09245): a multimodal LLM outputs driving decisions for a motion planner, trained on expert CARLA driving with rule-derived labels.
- [Information-Theoretic Graph Fusion with Vision-Language-Action Model for Policy Reasoning and Dual Robotic Control](/papers/2508.05342): a scene graph from one human RGB-D demonstration, planned by a 7B language model with a separate action head for two arms.
- [Showing versus Doing: Teaching by Demonstration](/papers/2610.14004): models demonstrations as teaching, and compares reward recovery from showing and doing trajectories.
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](/papers/2112.05124): point descriptors learned without keypoint labels, used to match demonstrated grasp and placement poses; simulated results with 10 demonstrations, and qualitative real-robot examples.
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](/papers/2309.02473): a survey of behavioural cloning, inverse RL, adversarial and observation-based imitation, with open challenges; it reports no experiments.
- [Robot Policy Learning with Temporal Optimal Transport Reward](/papers/2410.21795): an optimal-transport reward from one or two expert videos, made order-aware with a context window and a temporal band, evaluated on nine Meta-World tasks.
- [Reward Machine Inference for Robotic Manipulation](/papers/2412.10096): a reward machine inferred from unlabelled block-stacking demonstrations by clustering frame features, used to train a DQN in simulation.
- [Visual IRL for Human-Like Robotic Manipulation](/papers/2412.11360): AIRL on human keypoints and object positions, with a neuro-symbolic mapping and inverse kinematics to move the learned motion onto a cobot.