---
kind: concept
slug: sim-to-real-transfer
title: Sim-to-real transfer
summary: >-
  Moving a model trained on synthetic data onto a robot or real images. Twenty-four sources, from humanoid and quadruped locomotion and grasping to mobile manipulation, drone payloads and vision-model testing. Author-reported.
tags: [sim-to-real, robot-learning, locomotion, depth-perception, synthetic-data, manipulation, domain-randomization, gpu-simulation, humanoid-control, grasping, autonomous-parking, gaussian-splatting, teacher-student, bipedal-locomotion, payload-dynamics, off-policy-rl, mobile-manipulation, navigation, dexterous-manipulation, simulation-testing, pipe-inspection, survey, loco-manipulation]
papers: ["1603.01564", "1706.09911", "1810.08705", "2103.14127", "2108.10470", "2204.04340", "2303.03381", "2312.06639", "2312.16217", "2404.06229", "2406.01152", "2406.14947", "2408.07295", "2409.03930", "2410.19277", "2411.04005", "2411.14770", "2412.13621", "2412.15587", "2509.20322", "2512.01996", "2512.12993", "2601.07718", "2605.08713"]
related: [robot-simulation, robotic-grasp-generation, domain-randomization, domain-adaptation, domain-generalization, policy-gradient, imitation-learning, actor-critic, gaussian-splatting, curriculum-learning, motion-retargeting, multimodal-language-model, semantic-segmentation, knowledge-distillation, on-policy-distillation, proximal-policy-optimization, in-context-learning, reward-design, variational-autoencoder, hierarchical-reinforcement-learning, masked-image-modeling, multi-robot-navigation, safety-filter, object-detection, metaheuristic-optimization, partially-observable-markov-decision-process, model-predictive-control]
status: draft
updated_at: 2026-11-01
---

## Overview

A model trained on synthetic data has to work on inputs whose sensors, images,
actuators and contacts differ from the simulator's. This wiki has twenty-four
sources on the topic so far, each in its own setting. A humanoid policy is trained with
reinforcement learning in Isaac Sim and Isaac Lab, then reports zero-shot runs
on a Unitree G1 [arxiv:2601.07718 §4.1] [arxiv:2601.07718 app. B]. A
bipedal robot is trained to walk while carrying a moving load, with the load
modelled in simulation, and its load policies reach lower top speeds on hardware
than in simulation [arxiv:2204.04340 §V] [arxiv:2204.04340 tab. 2]. A
full-size Digit walker is trained in simulation by distilling a state-based
teacher into a student that sees only sensor history, and it walks outdoors
with no falls over a week of full-day tests [arxiv:2303.03381 § Outdoor
deployment]. Street-scene
networks are pre-trained on rendered images and then fine-tuned on real ones
[arxiv:1810.08705 §5.1] [arxiv:1810.08705 tab. 4]. A multimodal language model is
fine-tuned on simulated objects, then run on a real arm, with adaptation after
each real attempt [arxiv:2312.16217 §4.1] [arxiv:2312.16217 §3.3]. A GPU
simulator is used for fast reinforcement learning, and its hand task randomises
physical parameters during training. Its real-robot results come from a different
robot, a TriFinger [arxiv:2108.10470 §6.4.1] [arxiv:2108.10470 §6.4.2]. An
off-policy learning recipe trains a G1 locomotion policy in about 15 minutes on
one consumer GPU, in simulation, and its hardware test covers whole-body
tracking only [arxiv:2512.01996 §3.1] [arxiv:2512.01996 §3.2]. A
grasp network is trained only on simulated grasps and rendered point clouds,
then tested on a real Franka arm with one depth camera
[arxiv:2103.14127 §3.2] [arxiv:2103.14127 §4.3]. A
grasp detector trained on real depth scans is pretrained on simulated grasp
exemplars first [arxiv:1706.09911 §7.3]. A
humanoid loco-manipulation policy is split into a keypoint tracker and a task
generator that reads depth, and it lifts, pushes and kicks objects on a Unitree
G1, with qualitative hardware results only [arxiv:2509.20322 §4.2]. A
second humanoid controller, for a Digit V3 trained in MuJoCo, randomises its
dynamics and raises the difficulty of its torso pushes over a three-stage
curriculum. It fails on 5% of 20 hardware motions, against 0.450% in simulation
[arxiv:2408.07295 §4.4] [arxiv:2408.07295 tab. 3] [arxiv:2408.07295 §5.6]. The
measures used to close the gap fall into a few groups: making simulated input
look like real input, corrupting and delaying observations, correcting a contact
effect the simulator gets wrong, randomising simulator parameters during
training, training on the specific payload that will be carried, pre-training on
synthetic data then fine-tuning on real data, distilling a
state-based teacher into a sensor-only student [arxiv:2303.03381 § Teacher
state-policy supervision] [arxiv:2509.20322 §3.2], distilling a simulated height-map latent into a
depth-camera encoder [arxiv:2512.12993 §IV.C], and adapting at test time from real outcomes
[arxiv:2601.07718 §3.2] [arxiv:2601.07718 §3.3]
[arxiv:2108.10470 app. A.4.1] [arxiv:1810.08705 §5.1] [arxiv:1706.09911 §7.3]
[arxiv:2312.16217 §3.3]. Noise added to simulated depth during training is one
more such measure [arxiv:2103.14127 fig. 6]. A car-parking policy is trained in a Gaussian-splatting
copy of scanned garages and tested on a real vehicle, with a critic that uses
simulator-only maps [arxiv:2605.08713 §4.2] [arxiv:2605.08713 tab. 4]. A mobile manipulator with
a learned arm and base is trained in simulated houses and run in one unseen
real apartment with no further training [arxiv:2312.06639 tab. III]. A
quadruped is trained in simulation with slippery, raised pipe entrances and
random pushes, and then crosses real PVC pipes [arxiv:2412.13621 §II-A4]
[arxiv:2412.13621 tab. V]. A wheeled robot copies a planner in simulation, and
a geometric check guards its motion on hardware [arxiv:2406.14947 §II-C]
[arxiv:2406.14947 §IV-D]. A vision detector is first stress-tested in a
simulated work cell, and its failures are used to fine-tune it before real
images are tried [arxiv:2410.19277 §2.2] [arxiv:2410.19277 tab. 8]. Surveys
name the families of methods but give no transfer numbers [arxiv:2406.01152 §5]
[arxiv:2404.06229 §2.3].
For the choice of simulator itself, see [robot-simulation](/concepts/robot-simulation).

## Variants

The sections below draw on twenty-four papers, and the citations name each source. The
papers use different tasks, benchmarks and metrics, so no number on this page
compares one paper's method with another's. The list shows the parts of each system. The two surveys are summarised in the last subsection.

### Make simulated depth look like real depth

Two image pipelines map simulated and real depth into one perception space
[arxiv:2601.07718 §3.2] [arxiv:2601.07718 fig. 3]. The simulated pipeline adds
range-dependent Gaussian noise, masks contiguous pixels to imitate stereo
matching failures, blurs to imitate motion blur, and with a fixed probability
replaces the whole frame with noise [arxiv:2601.07718 app. A.4]. The real
pipeline crops and resizes to the simulator's geometry, fills zero-valued pixels
caused by occlusion or disparity shadows, and blurs to suppress sensor jitter
[arxiv:2601.07718 app. A.4]. The policy sees several depth frames spaced a fixed
number of steps apart, which covers a window of several steps
[arxiv:2601.07718 app. A.4] [arxiv:2601.07718 eq. 6].

### Distil a height-map latent into a depth-camera student

A Digit walker reads a 16-value latent from a convolutional VAE that encodes a
20 by 40 height map, and the latent is learned in simulation [arxiv:2512.12993 §II.B].
For the real robot, a second encoder is trained on depth images from two
cameras, on the torso and the pelvis. Its latent is pulled toward the
height-map latent with a mean-squared loss, and the VAE loss stays as a
regulariser [arxiv:2512.12993 §IV.C]. Simulated depth is post-processed to look
like the sensor: cropping, blur, edge shadowing, clipping at 2 m, inpainting,
pixel dropout and leg masking [arxiv:2512.12993 §IV.C] [arxiv:2512.12993 fig. 12].

The sim-to-real evidence here is a comparison of simulated and real depth
images. The paper reports no policy run on hardware, and lists hardware
deployment as future work [arxiv:2512.12993 fig. 12] [arxiv:2512.12993 §V].
The policy results are simulation results: above 80% success on every tested
terrain for the height-map policies, and one check on stairs in a second
simulator with realistic dynamics [arxiv:2512.12993 §IV.E] [arxiv:2512.12993 §IV.F].

### Noise and delay on the observations

During training the actor receives each proprioceptive input with uniform noise.
The ranges are ±0.2 rad/s on angular velocity, ±0.05 on projected gravity,
±0.01 rad on joint positions and ±0.5 rad/s on joint velocities
[arxiv:2601.07718 tab. 4]. The critic sees noise-free states and the base linear
velocity [arxiv:2601.07718 tab. 3]. Depth is delayed by one frame in training, to
imitate sensor latency [arxiv:2601.07718 §3.2] [arxiv:2601.07718 app. A.4]. On
hardware, depth is captured at 60 Hz and the policy runs at 50 Hz on the latest
depth frame [arxiv:2601.07718 §3.2] [arxiv:2601.07718 app. B].

### Correct a contact effect the simulator gets wrong

Reinforcement learning agents tend to step on the nearest terrain edge, which saves
energy. Contact in simulation is simplified, so this works there, but on hardware
it causes slips [arxiv:2601.07718 §3.3]. The fix finds sharp edges, models each
as a cylinder, and penalises foot points that penetrate a cylinder, scaled by the
point's speed [arxiv:2601.07718 §3.3] [arxiv:2601.07718 eq. 1]. The weight is
−4.0 [arxiv:2601.07718 tab. 5].

The effect shows on hardware, not in simulation. In simulation, the variant with
the penalty and the variant without it differ by at most 0.13 points on stair up,
stair down and deep gap [arxiv:2601.07718 tab. 1]. On the robot, the penalty
variant reached 10/10, 10/10 and 8/10 on those three terrains, against 8/10, 6/10
and 3/10 without it [arxiv:2601.07718 tab. 1]. The paper does not state how the
"no edge" variant was built [arxiv:2601.07718 §4.2].

### Staged gait policies and switching on the robot

Three specialised policies (stairs, platforms, running) are fine-tuned from a
walking checkpoint, for 40,000 iterations each [arxiv:2601.07718 app. A.8]. On the
robot, switching policies means stopping, standing, then starting the new policy
[arxiv:2601.07718 §4.1].

### Randomise the simulator during training

The Shadow Hand OpenAI task resamples physical parameters at each reset: object
dimensions, object and link masses, surface friction, joint damping, actuator
P-gain, joint limits and gravity [arxiv:2108.10470 tab. 18]. Scaling factors are
drawn uniformly or log-uniformly, and additive terms from normal distributions.
Parameters are re-randomised on each reset, with at least 720 steps between
resets [arxiv:2108.10470 app. A.4.1].

The policy sees only observations that could be measured on a real hand. The
value function also gets privileged simulator state, an asymmetric actor–critic
[arxiv:2108.10470 §6.4.1]. The authors follow the OpenAI randomisation setup, with
changes listed in the same section [arxiv:2108.10470 §6.4.1]. The reported result
is in simulation: 20 consecutive successes in under 1 h on one A100, against about
30 h for OpenAI's CPU-cluster run [arxiv:2108.10470 §6.4.1] [arxiv:2108.10470 fig. 15].
The standard hand task reaches the same milestone in under 35 min without
randomisation, but it uses different control rates and networks, so the two times
cannot isolate the effect of randomisation [arxiv:2108.10470 tab. 4]
[arxiv:2108.10470 tab. 17].

The card records two hardware claims, and neither applies the randomisation
settings above. A TriFinger real-robot task reports a 55% mean success rate,
labelled sim-to-real [arxiv:2108.10470 §6.4.2] [arxiv:2108.10470 fig. 17]. An
ANYmal rough-terrain policy, trained in under 20 min, is labelled sim-to-real
transfer [arxiv:2108.10470 §6.1.4] [arxiv:2108.10470 fig. 10]. The card records no
randomisation settings for either robot.

### Train on simulated grasps with depth noise, then test on a real depth camera

A grasp network is trained with no real grasps. For each training scene, a
virtual camera renders a point cloud, and a point counts as a positive contact
when a valid simulated grasp lies within 5 mm of it [arxiv:2103.14127 §3.2].
Training with Gaussian noise helps generalisation, according to the paper's
figure, which gives no noise level [arxiv:2103.14127 fig. 6]. On a Franka
Panda with an Intel RealSense L515, the authors report 90.20% grasp success on
51 unseen objects in nine cluttered scenes. The best baseline, 6-DoF GraspNet
with CollisionNet, scores 80.39% [arxiv:2103.14127 tab. 1] [arxiv:2103.14127 §4.3].
Card: [Contact-GraspNet](/papers/2103.14127). The hardware test uses one robot
and one depth camera, so it does not show transfer to other sensors
[arxiv:2103.14127 §4.3].

### Distil a state-based teacher into a sensor-history student

The teacher is an MLP trained on the full simulator state, with hidden sizes
[512, 512, 256, 128] [arxiv:2303.03381 § Model architecture]. The student is a
causal transformer. It reads a window of 16 observation-action pairs, one token
per pair, and outputs the next action [arxiv:2303.03381 § Model architecture]. Its
loss adds an RL term to λ times a KL term against the teacher, and λ is annealed
to zero, reaching zero near the middle of training [arxiv:2303.03381 eq. 2]
[arxiv:2303.03381 § Joint optimization with reinforcement learning]. Both terms
are optimised on-policy, with PPO as the RL algorithm [arxiv:2303.03381 § Policy
learning].

Training randomises robot dynamics, control parameters, environment physics,
observation noise and observation delay. Terrains are smooth planes, rough
planes and smooth slopes [arxiv:2303.03381 § Domain randomization]. The
randomisation table and the command ranges are in a supplement that is not in
the arXiv source, so the card records no values [arxiv:2303.03381 § Domain
randomization]. Isaac Gym cannot model Digit's closed leg chains, so the rods are
replaced by a stiff virtual spring with sub-stepping. The stiffness is not stated
[arxiv:2303.03381 § Closed kinematic chain].

A vendor simulator is used as a filter before hardware. It does not change the
network weights, and it only removes unsafe policies [arxiv:2303.03381 § Sim-to-real
transfer]. The policy runs at 50 Hz on hardware, with joint PD control at 1 kHz
[arxiv:2303.03381 § Sim-to-real transfer].

The paper reports no results table. On hardware, the policy walks on rubbers,
cloths, cables and bubble wrap at 0.15 m/s, and crosses slopes up to 8.7%, against
10% in training [arxiv:2303.03381 § Rough terrain]. In a simulator comparison, the
policy is comparable to the vendor's controller on slopes and better on steps
and unstable planks, with values only in a figure [arxiv:2303.03381 fig. 2].
Among student architectures, the transformer is best by a considerable margin,
and a longer context helps on both tested slopes [arxiv:2303.03381 fig. 8]. The
joint objective is best in the same figure [arxiv:2303.03381 fig. 8]. The
paper gives the values of these comparisons only in figures.

### Randomise dynamics and stage the torso pushes on a humanoid

The Digit V3 policy randomises joint damping, link masses, centre-of-mass
positions, encoder noise and ground friction in every curriculum stage
[arxiv:2408.07295 §4.5]. The paper gives no ranges for these, and refers to
earlier Digit locomotion work for them, so the card records no values
[arxiv:2408.07295 §4.5].

The training curriculum has three stages [arxiv:2408.07295 §4.4]. Stage 1 uses
single torso pushes of 80 to 800 N, each lasting one policy step, with a 1%
chance per step. Stage 2 uses continuous pushes of 20 to 150 N, lasting 20 to
50 policy steps. Stage 3 does not state its perturbations
[arxiv:2408.07295 §4.4]. The paper's no-curriculum variant fails on 4.875% of
mimicry motions in simulation, against 0.450% for the full method
[arxiv:2408.07295 tab. 5]. The two variants were not trained for the same number
of steps, and the paper reports neither count [arxiv:2408.07295 §5.5].

On hardware the same policy has an MEPE of 0.117 m, against 0.098 m in
simulation [arxiv:2408.07295 tab. 3]. The Future Work section names wide foot
placements and single-foot stances as sim-to-real gaps still open
[arxiv:2408.07295 §6].

### Off-policy learning with massively parallel simulation

FastSAC and FastTD3 are off-policy learners. They reuse past transitions from a
replay buffer, and they train on thousands of simulated robots at once
[arxiv:2512.01996 §2.1]. The authors name a few settings that matter at this
scale: action bounds set from each joint's limit and default position, layer
normalisation, a critic target that averages the two Q-values, a distributional
critic, and an initial entropy temperature of 0.001 [arxiv:2512.01996 §2.1]
[arxiv:2512.01996 alg. 1]. The reward has fewer than ten terms [arxiv:2512.01996 §2.2].

For locomotion, the randomised quantities are pushes, action delay, PD gains,
mass and friction, and centre of mass on the G1 only [arxiv:2512.01996 §3.1].
Pushes come every 5 to 10 s, and every 1 to 3 s in the strong variant
[arxiv:2512.01996 fig. 3].

The authors report velocity-tracking policies for the G1 and T1 learned in 15
minutes on one RTX 4090 under strong randomisation, and they say PPO trains
more slowly. The numbers are in figures only [arxiv:2512.01996 §3.1]
[arxiv:2512.01996 fig. 3]. Hardware is shown for G1 whole-body tracking, with a dance
motion of over two minutes among others [arxiv:2512.01996 §3.2] [arxiv:2512.01996 fig. 6].
The locomotion curves are simulation results [arxiv:2512.01996 §3.1]. The
released code differs from the paper in several settings, so the paper's
recipe is not the code's default [the card](/papers/2512.01996).

### Train on the specific payload in simulation

A Cassie robot carries one of four loads: a box on a tray, a cart on ropes, a
pole with a swinging weight at each end, or a water jug modelled as a weight on
springs [arxiv:2204.04340 §III-B]. The policy sees only its own joints and body,
and the load changes only the simulated robot [arxiv:2204.04340 §III-B].

Training with the specific load recovers most of the lost performance. For the
carrying pole, the pass rate goes from 0.0014 for the base walker to 0.9578 for
the specific policy [arxiv:2204.04340 tab. 2]. A general policy trained on all
five models is close to the specific ones, and on the tray box, cart and water
jug it is higher [arxiv:2204.04340 tab. 2]. Its pole pass rate is 0.8342
[arxiv:2204.04340 tab. 2].

Starting training from the base walker gives pass rates within about two points
of training from scratch on the tray box, and higher on the other three loads
[arxiv:2204.04340 tab. 3]. The authors say bootstrapped policies need on the order
of 100 million fewer samples to reach the same reward, which they read from a
learning curve [arxiv:2204.04340 §IV-C] [arxiv:2204.04340 fig. 4].

On hardware, the top speed in walk-in-place ramp tests is 3.0 m/s for the base
walker, against 3.9 m/s in simulation. The tray box and carrying pole policies
reach 1.0 and 0.8 m/s, against 2.1 and 2.5 m/s in simulation [arxiv:2204.04340 §V]
[arxiv:2204.04340 tab. 2]. The paper expects dynamics randomisation alone to fall
short with loads [arxiv:2204.04340 §V] [arxiv:2204.04340 §VI]. It does not
describe how the loads were mounted on the hardware [arxiv:2204.04340 §V].

### Pre-train on synthetic data, then fine-tune on real data

Synscapes is a set of 25,000 path-traced street scenes with per-pixel labels,
instance and depth maps [arxiv:1810.08705 §3]. Networks trained on it alone score
well below a Cityscapes-trained baseline on real Cityscapes images: DeepLab v3+
reaches 50.35 mean IoU against 76.56, and FRRN 45.20 against 68.27
[arxiv:1810.08705 tab. 4]. Trained on Synscapes and then fine-tuned on Cityscapes,
the scores rise above the baseline, to 78.85 for DeepLab v3+ and 74.52 for FRRN
[arxiv:1810.08705 tab. 4]. The two comparison synthetic sets also end above their
baselines after fine-tuning, by less: 77.57 and 77.45 for DeepLab v3+
[arxiv:1810.08705 tab. 4].

For detection, a Faster R-CNN trained only on Synscapes reaches mAP 0.092 on the
KITTI validation set, against 0.456 when trained on KITTI. Training on both gives
0.519 [arxiv:1810.08705 tab. 6]. The authors say the segmentation comparison alone
does not prove a smaller domain shift [arxiv:1810.08705 §4.1]. They say the gap
between the synthetic sets is most likely due to sensor simulation and feature
variation [arxiv:1810.08705 §4.2].

Grasp Pose Detection pretrains its classifier on 207,000 simulated grasp
exemplars made from 400 CAD models [arxiv:1706.09911 §7.3]. On the real BigBird
test set, the network trained on these alone reaches 83% accuracy, against
about 90% for real training data [arxiv:1706.09911 §7.3]. Simulated pretraining
followed by real fine-tuning reaches the same accuracy in 4,000 iterations,
against 20,000 without pretraining [arxiv:1706.09911 §7.3] [arxiv:1706.09911 fig. 9].
The robot trials are not described as using this pretraining
[arxiv:1706.09911 §8.6]. Card: [Grasp Pose Detection](/papers/1706.09911).

The earlier grasp-detection paper uses the same two-stage idea. Its 15-channel
classifier is trained only on 207,000 exemplars from 400 CAD models in 3DNET,
for 30,000 iterations, and reaches 83% accuracy on a BigBird test set. Its
best representation, trained on real BigBird data, reaches about 90% on the
same set [arxiv:1603.01564 §4.3]. Starting from the 3DNET weights, the network
reaches at 4,000 iterations the accuracy a random start reaches at 20,000, and
the paper says the advantage shrinks as training goes on
[arxiv:1603.01564 §4.3]. Card: [High precision grasp pose detection in dense
clutter](/papers/1603.01564).

For a box-detection model on a pick-and-place arm, a detector trained only on
synthetic images scores 0.911 mAP on about 100 real UC-1 images and 0.723 on
UC-2, with no training on real data [arxiv:2410.19277 tab. 7]. The real images
are scored in full there, and in a separate 20 % test split later. Starting
from the repaired synthetic model and training on 60 % of the real images gives
0.948 mAP on UC-1 and 0.892 on UC-2. Training from the standard pre-trained
weights on real images alone gives 0.806 and 0.755 [arxiv:2410.19277 tab. 8].
The fine-tuned models converge in 6 to 12 epochs, against 26 for real-only
training [arxiv:2410.19277 tab. 8]. The UC-2 camera has a fisheye effect that
the simulation does not model, and the authors blame it for the weaker UC-2
result [arxiv:2410.19277 §3.2.3].

### Adapt at test time from real trial outcomes

ManipLLM fine-tunes LLaMA-Adapter on simulated articulated objects, with four
fine-tuning tasks that share one image [arxiv:2312.16217 §3.1.2]. It answers with
a contact pixel and two gripper directions [arxiv:2312.16217 §3.1.2]. On
simulated PartNet-Mobility objects, it reaches 0.54 initial-movement success on
10 unseen categories, against 0.41 for the best baseline, Implicit3D
[arxiv:2312.16217 tab. 1]. On a real Franka arm with 7 categories and 5 trials
each, the per-category counts sum to 28 of 35; the paper does not report that
total [arxiv:2312.16217 tab. 3] [arxiv:2312.16217 §4.4].

After a real attempt, the model is asked the affordance yes/no question about its
predicted contact point, with the real outcome as the answer. Only the visual
adapter is updated [arxiv:2312.16217 §3.3]. The paper shows a real example with
and without this step only qualitatively [arxiv:2312.16217 §4.4]
[arxiv:2312.16217 fig. 5]. After the initial contact is established, an active
impedance loop pushes along several candidate directions and keeps the one that
moves the end-effector most [arxiv:2312.16217 §3.2]. The number of candidates and
the push sizes are not stated [arxiv:2312.16217 §3.2]. The public code contains no
impedance or test-time adaptation code [the card](/papers/2312.16217).

### Train in a scanned Gaussian-splatting simulator with a privileged critic

REAP trains a camera-based Soft Actor-Critic parking policy in a 3D Gaussian
Splatting simulator. The scene is built from handheld LiDAR and camera scans,
and the LiDAR points initialise the Gaussians [arxiv:2605.08713 §5.1]. The actor
sees only surround-view images. The critic also gets BEV maps from the
simulator, which are used only in training [arxiv:2605.08713 §4.2]. Camera
intrinsics and extrinsics are perturbed during training, and the target position
is disturbed too [arxiv:2605.08713 §4.4.1] [arxiv:2605.08713 §5.3].

The policy learns from a mix of two signals. A Reeds-Shepp planner supplies
behaviour cloning, and the probability of using it falls linearly from 1 to 0
over 8,000 epochs [arxiv:2605.08713 §4.2] [arxiv:2605.08713 §5.3]. Removing
the perception pre-training stage gives 50.5% simulated success, against 83.5%
for the full method, and removing the hybrid RL stage gives 0.0%
[arxiv:2605.08713 tab. 2].

The simulator comparison in the paper gives 70.0% real parking success for
3DGS, against 45.0% for a mesh simulator and 0.0% for a CAD simulator built in
CARLA. A 3DGS build takes about 1 h, and a CAD build more than 10 h
[arxiv:2605.08713 tab. 3]. The five real-vehicle scenarios reach 55% to 85%
success, with 20 trials each, on one vehicle with one seed
[arxiv:2605.08713 tab. 4]. The paper does not claim generalisation to unseen
outdoor scenes or to dynamic obstacles [arxiv:2605.08713 §6].

### Zero-shot mobile manipulation in simulated houses

One PPO policy drives a Stretch robot's base, arm height and extension, and
wrist together, from two RGB cameras and five values of arm proprioception
[arxiv:2312.06639 §IV-C]. It is trained in ProcTHOR houses. The authors added
door-opening and table-cleaning logic to the simulator, because the stock
version has none [arxiv:2312.06639 §IV-A]. The paper uses a pretrained DINOv2
backbone, and the released code keeps it frozen [the card](/papers/2312.06639).
Without the pretrained backbone, the pull task reaches 0% success in simulation
[arxiv:2312.06639 tab. IV].

On a real robot in one apartment with a new layout, and with no fine-tuning, the
policy cleaned 8 of 12 tables, opened 9 of 15 doors by pulling and 7 of 10 by
pushing [arxiv:2312.06639 tab. III]. The fridge was not run on hardware,
because the magnetic seal was too strong for the robot to pull
[arxiv:2312.06639 §V-C]. The paper does not state what share of a table counts
as a successful clean [the card](/papers/2312.06639).

### Precise pose beside an object, with history on the robot

Aim My Robot is trained by behaviour cloning on 500,000 simulated trajectories
in Isaac Sim, then refined with DAgger [arxiv:2411.14770 §IV]. Its trajectories
come from a planner, not from human driving [arxiv:2411.14770 §III-A]. The
network reads frozen MAE tokens for the current image and the reference image,
depth as position embeddings, and one token per LiDAR bin. It outputs base
waypoints and camera tilt [arxiv:2411.14770 §III-B].

In simulation the completion rate is 95.9% when the object is visible at the
start, against 89.0% for a baseline that uses a CAD model, a visible object and
a perfect map [arxiv:2411.14770 fig. 6] [arxiv:2411.14770 §V-A]. The reported
3 cm and 1 degree median errors come only from runs that finish within 1 m
[arxiv:2411.14770 §V]. In a real kitchen, with four frames of history, three
of three runs completed for five of six objects. With no history, the same
objects completed between one and three runs [arxiv:2411.14770 tab. III]. The
paper says history matters on the robot and gives no benefit in simulation
[arxiv:2411.14770 §IV] [arxiv:2411.14770 §V-C].

### Noise on executed commands while recording, and a geometric gate

LiCS records a planner's velocity commands, and the human joystick runs, in
Gazebo. Gaussian noise with σ = 0.25 is added to the command the simulator
executes, but the recorded target is the clean command
[arxiv:2406.14947 §II-A] [arxiv:2406.14947 §IV-A]. The network is a
Transformer that reads a 2D LiDAR scan and a normalised local goal
[arxiv:2406.14947 §II-B].

On a Jackal, a geometric check on the straight and turning swept regions guards
the motion. When it flags a motion, the layer slows down, rotates in place or
stops, using handcrafted rules [arxiv:2406.14947 §II-C]. The check runs only in
the hardware trials [arxiv:2406.14947 §IV-D]. On three cardboard tracks, the
robot completed two, with five trials each. The third was not completed, and
the paper blames collisions and stuck corners [arxiv:2406.14947 tab. 5]
[arxiv:2406.14947 §IV-D]. In simulation at 1.4 m/s, the policy completes 100%
of 66 benchmark worlds, against 99.49% for the planner it copies
[arxiv:2406.14947 tab. 2].

### Human wrist plans, distillation and augmentation for a dexterous hand

A Transformer planner, trained by behaviour cloning on human wrist motion,
proposes 6-DoF wrist targets. A PPO controller adds finger commands and a
wrist residual limited to ±4 cm and ±0.5 rad [arxiv:2411.04005 §4.1]
[arxiv:2411.04005 §4.2]. Joint velocities and object velocities cannot be
measured on the real hand, so a teacher that sees them is distilled into the
policy with DAgger. Object pose comes from four top-down cameras, using
FoundationPose at 15 Hz [arxiv:2411.04005 §4.4] [arxiv:2411.04005 app. C].

In simulation, single-object policies complete 83.8% of a trained trajectory
with the augmentation loop, and 59.6% without it [arxiv:2411.04005 tab. 4]. In
six real-world tasks, the policy completes 64.3% to 100% of each trajectory,
over 20 trials per rate. The vanilla RL baseline completes 4.5% to 56.1%
[arxiv:2411.04005 tab. 2]. The paper's domain randomisation table lists 16
settings, such as object and robot mass, friction, joint stiffness and
gravity [arxiv:2411.04005 app. E]. The released configuration turns
randomisation off and uses different ranges, so the table is not the code's
default [the card](/papers/2411.04005). The public repository has the RL
environment, but not the planner or the augmentation loop [the card](/papers/2411.04005).

### Start a dexterous hand from a grasp prior

An AnyGrasp two-finger grasp on the object's functional part is mapped to a
16-DoF Allegro hand and filtered for collisions. The grasp closest to the camera
is the start state, and then PPO refines the arm and hand from it
[arxiv:2412.15587 §3.1] [arxiv:2412.15587 fig. 2]. In SAPIEN, on an XArm6, the
method beats DexArt on all four tasks, for example 94.17% against 79.17% on
the bucket [arxiv:2412.15587 tab. 1]. On a UR5 with the same hand, the bucket
and laptop tasks reach 79.08% and 74.31% averages, over about ten trials each
[arxiv:2412.15587 tab. 2]. The real-robot policy is retrained in simulation with
the real camera pose, and the setup uses foam boards and a nail as props
[arxiv:2412.15587 §4.2].

### Simulation-only policy with privileged critic

A drone team with a camera and a cable-slung payload is trained by PPO with an
asymmetric actor-critic. The critic sees the full simulated state, and the actor
sees partial observations [arxiv:2409.03930 §3.2]. This is the same split as in
the Shadow Hand task above. In Gazebo the method reaches success rates of 0.824
to 0.903, against 0.502 to 0.786 for Q-learning, DQN and Sarsa
[arxiv:2409.03930 tab. 1]. The reward, observation, action space, network and
training budget are not stated, so the policy cannot be rebuilt from the paper
alone [arxiv:2409.03930 §3.2]. The paper says its experiments cover both
simulation and the real world, but the source reports only simulation
[arxiv:2409.03930 §4] [arxiv:2409.03930 §5].

### Make the simulated site harsher than the real one

A quadruped learns to cross pipes in Isaac Gym, with the walls set to low
friction (static 0.2, dynamic 0.1), a raised entrance and random pushes on the
base [arxiv:2412.13621 §II-A4]. The paper groups these under sim-to-real
modifications. Friction is also randomised between 0.6 and 2, and 0 to 3 kg is
added to the base mass [arxiv:2412.13621 §III-A]. Up to four obstacles of
0.1 to 0.3 m high are placed on the pipe surface in the last stage
[arxiv:2412.13621 §II-A3] [arxiv:2412.13621 tab. III]. Training uses a height
scan of the floor and the ceiling, which the simulator provides and the robot
does not [arxiv:2412.13621 §II-B2]. The paper's second stage, a depth-camera
student, is shown only in a figure, with no training details
[arxiv:2412.13621 fig. 1].

In simulation, the 0.2 m pipe with obstacles succeeds in 93.75% of 16 runs
[arxiv:2412.13621 tab. IV]. On three real PVC pipes of 0.19 to 0.24 m radius,
the robot crossed without obstacles in every trial. On the smallest pipe with
obstacles, it crossed in 46.5% of trials [arxiv:2412.13621 tab. V]. The number
of real trials is not stated, and the paper says real success was much lower
than in simulation with the same policies [arxiv:2412.13621 §III-D].

### Keypoint tracker shared across humanoid tasks, with a depth-based task generator

The tracker's command is an 18-value vector: the root's position error, plus the
offsets of the head, both hands and both feet from the root [arxiv:2509.20322
§3.1] [arxiv:2509.20322 eq. 1]. A tracker is trained once on human motion. A
teacher sees the reference motion for the next 2 seconds and privileged contact
forces, and a student that sees proprioception and the command is distilled from
it with DAgger [arxiv:2509.20322 §3.1]. Each task then gets its own generator,
which outputs the command. Its teacher sees object state and is trained with PPO
on a task reward. Its student reads depth through a CNN and is distilled with
DAgger [arxiv:2509.20322 §3.2]. Because the tracker is shared, a new task retrains
only the generator [arxiv:2509.20322 §3].

Two stabilisers are added. During tracker training, each command dimension is
multiplied by a factor drawn from U(0.5, 1.5). The generator's output is clipped
to a band of μ ± 1.64σ, where μ and σ are estimated from human motion
[arxiv:2509.20322 §3.3]. The ablations in simulation say that removing the noise
breaks the push-box task, and that removing the clip makes returns collapse late
in training [arxiv:2509.20322 §4.5] [arxiv:2509.20322 tab. 3]. Real depth is
approximated with random masks on simulated depth. The camera tilt is randomised
by ±5°, and a binary command pauses the task half of the time during training
[arxiv:2509.20322 §3.4].

On a Unitree G1 with a RealSense D435i, the policies lift a box, push a box,
dribble a ball and kick a box, indoors and outdoors. The paper gives no success
rates for these runs [arxiv:2509.20322 §4.2]. In simulation, the vision student
covers 37 m in one-minute push-box rollouts, against 152 m for the state-based
teacher, and 135 m against 189 m on the ball-kick task [arxiv:2509.20322 tab. 2].
Without vision, the student covers 2 m on the push box [arxiv:2509.20322 tab. 2].
The released code is a MuJoCo sim2sim demo with four checkpoints, and the
training code is not in it [the card](/papers/2509.20322).

### Survey taxonomies

Two surveys sort the methods into families, and neither reports transfer
numbers. A small-car survey lists four routes: zero-shot transfer, transfer
learning, domain adaptation and domain randomisation [arxiv:2404.06229 §2.3].
It notes that domain randomisation is offered on two platforms, and reports no
transfer result for any platform [the card](/papers/2404.06229). A legged-locomotion
survey covers avoiding simulator exploits, system identification, domain
randomisation and domain adaptation, the last by inferring the environment
[arxiv:2406.01152 §5.1] [arxiv:2406.01152 §5.2] [arxiv:2406.01152 §5.3]
[arxiv:2406.01152 §5.4]. It names mass, friction, motor strength and latency as
randomised quantities, with no ranges [arxiv:2406.01152 §5.3]. It contains no experiments of its own
[the card](/papers/2406.01152).

## When to use which

The rows use constraints from each paper's own setup. The robot results come from
one robot, the humanoid results from one Digit V3 and MuJoCo, the manipulation
results from one arm, the grasp results from one Franka and one depth camera,
the hand results from one TriFinger and simulation, and the parking results from
one vehicle, so every hardware claim below holds for those systems alone.

| Situation | Use | Why |
|---|---|---|
| Forward depth camera on the robot | Map simulated and real depth into one space, as both pipelines do | The paper uses this design [arxiv:2601.07718 §3.2]. Its ablations run in simulation only [arxiv:2601.07718 tab. 2], so the card records no hardware test of this step on its own. |
| Terrain with stairs, platforms or gaps where feet can slip on edges | Add an edge-contact penalty | On hardware the penalty variant did better on stairs and deep gaps [arxiv:2601.07718 tab. 1]. Simulation success barely separates the two. |
| Simulator parameters (mass, friction, damping, gains) are uncertain | Randomise them at each reset, using the ranges in the paper's table | The Shadow Hand OpenAI task does this [arxiv:2108.10470 tab. 18]. Its results are simulation-only, and the card has no randomisation settings for its hardware runs [arxiv:2108.10470 §6.4.2]. |
| Policy must run on measurable inputs but training can use simulator state | Asymmetric actor–critic: policy sees measurable observations, value function sees privileged state | This is the setup in the Shadow Hand OpenAI task [arxiv:2108.10470 §6.4.1]. The card reports no symmetric-critic comparison. |
| Large training budget on one GPU | Keep physics and learning on the GPU with thousands of environments | Ant reaches reward 3000 in about 20 s on one A100 [arxiv:2108.10470 §6.1.1]. These are simulation times, not transfer results. |
| Humanoid with uncertain dynamics and a curriculum budget | Randomise the five quantities named in the paper, and use a staged torso-push curriculum | The Digit V3 policy did this and fails on 5% of hardware motions, against 0.450% in simulation [arxiv:2408.07295 tab. 3]. Its ablation without the curriculum is in simulation only [arxiv:2408.07295 tab. 5]. Ranges are not given here; the paper points to earlier Digit locomotion work [arxiv:2408.07295 §4.5]. |
| Collision-aware 6-DoF grasps from one depth view of a cluttered scene | A per-point grasp predictor trained on simulated grasps with depth noise | The paper reports 90.20% on 51 unseen objects, against 80.39% for the best baseline [arxiv:2103.14127 tab. 1]. Its confidence thresholds, 0.23 and 0.19, belong to this network and need recalibration for another [arxiv:2103.14127 §4.1]. |
| Labelled real data for a perception task is scarce or costly | Pre-train on synthetic data, then fine-tune on the real set | Pre-training then fine-tuning ends above the real baseline [arxiv:1810.08705 tab. 4]. Synthetic-only training scores well below it [arxiv:1810.08705 tab. 4], so this setup needed the real fine-tune. For grasp detection, simulated pretraining cut the iterations to matching accuracy from 20,000 to 4,000 [arxiv:1706.09911 §7.3]. A CAD-only grasp classifier reached 83% on real BigBird scans, against about 90% for real-data training [arxiv:1603.01564 §4.3]. |
| Need to know how errors depend on distance, occlusion or actor orientation | Bin the errors by one scenario parameter on synthetic data | Parameters are sampled independently per image, which makes the bins usable [arxiv:1810.08705 §3] [arxiv:1810.08705 §6]. Correlation is association, not causation [arxiv:1810.08705 §6.1.3]. |
| Planning to use the Synscapes dataset commercially | Confirm the dataset's licence first | The paper's CC BY 4.0 licence covers the paper. The paper does not state the dataset's licence or download location [arxiv:1810.08705 §3]. |
| Real robot with a new gap, and each trial has a clear success signal | Adapt the visual adapter from trial outcomes | The paper describes the step [arxiv:2312.16217 §3.3], but gives no quantitative comparison with and without it [arxiv:2312.16217 §4.4]. |
| Robot working near people | Keep the paper's safety setup | Testing used continuous supervision, an emergency stop and an exclusion zone. The paper does not establish safety for unsupervised use [arxiv:2601.07718 app. B]. |
| Many hardware trials are costly | Plan around small trial counts | Each terrain and gait pair was tried 10 times [arxiv:2601.07718 §4.1]. The paper says its results do not establish generalisation across robots or dynamic environments [arxiv:2601.07718 §5]. |
| Camera-based vehicle must park in a scanned site, and a scanned environment is available | Build a Gaussian-splatting replica from handheld scans and train with a critic that sees simulator-only maps | The paper's 3DGS build reaches 70.0% real success, against 45.0% for a mesh and 0.0% for a CAD build [arxiv:2605.08713 tab. 3]. These are one-vehicle results, and the paper does not establish generalisation to dynamic obstacles [arxiv:2605.08713 §6]. |
| Humanoid must manipulate objects from depth and proprioception, and a new task should not retrain the whole-body controller | Train one keypoint tracker from human motion, then a depth-based task generator per task, with command noise and clipping to the human-motion band | The paper retrains only the generator per task [arxiv:2509.20322 §3]. The real-robot runs are qualitative, and the ablations are simulation-only [arxiv:2509.20322 §4.2] [arxiv:2509.20322 tab. 3]. |
| Legged robot with no cameras or other exteroception, and a teacher that can use full simulator state | Distil a state-based teacher into a student that reads a window of observation-action history | The paper's student is a causal transformer with a KL term to the teacher [arxiv:2303.03381 eq. 2]. Its architecture and objective comparisons give values only in figures [arxiv:2303.03381 fig. 8]. |
| A vendor or high-fidelity simulator is available for a check before hardware | Run the trained policy there, keep the weights unchanged, and drop unsafe policies | The paper uses this filter alone [arxiv:2303.03381 § Sim-to-real transfer]. The simulator is proprietary, so the paper's check cannot be repeated [the card](/papers/2303.03381). |
| Sim terrain differs from the test site | Do not compare sim and hardware numbers directly | Simulated gaps are 0.60 m [arxiv:2601.07718 tab. 8] and the hardware gap is 50 cm [arxiv:2601.07718 §4.1]. |
| A load moves on the robot (sloshing, swinging, dragged) and the policy gets no load sensing | Train with the load model in simulation, and consider one general policy over all loads | On the four loaded models, specific policies reach pass rates of 0.71 to 1.00 and the general policy 0.83 to 1.00 [arxiv:2204.04340 tab. 2]. Hardware top speeds fall well below simulation, and the paper expects dynamics randomisation alone to fall short [arxiv:2204.04340 §V] [arxiv:2204.04340 §VI]. |
| Training is slow because each hardware failure means another full run | Use an off-policy learner on a GPU simulator with thousands of environments, and keep the reward to a few terms | The authors report a 15-minute run on one RTX 4090, but the figures are not transcribed and hardware is shown only for G1 whole-body tracking [arxiv:2512.01996 §3.1] [arxiv:2512.01996 §3.2]. The released code differs from the paper, so test the settings on a small task first [the card](/papers/2512.01996). |
| Biped needs perception, the height map exists only in simulation, and the robot has depth cameras | Distil the simulated height-map latent into a depth-camera encoder, and post-process simulated depth to match the sensor | The depth student's latent is pulled toward the simulated height-map latent [arxiv:2512.12993 §IV.C]. No policy ran on hardware, so the paper gives no real-robot success rate [arxiv:2512.12993 §V]. |
| Robot must reach a precise pose beside an object, with no CAD model or map on board | Train a transformer on simulated trajectories, refine with DAgger, and keep four frames of history on the real robot | Real kitchen runs completed 3 of 3 for five of six objects with history, against 1 to 3 without it [arxiv:2411.14770 tab. III]. The simulated baseline has a CAD model and a perfect map, so it is not a like-for-like test [arxiv:2411.14770 §V-A]. |
| Mobile manipulator must work in unseen houses, with no real training data | Train one PPO policy in simulated houses that include the task logic, with a pretrained image backbone | One unseen apartment with no fine-tuning: 60% to 70% on door and table trials [arxiv:2312.06639 tab. III]. The paper gives no success threshold for cleaning [the card](/papers/2312.06639). |
| Quadruped must cross narrow pipes with slippery walls, and the simulator can give a state the robot cannot measure | Make the simulated pipe harsher, and train with a height scan that exists only in simulation | Real PVC pipes: all trials crossed without obstacles, and 46.5% with obstacles on the smallest pipe. The paper says real success is far below simulation [arxiv:2412.13621 §III-D]. |
| Vision model on an arm, a simulated copy of the work cell, and few real images | Search the simulator for failures with a genetic algorithm, fine-tune on them, then fine-tune on the real images | Against random search: about 50% and 26% more failures (author-reported). Real mAP 0.948 and 0.892, against 0.806 and 0.755 for real-only training [arxiv:2410.19277 tab. 8]. A simulated test takes 280 to 310 s, so one 220-evaluation run takes 17 to 19 hours [arxiv:2410.19277 §3.1]. |
| Wheeled robot must navigate clutter, with a planner or a human to imitate | Behaviour cloning with noise on executed commands during recording, and a geometric check before motion on hardware | Two of three cardboard tracks were completed on the robot, with five trials each [arxiv:2406.14947 tab. 5]. In simulation, 100% of 66 benchmark worlds at 1.4 m/s [arxiv:2406.14947 tab. 2]. |
| Dexterous hand must track object trajectories, with object velocities unmeasurable on the robot | Distil a privileged teacher into a policy with DAgger, and augment the human data in rounds | Real completion 64.3% to 100% on six objects, against 4.5% to 56.1% for vanilla RL [arxiv:2411.04005 tab. 2]. The released code has no planner or augmentation loop [the card](/papers/2411.04005). |
| Dexterous hand must open articulated objects, with a grasp on the functional part available | Start from an AnyGrasp-based grasp on that part, then refine with PPO | Real UR5, about ten trials per task: 79.08% (bucket) and 74.31% (laptop) on average [arxiv:2412.15587 tab. 2]. The props and the retrained simulation policy are part of the setup [arxiv:2412.15587 §4.2]. |
| Drone team must carry an unknown payload, with a camera only | Asymmetric actor-critic PPO in a simulator | No hardware result is reported, so do not claim transfer from this paper alone [arxiv:2409.03930 §5]. |
| Choosing a small research car for a transfer study | Use the survey's platform table to shortlist, then verify each price and specification with the vendor | The survey reports no transfer numbers, and its prices are undated [the card](/papers/2404.06229). |

## Implementation sketch

Each block is one step from one source. They are not a single pipeline. Values
are the paper's where it gives them.

```pseudocode
# Add uniform noise to proprioception for the actor  [arxiv:2601.07718 tab. 4]
def noisy_proprio(s):
    s.ang_vel += uniform(-0.2, 0.2)       # rad/s
    s.gravity += uniform(-0.05, 0.05)
    s.joint_pos += uniform(-0.01, 0.01)   # rad
    s.joint_vel += uniform(-0.5, 0.5)     # rad/s
    return s

# Give the actor the depth frame from one step earlier  [arxiv:2601.07718 §3.2]
def depth_for_actor(depth_buffer):
    return depth_buffer.previous()

# The critic sees the clean state and base linear velocity  [arxiv:2601.07718 tab. 3]
def critic_input(clean_state, base_lin_vel):
    return concat(clean_state, base_lin_vel)

# Sample humanoid dynamics for each episode; ranges come from the caller
# Quantities named in the paper; no values given  [arxiv:2408.07295 §4.5]
def randomise_humanoid(sim, ranges):
    sim.joint_damping = sample(ranges.damping)
    sim.link_mass = sample(ranges.mass)
    sim.com_offset = sample(ranges.com)
    sim.encoder_noise = sample(ranges.encoder)
    sim.ground_friction = sample(ranges.friction)

# Stage 1 torso push: one step, 80 to 800 N  [arxiv:2408.07295 §4.4]
# Each policy step has a 1% chance of a push
def stage1_push():
    if uniform(0, 1) < 0.01:
        return uniform(80, 800)   # N
    return 0.0

# Add Gaussian noise to simulated depth before building the point cloud
# [arxiv:2103.14127 fig. 6]; the paper does not state sigma
def noisy_depth(depth, sigma):
    return depth + normal(0.0, sigma, size=depth.shape)

# Resample simulator parameters at each reset  [arxiv:2108.10470 tab. 18]
def randomise_on_reset(sim):
    # Scaling factors drawn uniformly  [arxiv:2108.10470 tab. 18]
    sim.object_size *= uniform(0.95, 1.05)
    sim.link_mass *= uniform(0.5, 1.5)
    sim.friction *= uniform(0.7, 1.3)
    # Scaling factors drawn log-uniformly  [arxiv:2108.10470 tab. 18]
    sim.joint_damping *= loguniform(0.3, 3.0)
    sim.p_gain *= loguniform(0.75, 1.5)
    # Additive terms drawn from normal distributions  [arxiv:2108.10470 tab. 18]
    sim.joint_limit_offset = normal(0.0, 0.15)   # rad
    sim.gravity += normal(0.0, 0.4, size=3)      # m/s^2, per coordinate

# Pre-train on synthetic images, then fine-tune on real ones  [arxiv:1810.08705 §5.1]
def train_two_stage(model, synthetic_set, real_set):
    # Stage 1: synthetic labels only  [arxiv:1810.08705 tab. 4]
    train(model, synthetic_set)
    # Stage 2: continue on the real set from the synthetic weights  [arxiv:1810.08705 §5.1]
    train(model, real_set)
    return model

# After one real trial, update the visual adapter only  [arxiv:2312.16217 §3.3]
def adapt_after_trial(model, rgb, pixel, succeeded):
    # Ask the yes/no affordance question about the predicted point  [arxiv:2312.16217 §3.3]
    question = affordance_question(rgb, pixel)
    # The real outcome is the answer target  [arxiv:2312.16217 §3.3]
    target = "yes" if succeeded else "no"
    loss = model.loss(rgb, question, target)
    model.step_visual_adapter(loss)

# Student objective: RL loss plus a KL term to the teacher  [arxiv:2303.03381 eq. 2]
# The weight lam is annealed to zero near mid-training; its shape is not stated
# [arxiv:2303.03381 § Joint optimization with reinforcement learning]
def student_objective(rl_loss, kl_term, lam):
    return rl_loss + lam * kl_term   # KL direction as written in eq. 2

# Record a step with noise on the executed command  [arxiv:2406.14947 §II-A]
# The stored label is the clean expert command, not the noisy one
def record_step(lidar, goal, expert_cmd):
    # Gaussian noise on (v, omega); sigma = 0.25 in the paper, units not stated
    noise = normal(0.0, 0.25, size=2)
    executed = expert_cmd + noise          # what the simulated robot does
    sample = (lidar, goal, expert_cmd)     # what the network learns to copy
    return executed, sample
```

## Contradictions

No two sources report conflicting results for the same setup. The disagreements
below sit inside single sources, and each side is cited.

- **KL direction.** Equation 2 writes the student term as
  D_KL(π_o ‖ π_s) [arxiv:2303.03381 eq. 2]. The text describes distilling the
  teacher "through" KL divergence [arxiv:2303.03381 § Teacher state-policy
  supervision]. Forward and reverse KL differ, so this page follows eq. 2 as
  written. The card asks the implementer to check the supplement
  [the card](/papers/2303.03381).
- **Standing against SaW.** The text calls the Digit V3 policy "on par" with the
  Stand-and-Walk baseline [arxiv:2408.07295 §5.3]. The table shows SaW drifts less
  under a 50 N push (0.109 m against 0.256 m) [arxiv:2408.07295 tab. 2]. The
  policy drifts less under 100 N and when walking.
- **Best on setA.** The text says MHC_A "excels on setA" [arxiv:2408.07295 §5.4].
  The table gives MHC_AB the best setA score on both MEPE and root drift
  [arxiv:2408.07295 tab. 4]. The card follows the table.

- **Hardware validation.** The Digit perception paper's abstract describes a
  preliminary hardware validation [the card](/papers/2512.12993). The body
  compares simulated and real depth images only, and states that no policy ran on
  hardware [arxiv:2512.12993 fig. 12] [arxiv:2512.12993 §V]. Read the claim as a
  sensor comparison.
- **Synscapes fine-tuning gain.** The text says Synscapes gives "more than
  twice the gain compared to both Richter and Synscapes" [arxiv:1810.08705 §5.1]. The
  table gives +2.29 for Synscapes, +1.01 for the GTA-based set and +0.89 for
  Synthia, against the 76.56 baseline [arxiv:1810.08705 tab. 4]. The card records
  that the second name in the sentence is probably a typo for Synthia. Check the
  table, not the sentence.
- **Depth noise.** The paper's simulated pipeline adds range noise, disparity
  artifacts and out-of-distribution replacement [arxiv:2601.07718 app. A.4]. The
  released parkour training configuration applies only crop, blur and
  normalisation, according to [the card](/papers/2601.07718).
- **Edge penalty constant.** Equation 1 uses ε = 10⁻³ [arxiv:2601.07718 eq. 1].
  Appendix A.5 lists 10⁻⁶ [arxiv:2601.07718 app. A.5]. The released code uses
  10⁻⁶, according to [the card](/papers/2601.07718).
- **Simulator camera rate.** Hardware depth runs at 60 Hz [arxiv:2601.07718 §3.2].
  The released simulation updates the camera every 0.02 s, according to
  [the card](/papers/2601.07718).
- **Simulated parking success.** The main table gives REAP-SAC 83.5% on standard
  slots [arxiv:2605.08713 tab. 1], and the ablation baseline repeats that number
  [arxiv:2605.08713 tab. 2]. The simulator table gives the same 3DGS method
  82.5% [arxiv:2605.08713 tab. 3]. The paper does not explain the gap, and the
  ablation table does not say which slot type it covers [arxiv:2605.08713 tab. 2].
- **Success thresholds.** ManipLLM's paper gives thresholds of 0.01 and 0.1
  [arxiv:2312.16217 §4.1], but not the cosine rules. The code's training labels and
  test rule differ, according to [the card](/papers/2312.16217). Compare results
  only under one rule.
- **Repair gain.** The MARTENS abstract gives 25% to 50% more failures than
  random search, and its conclusion gives 26% to 50%. The body gives 50% for
  UC-1 and 26% for UC-2 [arxiv:2410.19277 §1] [arxiv:2410.19277 §3.2.1]. The
  printed failure counts give 49% and 26% [arxiv:2410.19277 tab. 2]
  [arxiv:2410.19277 tab. 3]. This page quotes the body's figures.
- **Fine-tuning epochs.** The MARTENS abstract says the fine-tuned models reach
  their scores in fewer than 10 epochs. Table 8 gives 12 epochs for the UC-2
  model that starts from the repaired detector [arxiv:2410.19277 tab. 8].
- **Dexterous hand, text and table.** The text says the method improves on
  vanilla RL by more than 50% on the real objects. Table 2 gives a 39.4-point
  gap on the microwave against fingertip mapping [arxiv:2411.04005 §5.4]
  [arxiv:2411.04005 tab. 2]. The page quotes the table.
- **Grasp-prior headline numbers.** The text gives 92%, 87% and 87% for the
  bucket, laptop and toilet at later training steps. Table 1 gives 94.17, 90 and
  87.88 for the same rows [arxiv:2412.15587 §4.1.3] [arxiv:2412.15587 tab. 1].
  The paper does not say which checkpoint each figure comes from.
- **Push-box mass and speed.** The paper's vision policy pushes a 3.8 kg box on
  hardware. The 37 m per minute figure matches the simulated 4 kg Push Box row,
  and the paper does not say which box it measures [arxiv:2509.20322 tab. 2]. The
  card records the same mismatch [the card](/papers/2509.20322).
- **Real-world claim without results.** The drone paper says its method was shown
  in both simulation and the real world. The source reports only simulation
  [arxiv:2409.03930 §4] [arxiv:2409.03930 §5].
- **Hardware speed.** The LiCS abstract gives 1.5 m/s as the safe maximum. Its
  hardware section says 1.5 m/s was tried and 1.3 m/s performed best, without
  saying which trials used which speed [arxiv:2406.14947 §IV-D]. The card
  records the abstract's wording [the card](/papers/2406.14947).

## Papers

- [High precision grasp pose detection in dense clutter](/papers/1603.01564): a 15-channel CNN grasp classifier, pretrained on CAD-rendered depth and tested on a Baxter with author-reported 93% grasp success.
- [Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads](/papers/2204.04340): a proprioception-only PPO walker for the Cassie biped, trained with four dynamic loads in simulation and tested on hardware at lower speeds.
- [Learning Sim-to-Real Humanoid Locomotion in 15 Minutes](/papers/2512.01996): an off-policy recipe (FastSAC and FastTD3) for massively parallel humanoid training, with a G1 whole-body tracking hardware test.
- [Learning Terrain Aware Bipedal Locomotion via Reduced Dimensional Perceptual Representations](/papers/2512.12993): a height-map VAE latent for a Digit walker, distilled into a depth-camera encoder, with simulation results only.
- [Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids](/papers/2601.07718): a humanoid parkour policy trained in simulation with depth and an edge-contact penalty, reported zero-shot on a Unitree G1.
- [Grasp Pose Detection in Point Clouds](/papers/1706.09911): a sampler and 15-channel CNN for grasps on depth clouds, pretrained on simulated exemplars and fine-tuned on real BigBird scans, with a Baxter clutter test.
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](/papers/2103.14127): a per-point grasp network trained on simulated grasps, with depth noise and a real Franka test using an Intel RealSense L515.
- [Real-World Humanoid Locomotion with Reinforcement Learning](/papers/2303.03381): a causal-transformer walker for the full-size Digit, trained in simulation with PPO and a KL term to a state-based teacher, deployed zero-shot outdoors.
- [Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots](/papers/2408.07295): one masked-directive LSTM policy for a Digit V3 humanoid, trained in MuJoCo with domain randomization and a three-stage curriculum, with hardware trials and no released code.
- [Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing](/papers/1810.08705): a 25,000-image synthetic street dataset, with results for pre-training and fine-tuning on Cityscapes and KITTI.
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](/papers/2108.10470): a GPU physics simulator for robot learning, with a domain-randomised Shadow Hand task and sim-to-real claims on ANYmal and a TriFinger.
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](/papers/2312.16217): a multimodal model fine-tuned on simulated articulated objects, tested on a real arm with test-time adaptation.
- [REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real Transfer](/papers/2605.08713): a camera-to-action parking policy trained with SAC in a scanned Gaussian-splatting simulator, reported on a real vehicle, with no released code.
- [Harmonic Mobile Manipulation](/papers/2312.06639): one PPO policy for base and arm on a Stretch robot, trained in ProcTHOR and run zero-shot in one real apartment, with Apache-2.0 code that differs from the paper in several values.
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](/papers/2404.06229): a survey of small research car platforms and their tasks, with no experiments and no transfer numbers.
- [Learning-based legged locomotion; state of the art and future perspectives](/papers/2406.01152): a survey of learned legged locomotion, covering privileged teacher-student training and the sim-to-real methods, with no experiments.
- [LiCS: Navigation using Learned-imitation on Cluttered Space](/papers/2406.14947): a small Transformer copies a local planner from simulated data, with noise on executed commands and a geometric check on a Jackal.
- [VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and Generation](/papers/2509.20322): a shared keypoint tracker from human motion and a depth-based task generator, with qualitative G1 hardware runs and a MuJoCo sim2sim demo release.
- [DRAL](/papers/2409.03930): a PPO drone team with an asymmetric actor-critic, evaluated in Gazebo only.
- [In-Simulation Testing of Deep Learning Vision Models in Autonomous Robotic Manipulators](/papers/2410.19277): a genetic search over an Isaac Sim work cell finds detector failures, which are then repaired and tested on a few real images.
- [Object-Centric Dexterous Manipulation from Human Motion Data](/papers/2411.04005): a human-data wrist planner and a PPO finger controller for two dexterous hands, with distillation, domain randomisation and six real objects.
- [Aim My Robot: Precision Local Navigation to Any Object](/papers/2411.14770): a transformer that reaches a centimetre-precise pose beside an object, trained in simulation with DAgger and tested in a real kitchen.
- [Learning Quadrupedal Robot Locomotion for Narrow Pipe Inspection](/papers/2412.13621): a Go2 quadruped trained in Isaac Gym with a privileged height scan and harsher simulated pipes, tested on three real PVC pipes.
- [Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge](/papers/2412.15587): a grasp-prior start followed by PPO refinement for an Allegro hand, with simulation results and a small real UR5 test.