Domain randomization

Overview

Domain randomization trains a policy under many random variants of one setting, so it cannot rely on details that differ between variants. The variants are drawn from a set the designer chooses, and the policy is pushed to work across all of them 2003.02979 §5 2108.10470 app. A.4.1 2408.07295 §4.5 2303.03381 §Domain randomization.

The name is used for three kinds of variant in the current cards:

• Relabelling of known symmetries. Other-play trains cooperative agents that have never met. Each agent is shown the game through a random relabelling drawn from the game's symmetries, so a habit that only works with its own copy cannot survive 2003.02979 §1 2003.02979 §5. The paper describes its method as "a very specific kind of asymmetric domain randomization" 2003.02979 §5, and its related-work section ties the idea to the original domain-randomization work 2003.02979 §2.
• Physics, dynamics and terrain in simulation, for transfer to a robot. Isaac Gym randomises masses, friction, joint damping, actuator gains, gravity and object size for its Shadow Hand task, following OpenAI's setup 2108.10470 §6.4.1 2108.10470 app. A.4.1. The ranges are listed in 2108.10470 tab. 18. LocoMamba samples PD gains, masses, motor strength and friction for a quadruped, and lists its ranges in a table 2508.11849 tab. 3. The two Digit cards randomise a related set of dynamics, plus terrain and sensor timing, but neither card states its numerical ranges 2408.07295 §4.5 [arxiv:2303.03381 §Domain randomization]. The Cassie and Holosoma cards also randomise dynamics, and their text gives no numerical ranges 2204.04340 §III-A 2512.01996 §3.1. Two further cards use the same idea for smaller robots and for data collection: a diffusion policy trained on data from a randomised source policy 2411.01349 §3, and a pipe-crawling quadruped 2412.13621 §III-A. A Mini Cheetah controller fine-tuned with RL randomises motor friction, PD gains, ground friction and command delay, with a wider friction range than the work it cites 2311.02304 §III-C. The sense the name usually carries for simulator-to-robot work is this one.
• Random scene and layout content. Kubric is a data generator, not a policy trainer. Each worker draws a random scene, with random camera, objects, materials and velocities, and renders it with ground-truth annotations 2203.03570 §3.2. The card tags this with the concept, but its results compare datasets rather than randomised and fixed generators 2203.03570 §4.2. A field-robot card jitters crop-row orientation at each episode start, with the magnitude not stated 2412.18865 §III-C.

For the sim-to-real method as a whole, a survey of learning-based legged locomotion names domain randomisation as one of its transfer methods. Its table lists which quantities are randomised but gives no ranges 2406.01152 §5.3. Card: Learning-based legged locomotion; state of the art and future perspectives.

No card here randomises rendering as a way to make a policy transfer to real images. Kubric randomises materials and camera pose when it generates data, but no card tests whether that carries over to real images.

Variants

Asymmetric relabelling for zero-shot coordination

Each agent i draws its own relabelling φ_i uniformly from the symmetry set Φ at the start of every episode. The agent observes and acts through φ_i, while the environment itself is not relabelled 2003.02979 §5. The paper applies this on top of an existing self-play method (SAD) and reports cross-play gains in Hanabi 2003.02979 tab. 1. Card: Other-Play for Zero-Shot Coordination.

Physics parameter randomisation in simulation

Each environment gets its own physics parameters. The parameters are sampled from the ranges in 2108.10470 tab. 18, and the sample is redrawn on every reset, with at least 720 steps between resets [arxiv:2108.10470 app. A.4.1]. The ranges are multiplicative for most parameters and additive for joint limits and gravity. Two further random forces act on the object: one applied with a per-episode probability drawn log-uniformly from 0.1% to 10%, and one whose strength decays over time 2108.10470 app. A.4.1.

The paper's Shadow Hand OpenAI variant uses an asymmetric actor–critic. The policy sees only observations that could be measured on a real robot, while the value function also receives simulator state 2108.10470 §6.4.1 2108.10470 §4. The authors list where their setup differs from OpenAI's. Among those differences, their randomisations leave out action delay and motor backlash 2108.10470 §6.4.1. Card: Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning.

Per-episode physics ranges for a vision-based quadruped

LocoMamba samples its physics per episode, inside the training loop 2508.11849 alg. 1. The ranges are given in a table. PD gains K_P is drawn from [40, 90] and K_D from [0.4, 0.8]. Link inertia, body mass and motor strength are scaled by factors in [0.5, 1.5], [0.8, 1.2] and [0.8, 1.2]. Lateral friction is drawn from [0.5, 1.25], motor friction from [0, 0.05], and sensor latency from [0, 0.04] s 2508.11849 tab. 3. Depth frames also receive salt-style pixel noise, which is a sensing variant rather than a physics one 2508.11849 §4.4.

The training and tests run in PyBullet. Results are author-reported and come from simulation only, with zero-shot tests on rugged and dynamic terrain 2508.11849 §5.5. The authors state that real-robot tests were not done 2508.11849 §6. Card: LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba.

Imitation, then RL fine-tuning with friction, gain and delay randomisation

A quadruped policy is first cloned from a DDP-based MPC expert by DAgger, and then fine-tuned with PPO on rough, discrete rough, step and cliff terrain 2311.02304 §III. The expert cannot run in real time on the robot, so it only labels data in simulation 2311.02304 §III-A. Randomisation applies in the fine-tuning stage: observation noise, motor friction, PD gains, ground friction and motor command delay 2311.02304 §III-C. Ground friction is drawn from [0.3, 1.0], which the card notes is wider than in the work it cites (Ji et al.). The delay is a uniform draw over [0, 10] ms, applied to the PD target 2311.02304 §III-C. The imitation stage uses flat terrain only, and the card does not say whether randomisation applies there 2311.02304 §III-B.

Results are author-reported. On hardware, IFM(CR) crosses a 7.5 cm step in 9 of 10 trials, against 8 of 10 for vanilla RL with the contact reward 2311.02304 tab. 2. In simulation, the authors report better survival than MPC on rough, slippery and conveyor terrain, but the values appear only in a figure 2311.02304 §IV-B fig. 4. The hardware slippery test uses friction 0.22, which is outside the training range 2311.02304 §IV-C §III-C. The card also notes that the simulated slippery results rely on training terrain that does not list slippery ground 2311.02304 §IV-B §III-C. Card: Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion.

Load variants for a bipedal robot

Cassie walks with five fixed simulated models: the unloaded robot and four loaded ones, a box on a tray, a dragged cart, a pole with a weight at each end, and a jug whose sloshing is modelled as springs. The load changes only the simulator, not the policy's inputs 2204.04340 §III-B. Within every model, dynamics are randomised over joint damping, joint mass and ground friction, and the card says these ranges are not stated [arxiv:2204.04340 §III-A]. The load variants are therefore a set of fixed models, not a sampled distribution.

Training a policy on its own load gives most of the performance back. The carrying-pole pass rate rises from 0.0014 for the base walker to 0.9578 for the specific policy and 0.8342 for the general one 2204.04340 tab. 2. The general policy is trained by splitting each iteration's timesteps evenly across the five models 2204.04340 §III-B. Card: Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads.

Dynamics randomisation for a whole-body humanoid policy

One policy is trained across three curriculum stages, and dynamics are randomised in all of them 2408.07295 §4.5 2408.07295 §4.4. The card lists joint damping, link masses, centre-of-mass positions, encoder noise and ground friction as the randomised quantities. The paper says the ranges match those of earlier Digit locomotion work, but it does not state them 2408.07295 §4.5. An implementation therefore needs its own ranges, for example taken from the cited prior work, which has no card here.

The policy is also run on a real Digit V3. The authors report failures on 5% of 20 motions over 3 runs each on hardware, against 0.450% in simulation 2408.07295 §5.6 2408.07295 tab. 3. Card: Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots.

Dynamics, terrain and delay randomisation with a teacher–student split

The original Digit walking policy is trained in Isaac Gym with randomised robot dynamics, control parameters, physics, observation noise and observation delay, over smooth planes, rough planes and smooth slopes [arxiv:2303.03381 §Domain randomization]. A teacher sees the full simulator state. A causal transformer student sees only observations and past actions, and is trained with an RL loss plus a KL term to the teacher that is annealed to zero 2303.03381 §Joint optimization with reinforcement learning. The card gives no randomisation ranges; they are said to be in a supplement that is not in the arXiv source 2303.03381 §Domain randomization.

The authors state that combined dynamics, terrain and delay randomisation gave high-quality transfer 2303.03381 §Sim-to-real transfer. They report zero-shot outdoor walking with no falls over a week of full-day tests 2303.03381 §Outdoor deployment. The card reports no run without randomisation. Card: Real-World Humanoid Locomotion with Reinforcement Learning.

Push, delay and gain randomisation for off-policy humanoid training

Holosoma trains G1 and T1 locomotion with off-policy RL on a large parallel simulator. Locomotion randomises push perturbations, action delay, PD gains, mass and friction, and centre of mass for the G1 only 2512.01996 §3.1. Pushes arrive every 5 to 10 s, or every 1 to 3 s in the Push-Strong variant, with episodes capped at 20 s 2512.01996 fig. 3. Whole-body tracking samples a random motion segment per episode and randomises ground friction, centre of mass, a bias on joint positions, body mass, PD gains and external pushes 2512.01996 §3.2. The text gives no numerical ranges for these quantities. The card's hardware results cover G1 whole-body tracking only; the locomotion results are simulation curves 2512.01996 §3.1 fig. 6. Card: Learning Sim-to-Real Humanoid Locomotion in 15 Minutes.

Dynamics, terrain and perturbation settings for diffusion-policy data

An AMP-trained source policy for a simulated Unitree H1 is trained under domain randomisation. The training ranges in 2411.01349 tab. 2 are body friction [0.7, 1.3], added base mass [−2, 2] kg, link-mass and PD-gain multipliers [0.8, 1.2], centre-of-mass displacement [−0.15, 0.15] m, and joint friction and damping coefficients. The same source policy then collects data under eight setups, one randomisation cluster switched on at a time: none, dynamics, perturbations, terrain, initial state, humanoid scale, terrain with perturbations, and "All" 2411.01349 §3.2. Each setup is run at three dataset sizes, 500K, 2M and 8M transitions, and each diffusion policy is trained on one of the 24 resulting datasets 2411.01349 §5. The card notes that the terrain parameters and the initial-state joint range are not stated 2411.01349 §3.2.

The randomised evaluation target uses narrower ranges than training: 2411.01349 tab. 4 lists friction [0.8, 1.2], base mass [−1, 1] kg and no external perturbations. The author-reported results are: at 8M, the "All" policy matches the source RL policy on the flat, non-randomised target, with success 1.0 for both 2411.01349 tab. 3. On the randomised target at 8M, perturbation and terrain randomisation give the strongest policies, and dynamics randomisation reaches 0.49 2411.01349 fig. 3. Each result is the mean over three seeds 2411.01349 §5. Card: The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control.

Friction, mass and random pushes for a pipe-crawling quadruped

A Unitree Go2 is trained with PPO in Isaac Gym to walk through narrow pipes, in three curriculum stages 2412.13621 §III-A §III-B. The sampled randomisations are friction in [0.6, 2], which surface it applies to not stated, and added base mass in [0, 3] kg 2412.13621 §III-A. A random external force acts on the base during training, with no stated magnitude 2412.13621 §II-A4. The pipe walls also get fixed low friction (static 0.2, dynamic 0.1), and the entrance is raised by 0.1 m [arxiv:2412.13621 §II-A4]. These are environment changes, not per-episode samples.

Simulation results are author-reported: 100% success at stage 3 for 0.3 m and 0.2 m pipes without obstacles, and 93.75% with obstacles at 0.2 m 2412.13621 tab. IV. The policy crossed three real PVC pipes without obstacles. The text says the real success rate is much lower than in simulation, and the real trial count is not stated 2412.13621 §III-D. Card: Learning Quadrupedal Robot Locomotion for Narrow Pipe Inspection.

Per-episode crop-row orientation jitter for a field robot

A four-wheel-steering farm robot is trained in Gazebo with deep RL to follow crop rows and move between waypoints. The robot and goal are placed at random at each episode start, and the crop rows get "a small random change in orientation" each episode 2412.18865 §III-C. The magnitude is not stated 2412.18865 §III-C. The author-reported success rate is 351 of 420 randomised trials, 83.57%, from simulation only [arxiv:2412.18865 tab. IV] §V. There are no real-robot results to test whether the jitter carried over. Card: Autonomous Navigation of 4WIS4WID Agricultural Field Mobile Robot using Deep Reinforcement Learning.

Random scene generation for synthetic training data

Kubric's worker script builds one scene per run: a ground plane, a light, a random camera, and objects from an asset library with random materials and random initial velocities, placed without overlap 2203.03570 §3.2. Simulation and rendering then produce RGB, depth, segmentation, flow and object metadata 2203.03570 §3. The MOVi video sets vary the number of objects and, in MOVi-E, move the camera 2203.03570 §4.1. This is randomisation of content for training data. It is not randomisation of a policy's physics, and the card does not say that its scene sampling was tuned to transfer to real images. Card: Kubric: A scalable dataset generator.

When to use which

Implementation sketch

Relabelling: change what the agent sees and does, not the environment. The paper's training loop is the reference for this variant 2003.02979 §5.

# Asymmetric relabelling per episode  [arxiv:2003.02979 §5]
# Input:  env, symmetry set Phi (sample uniformly), one policy per agent
# Output: trained policies

def train_with_relabelling(env, agents, symmetries):
    # Repeat until the training budget is spent  [arxiv:2003.02979 §5]
    while training:
        # Draw an independent relabelling for each agent  [arxiv:2003.02979 §5]
        phis = [sample_uniform(symmetries) for agent in agents]
        # Each agent acts and observes through its own relabelling; the
        # underlying game is unchanged  [arxiv:2003.02979 §5]
        trajectories = run_episode(env, agents, phis)
        # Apply the base self-play update  [arxiv:2003.02979 §5]
        for agent, trajectory in zip(agents, trajectories):
            update(agent, trajectory)
    return agents

Physics: resample the parameters for the environments being reset. The values are the ranges in tab. 18 of the Isaac Gym paper. The two Digit cards give no numerical ranges, so the same sketch applies there once you have chosen them. LocoMamba's table gives the same kind of ranges for a quadruped, with some parameters scaled by a factor and others drawn as absolute intervals 2508.11849 tab. 3.

# Per-reset physics randomisation  [arxiv:2108.10470 app. A.4.1] [tab. 18]
# Input:  sim (many envs on GPU), ranges (tab. 18), reset_ids
# Output: sim with fresh physics parameters for the reset envs

def randomise_on_reset(sim, ranges, reset_ids):
    # Multiplicative terms, e.g. masses and friction  [tab. 18]
    for param in ranges.scaled:
        factor = sample(param.dist, param.low, param.high, len(reset_ids))
        sim.scale(param.name, factor, reset_ids)
    # Additive terms, e.g. joint limits and gravity  [tab. 18]
    for param in ranges.additive:
        offset = sample(param.dist, param.sigma, len(reset_ids))
        sim.shift(param.name, offset, reset_ids)

Contradictions

No disagreement between the cards on the same quantity. The thirteen cards use the name for different purposes (symmetry relabelling, physics, dynamics and terrain randomisation for legged, humanoid, hand and field robots, and synthetic scene generation), so their results are not compared directly. The hardware-facing numbers measure different things on different robots and tasks, so they should not be set side by side: the Digit V3's 5% failure rate over 20 motions 2408.07295 §5.6, the original Digit's zero falls over a week of outdoor walking 2303.03381 §Outdoor deployment, Isaac Gym's 55% TriFinger success 2108.10470 §6.4.2, the pipe card's three real pipes, whose trial count is not stated 2412.13621 §III-D, the Mini Cheetah card's 9 of 10 hardware steps at 7.5 cm [arxiv:2311.02304 tab. 2], and the Cassie hardware top speeds, 3.0 m/s for the base walker against 3.9 m/s in simulation and 0.8 m/s for the carrying pole against 2.5 m/s [arxiv:2204.04340 §V] tab. 2. The gaps noted above still stand. The Isaac Gym card does not say which randomisations, if any, its TriFinger real-robot run used 2108.10470 §6.4.2. The original Digit card names its randomised factors but not their ranges 2303.03381 §Domain randomization. The Cassie and Holosoma cards name their randomised quantities but give no numerical ranges 2204.04340 §III-A 2512.01996 §3.1.

Within the diffusion-policy card, the text and the table disagree on the randomised target at 2M. The text says only perturbations give strong results at that size, and that the other setups perform poorly 2411.01349 §5.2. The table gives 0.96 success for terrain with perturbations, 0.99 for "All", and 0.57 for terrain alone 2411.01349 fig. 3. The card also notes that the label "All" is ambiguous, because the commented-out tables in the source name it "All but scales" 2411.01349 Pitfalls. The two readings give different headline numbers, so check the composition before comparing.

Papers

• Other-Play for Zero-Shot Coordination: applies per-agent random relabelling of known game symmetries to self-play, and reports higher cross-play and human scores in Hanabi (author-reported).
• Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning: GPU physics simulator whose Shadow Hand OpenAI task randomises physics parameters over the ranges in tab. 18; speed-ups are author-reported.
• Real-World Humanoid Locomotion with Reinforcement Learning: walking policy for the original Agility Digit, trained in Isaac Gym with dynamics, terrain and delay randomisation and a teacher–student split; zero-shot outdoor results are author-reported.
• Kubric: A scalable dataset generator: Apache-2.0 framework that renders random scenes with physics and ground-truth annotations; its dataset comparisons are author-run.
• Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots: one masked-directive LSTM controller for the Digit V3, trained in MuJoCo with dynamics randomisation and reported on hardware (author-reported; no code released).
• Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads: PPO walkers for Cassie under four dynamic loads, with specific, general and bootstrapped policies; simulation results and hardware top speeds are author-reported.
• LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba: quadruped PPO policy with per-episode physics ranges in tab. 3; simulation-only results against a Transformer baseline (author-reported).
• Learning Sim-to-Real Humanoid Locomotion in 15 Minutes: off-policy FastSAC and FastTD3 recipe for G1 and T1 locomotion and G1 whole-body tracking under push, delay, gain, mass and friction randomisation; timings and results are author-reported, mostly in figures.
• Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion: clones a DDP-based MPC expert into a small MLP with DAgger, then fine-tunes it with PPO under friction, gain and delay randomisation; hardware and simulation results are author-reported.
• Learning-based legged locomotion; state of the art and future perspectives: survey of legged locomotion learning that covers domain randomisation as a sim-to-real method; it has no new experiments and gives no ranges.
• The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control: compares eight randomisation setups for collecting data from an RL source policy on a Unitree H1, at three dataset sizes; results are author-reported and simulation-only.
• Learning Quadrupedal Robot Locomotion for Narrow Pipe Inspection: PPO quadruped for narrow-pipe crossing with friction, mass and push randomisation; simulation success is high and real-pipe results are author-reported with no trial count.
• Autonomous Navigation of 4WIS4WID Agricultural Field Mobile Robot using Deep Reinforcement Learning: deep RL for crop-row navigation with only per-episode orientation jitter as randomisation; simulation-only, author-reported.