Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer

In plain words

Opening a door is harder for a robot than it looks. The robot has to find the handle in a camera view that moves with its body, turn or push the handle against a spring, follow the door as it swings, and keep its balance the whole time. This work teaches a humanoid to do this from camera images alone.

The training has three stages. First, a "coach" policy learns in a physics simulator, where it can read the exact position of the door and the hands. Second, a "student" that sees only camera images learns to copy the coach while it moves the robot itself. Third, the student practises on its own, rewarded for succeeding. This lets it pick up habits the coach never showed, such as keeping the handle in view.

The simulator generates many door types with many surface looks and lighting, so the policy is not tuned to one door. The authors report that the robot opened real doors it never saw in training, and finished faster than human operators who used the same controller.

TL;DR

• DoorMan is a humanoid door-opening policy. At run time it takes RGB images and proprioception, with no depth and no door-pose estimate §2.1.
• Training has three phases: a teacher with privileged state trained by PPO, DAgger distillation into a ResNet-LSTM student, and GRPO fine-tuning with a binary success signal §2.1 §2.3 fig. 3.
• Staged resets restart the teacher from cached later-stage snapshots. In the paper's ablation, the teacher did not reach the grasp stage without this buffer §2.2 §3.4.
• Author-reported real-world result: 83% success, against 80% for expert and 60% for non-expert teleoperators, and faster completion than both groups §1 §3.1.
• Official code for DoorMan is on the doorman branch of NVlabs/GR00T-VisualSim2Real (Apache-2.0). It differs from the paper in several places, and GRPO was not found in the branch files that were read (see Paper vs code).

Problem

Door opening stresses a whole humanoid system at once. The robot must find the grasp point from a moving egocentric camera, rotate a spring-loaded handle, follow the circular path of the door panel, and stay balanced under hinge forces §1.

Earlier door-opening systems fall short for a general policy. They depend on depth sensing, hand-crafted features or primitives, and wheeled platforms. Some simplify contact with hooks or need accurate object localization. DARPA-era systems relied on scripting and operator intervention, and teleoperation pipelines are labour-intensive and brittle §1 §4.

Teleoperation is a bad teacher here. Operators struggle to judge the spring force of the handle and the hinge, and to track the swinging path §3.1. The paper also argues that an imitation policy's upper bound is set by the quality of the teleoperated data §3.

The paper names two challenges: the algorithm must be simple, scalable and robust to partial observability, and the visual sim-to-real gap spans a large range of appearance and physics §1.

The method assumes a pretrained whole-body controller for legged locomotion, HOMIE [ben2025homie], so that walking does not have to be learned from scratch §2.1. The policy runs at 50 Hz §2.1.

TL;DR for implementers

Read Method for the training pipeline, Hyperparameters for values the paper gives, and Paper vs code before copying any number. Several values the paper gives differ from the code, and the paper leaves the GRPO hyperparameters unstated.

Method

The paper calls the method DoorMan §1. Training has three phases: a teacher policy with privileged state, a vision-based student distilled from the teacher, and a GRPO fine-tuning of the student §2 fig. 3.

Setup. The problem is a partially observable MDP §2.1. The policy outputs target joint positions, which low-level PD controllers track. The paper gives the action dimension as 33 §2.1 (see Pitfalls). The HOMIE controller handles locomotion.

Teacher §2.1. The teacher receives privileged observations that are not available outside simulation: the root-to-door transform, the left and right hand-to-handle transforms, net contact wrenches on 18 hand bodies, and root linear velocity §2.1. It is trained with PPO and a stage-dependent reward in six stages: walk to door, pre-grasp, grasp, open, swing, and pass through the door app. A. Reward terms and weights are in app. A tab. 2.

Staged reset exploration §2.2. When an environment enters a new stage, the simulator state is cached in a rolling buffer that keeps the 100 most recent snapshots for that stage §2.2. At reset, the robot starts either from the initial state or from a cached middle-stage snapshot, with a nonzero probability. A staged reset law α gives the fraction of rollouts per stage §2.2 eq. 1. The initial distribution becomes a mixture of the stage reset distributions §2.2 eq. 2, and the occupancy measure shifts toward later stages §2.2 eq. 3. The authors' argument is that rare bridge crossings are rarely sampled from the start, so this shift gives later stages more gradient weight.

Student and DAgger distillation §2.1. The student gets RGB images and non-privileged proprioception: joint angles, joint velocities and root angular velocity. The image goes through a vision encoder (a ResNet, variant not stated). The visual features are merged with the proprioception vector. The merged vector runs through a two-layer LSTM (512 units each), followed by an MLP of widths 512, 256 and 128. The target joint angles come out of that MLP. The encoder is fine-tuned jointly with the policy. The student is trained with DAgger, so the supervision covers the student's own state distribution, not only the teacher's §2.1. The paper does not name the DAgger loss.

GRPO fine-tuning §2.3. The student is fine-tuned on its own rollouts with GRPO, an actor-only PPO variant with no value function. For a group of G rollouts, it normalises each return by the group mean and standard deviation §2.3 eq. 4. It then applies the clipped PPO surrogate to the new-to-old action probability ratio §2.3 eq. 5. The reward is mainly binary task success, plus joint-velocity, joint-acceleration and action-rate penalties §2.3. The paper reports that the student learns behaviours the teacher never showed, such as keeping the handle centred in view §2.3.

Simulation and randomization §2.4 app. B. Doors are generated procedurally in IsaacLab §2.4. There are five door types in three categories: push door with rotary handle, pull door with rotary handle, and push door with push bar §2.4. Door dimensions, handle location, hinge damping and handle resistive torque are randomized, and a latch model captures the sudden change in dynamics when the door opens §2.4. Surfaces get PBR materials that are redrawn every 0.9–1.1 s, and 5233 additional dome-light textures are used app. B. The camera is mounted on the robot, and its extrinsics and intrinsics are randomized slightly §2.4. Door property ranges are in app. B tab. 3.

# DoorMan: three training phases  [§2] [fig. 3]
# Input:  simulator (IsaacLab), pretrained controller (HOMIE)
# Output: RGB student policy

def train_doorman(sim, homie):
    # Phase 1: privileged teacher with PPO and staged resets  [§2.1] [§2.2]
    teacher = train_teacher(sim, homie)

    # Phase 2: distil the teacher into an RGB student  [§2.1]
    student = distill_with_dagger(sim, homie, teacher)

    # Phase 3: GRPO fine-tuning on the student's own rollouts  [§2.3]
    student = finetune_with_grpo(sim, homie, student)
    return student

# Phase 1: teacher  [§2.1] [§2.2] [app. A tab. 2]
# Snapshots per stage: 100 most recent  [§2.2]  (see Paper vs code)

def train_teacher(sim, homie):
    snapshots = {stage: RollingBuffer(size=100) for stage in range(6)}
    teacher = init_teacher_policy()
    while not converged:  # stopping rule not stated
        # Sample a reset stage from the staged reset law  [§2.2] [eq. 1]
        stage = sample_stage(alpha)
        # Start from the initial state or a cached snapshot  [§2.2]
        sim.reset_to(stage, snapshots)
        rollouts = collect(sim, teacher, homie)  # privileged observations
        # Cache a snapshot whenever an environment enters a new stage  [§2.2]
        for env_id in sim.entered_new_stage(rollouts):
            snapshots[sim.stage(env_id)].push(sim.snapshot(env_id))
        # PPO update with the stage-dependent reward  [§2.1] [app. A]
        teacher = ppo_update(teacher, rollouts)
    return teacher

# Phase 2: DAgger distillation into the RGB student  [§2.1]
# Image and proprioception in, target joint angles out

def distill_with_dagger(sim, homie, teacher):
    student = init_student_policy(
        encoder="ResNet",  # variant not stated [§2.1]
        lstm_units=512,  # two layers [§2.1]
        mlp_sizes=(512, 256, 128),  # [§2.1]
    )
    while not converged:  # stopping rule not stated
        # Roll out the student, so the data comes from its own states  [§2.1]
        rollouts = collect(sim, student, homie)
        # Label each visited state with the teacher's action  [§2.1]
        labels = teacher.act(rollouts.privileged_obs)
        # Regress the student toward the labels  [§2.1]  (loss not stated)
        student = fit(student, rollouts.rgb_and_proprio, labels)
    return student

# Phase 3: GRPO fine-tuning of the student  [§2.3] [eq. 4] [eq. 5]
# Group size G, clip epsilon and step count are not stated

def finetune_with_grpo(sim, homie, student):
    while not converged:  # stopping rule not stated
        # Sample a group of G rollouts from the current student  [§2.3]
        group = [collect_episode(sim, student, homie) for _ in range(G)]
        # Episode returns: binary success plus shaping penalties  [§2.3]
        returns = [episode.total_return for episode in group]
        # Group-relative advantage, normalised by mean and std  [§2.3] [eq. 4]
        advantages = [
            (r - mean(returns)) / std(returns)  # std can be zero, see Pitfalls
            for r in returns
        ]
        # Clipped surrogate, actor only, no value loss  [§2.3] [eq. 5]
        student = clipped_update(student, group, advantages, clip_epsilon)
    return student

Hyperparameters

Only values stated in the paper are listed. Code values are in Paper vs code.

Door property ranges app. B tab. 3:

Results

Author-reported throughout. Table 1 reports 120 unseen-door trials per configuration tab. 1; §3 says simulation evaluation uses holdout textures. Real-world results use the Unitree G1 in real scenes that were not in training, compared with teleoperators who use the same whole-body controller §2.4 §3 §3.1 app. C.

Real world vs teleoperation §1 §3.1 fig. 6.

The paper says DoorMan is "28% better than non-experts" in success. The 83% and 60% figures give 38% relative (23 points), so the paper's basis for 28% is unclear (see Pitfalls).

Visual randomization ablation tab. 1 §3.2. Success rates (%) on unseen doors, 120 trials per row.

The text says removing dome-light randomization costs 15–30 points, most of all on the pull lever. Ten percent of textures comes within 4–8 points of the full set §3.2.

GRPO fine-tuning §3.3 fig. 7a. When teachers reach 80–90% success, the initial student plateaus at 50–70%. After GRPO bootstrapping, student sub-task success is 80.8–85.8% §3.3.

Staged reset ablation §3.4 fig. 7b. With a buffer of 100 snapshots, the teacher reaches most stages within 500 iterations and all stages by about 1700. With 10 snapshots it takes more than 4000 iterations. With no buffer the exploration fails at stage 2 (grasp) §3.4.

Paper vs code

Official code. The paper links no repository. A GitHub search on the paper's title returned NVlabs/GR00T-VisualSim2Real, whose README names DoorMan and cites arXiv 2512.01061. The owner is NVIDIA's NVlabs organisation, and the title-page affiliations list NVIDIA first. The DoorMan code is on the doorman branch at commit 016c70c1e4e76f521963c36691ee69a6ab3ac9cd (2026-04-20). The default branch covers a different codebase (VIRAL), according to the README. The license is Apache-2.0, in the branch's LICENSE file and README.

The branch files were read via GitHub, not a clone: the default-branch clone failed at checkout on Git LFS assets, and the branch cannot be cloned with the permitted commands. Values are as read from those files. Line numbers were not captured, so the citations below give file paths only.

Discrepancies.

• GRPO is not in the code I could read. The paper reports GRPO fine-tuning §2.3 eq. 4 eq. 5. The DAgger trainer (gr00t/rl/trl/trainer/distill_trainer_obj_pred_homie_api.py) and PPO trainer (gr00t/rl/trl/trainer/ppo_trainer_homie_api.py) show no group-relative advantage. The PPO trainer uses a value loss and batch-level advantage normalisation. The branch listing was truncated, so an unlisted GRPO file cannot be ruled out. The README gives no GRPO command, and the configs carry no GRPO keys.
• Student recurrent size. The paper says 512 units per LSTM layer §2.1. The student config uses an LSTM with 256 hidden units and 2 layers door_open_homie_dagger-lstm.yaml. The teacher config also uses 256 units door_open_homie_lstm.yaml.
• Student vision encoder. The config uses a pretrained ResNet-18 with trainable weights and a 128-dim feature door_open_homie_dagger-lstm.yaml. The paper names no variant or feature size.
• Auxiliary object-position head. The student config adds an object-position head (MLP 256, 128, 64, output 3) on the vision features, with obj_pred_loss_coef: 0.0 door_open_homie_dagger-lstm.yaml. The README's student command sets it to 1.0 README.md. The paper describes no auxiliary head.
• DAgger settings. The student config has 4096 environments, 8 steps per environment, 4 mini-batches and teacher_rollout_ratio: 0.0 door_open_homie_dagger-lstm.yaml. The README's command uses 64 environments, 32 steps, 64 mini-batches and a ratio of 0.3 README.md. The README's values are applied on the command line and override the config. The paper states none of these.
• Teacher PPO settings. The teacher config has 4096 environments, 64 steps per environment, 5 epochs, 4 mini-batches, learning rate 1e-4 for actor and critic, entropy coefficient 0.01, initial noise std 0.8 (clamped), desired KL 0.005, gamma 0.9975 and lambda 0.985 door_open_homie_lstm.yaml. The README's teacher command sets entropy 0.001, 32 steps and 1024 environments README.md. The paper gives none of these values.
• Staged reset. The env config uses staged_reset_ratios: [0.5, 0.1, 0.1, 0.1, 0.1, 0.1] and staged_reset_max_samples_per_stage: 200 door_open_homie.yaml. The paper gives a buffer of 100 snapshots and no ratios §2.2. The ratios sum to 1, so they may be the α of eq. 1. The code that consumes them sits in base classes outside the files I read, so I did not verify that. The student config sets enable_staged_reset: False door_open_homie_dagger-lstm.yaml. The teacher config does not override it, so the env default True applies door_open_homie.yaml.
• Reward weights. All 33 weighted terms in the paper's reward table match reward_scales in the code, term by term app. A tab. 2 reward_door_open_homie.yaml. I did not check the Gaussian sigmas or the functional forms against the reward code. The config also enables a reward-penalty curriculum that scales 12 named terms between 0.2 and 1.0, which the paper does not mention reward_door_open_homie.yaml.
• Robot and action dimension. The paper gives an action dimension of 33 and a G1 with 29 body and 14 hand joints §2.1. The robot config is g1_43dof door_open_homie_dagger-lstm.yaml. The config sets actions_dim: 31, and the policy outputs 19 values: 3 locomotion commands, 14 upper-body joints and 2 finger primitives. Each finger primitive maps to a preset hand pose door_open_homie_dagger-lstm.yaml door_open_homie.yaml.
• Limits the paper does not state. The env config sets max_episode_length_s: 20, per-stage time limits [250, 100, 100, 100, 100, 200], and reset on completion after 50 s door_open_homie.yaml. The camera is 216×384 RGB, attached to d435_link door_open_homie_dagger-lstm.yaml.
• Undocumented reset data. The env config has a reset_from_dataset block that uses LAFAN-G1 motion files, plus a pred_model and a random_model (MLPs of 3101 inputs and 1024×4 hidden units) with a baseline error door_open_homie.yaml. The paper describes neither. I did not trace how they are used.
• Teacher checkpoint name. The student config points to a teacher checkpoint named door_open_homie_pregrasp door_open_homie_dagger-lstm.yaml. The env class DoorPregrasp handles all six stages door_open_homie.py. Read the name as a run label, not as stages 0–1 only.
• Possible bug. Reading distill_trainer_obj_pred_homie_api.py, the teacher branch writes policy_state_dict["gt_action"], but the key set earlier is gt_actions. If that branch runs, it raises a KeyError. I have not run this.

No discrepancies were checked for door randomization ranges, the 50 Hz rate, or the evaluation harness.

Pitfalls

• Paper inconsistencies to check before implementing.
  • The paper says the G1 has 29 body and 14 hand joints, and then gives an action dimension of 33 §2.1. 29 + 14 = 43, not 33. The code uses 43 DoF in the robot config, and its policy outputs 19 values (see Paper vs code).
  • The paper gives the speed-up over experts as 23.8% in §3.1 and 23.1% in the §1 introduction §1 §3.1.
  • "28% better than non-experts" in §3.1 is not derivable from the 83% and 60% figures, which give 38% relative (23 points) §1 §3.1.
  • §2.1 uses ξ_RD for both the root-to-door transform and the right-hand-to-handle transform, so the notation is ambiguous.
• GRPO is under-specified. The paper gives no group size G, clip ε, learning rate, number of steps, or KL term. Eq. 4 divides by std(R), so an all-success or all-failure group divides by zero. With a binary success reward, that case is common. The paper does not say how to handle it, and the code could not be checked for it.
• The staged reset is expensive and needs exact sim state. Each cached snapshot holds the generalized coordinates of every articulated and rigid object in the scene §2.2. The buffer size is 100 in the paper and 200 per stage in the config. The stage probabilities α are not stated in the paper.
• The student never sees privileged inputs. The teacher uses contact wrenches, exact handle transforms and root velocity §2.1. The student must infer these from RGB and proprioception. The paper's fix is GRPO on its own rollouts §2.3; its gains are reported in simulation only §3.3.
• Results depend on the locomotion controller. Every number is measured with HOMIE as the low-level controller, and the paper states that teleoperators used the same controller §2.1 §3.
• Teleoperation comparison is small and not fully specified. Expert and non-expert group sizes and the number of real-world trials are not stated §3.1 app. C.
• Table 1 setting is implied, not stated. Section 3 says simulation evaluation uses holdout textures §3. Table 1's caption says only "unseen-door trials" tab. 1. Treat it as simulation unless the authors say otherwise, and note the paper does not state that it was also run in the real world.
• Real-world door set is thinly described. Beyond the three door categories and varied handles and wall panels, the paper gives no list of real doors §3 fig. 2.

Lineage

• Builds on: HOMIE [ben2025homie] as the pretrained whole-body controller §2.1; DAgger [ross2011dagger] for distillation §2.1; GRPO [grpo2024] for fine-tuning §2.3; the first-return exploration idea [first_return2021] and a stage-based reward design [wococo2024] for staged resets and stage rewards §2.2; IsaacLab [isaaclab2025] for simulation §2.4.
• Compared against: teleoperation by expert and non-expert operators using the same controller §3.1; solid-colour randomization, like earlier RGB sim-to-real work [tobin2017domain] [zhu2018reinforcement] §3.2; the procedural generator Infinigen-Sim [ifinigensim2025], which the paper says it improves on §2.4; scripted and hook-based door systems §1 §4.
• Superseded by: not known.