Harmonic Mobile Manipulation

In plain words

Mobile manipulation means moving the body and using an arm at the same time. Opening a door is the typical case: the robot has to reach for the handle while its body moves so it does not hit the door. Many robot systems avoid the problem by splitting the job in two, first driving to a spot and then reaching. That split works for simple pick-and-place, but it breaks when the body has to move during the reach.

This paper trains one neural controller that chooses the base motion and the arm motion together at every step. It looks at two camera images and at the arm's own joint readings. It is learned by trial and error in simulated apartments. The controller earns points for moving toward the target and for moving its hand toward the handle or table, earns a large bonus for finishing, and loses a little for wasted motion. The base simulator had no door opening and no table cleaning, so the authors added both. The trained controller was then run on a real robot in an apartment it had never seen, with no further training.

For an engineer the useful part is the recipe: a frozen pretrained image model, a small recurrent network, and a reward with explicit shaping terms. The card below lists those values and the places where the released code differs from the paper.

TL;DR

• One PPO-trained policy controls base motion, arm height and extension, and wrist rotation together, from RGB and proprioception only §IV-C §IV intro.
• Four simulated tasks in ProcTHOR: opening a door by pushing or pulling, cleaning a table, opening a fridge §III §V-B.
• Author-reported average success over the four tasks is 47%. The paper puts this 17.6 points above the ReLMoGen adaptation §V-B; the ReLMoGen average from tab. I is 29.4% (computed here as the mean of its four rows).
• Real-robot trials in one unseen apartment without fine-tuning: cleaning 66.6% (8/12), pull 60.0% (9/15), push 70.0% (7/10) tab. III.
• The official code is public (Apache-2.0), but several reward and training values differ from the paper. Use the Paper vs code section before copying numbers.

Problem

Navigation and manipulation are usually split into separate skills or stages, either with a language-model planner or with motion planners §I. The authors argue that such splits fail for tasks that need both at once, such as opening a door, where the body can collide with the door §I. They also argue that the split is inefficient, and that the Stretch arm's limited reach makes concurrent base and arm control necessary for large tables §I.

Existing simulators were also judged inadequate. ProcTHOR offers visual diversity but has no door-opening or table-cleaning logic §I §IV-A. The paper does not measure how well the split approaches work on the new tasks outside its own comparisons §V-B.

Method

Overview

The paper proposes HarmonicMM, an end-to-end RL policy for four tasks grouped in a new benchmark called the Daily Mobile Manipulation Task Suite §III. The policy receives RGB from two cameras and a 5-dimensional arm proprioception vector, and emits continuous base and arm commands at every step §IV §IV-B. It is trained with DD-PPO in AllenAct, in ProcTHOR houses with added articulation and cleaning logic §V-A §IV-A. The two-camera design (a Navigation Camera and a Manipulation Camera on the arm) is shown in fig. 2.

Steps of the method:

1. Simulator changes. Door and fridge openness is set from the end-effector's motion: a push or pull that lines up with the door normal opens it in small increments, and collisions with the robot body are checked §IV-A. The simulator's door state is not governed by physical rules in stock ProcTHOR §IV-A. Table cleaning places dirt objects on the table, attaches a sponge to the end effector, and removes dirt on contact. The removed count feeds the reward §IV-A.
2. Action space. Five continuous dimensions: base translation and rotation, arm height and extension, and wrist rotation. Door and fridge tasks add a grasp dimension §IV intro fig. 3. In code the base action class has 5 dimensions and the grasp task has 6 opening_door.py opening_door.py.
3. Observations. Two RGB views, the Navigation Camera and the Manipulation Camera, plus a 5-dimensional proprioception vector covering arm base lift, arm extension and wrist rotation §IV-B fig. 2. The paper says RGB only, with no depth or LiDAR §IV-B. Input is resized to a square 224×224 in code main.yaml dinovit_aug_preprocessor.py.
4. Frozen visual backbone. Each RGB view goes through a DINOv2 ViT-S/14 model, pretrained and not updated (run under no_grad) app. Network Architecture dinovit_aug_preprocessor.py. The 16×16 patch-token grid is pooled to 7×7 app. Network Architecture dinovit_aug_preprocessor.py.
5. Per-camera compression. Each camera has its own compressor, E_Nav for the navigation view and E_Manip for the manipulation view. The paper lists two 1×1 convolutions with 16 channels app. Network Architecture. The code uses 1×1 convolutions from 384 to 64 channels and then to 16, with ReLU after each ncamera_dino.py.
6. Joint embedding. The two flattened 16×7×7 maps are concatenated with the raw 5-dimensional proprioception, giving 2·784 + 5 = 1573 features app. Network Architecture. In the code this path skips the proprioception MLP ncamera_dino.py ncamera_dino.py.
7. Memory. One GRU layer with 512 units consumes the joint embedding app. Network Architecture ncamera_dino.py opening_door_multimodalv2_dinovitgru.py.
8. Heads. The actor maps the 512-d state to the means of the action dimensions. The critic maps it to one value. Action spread is a learned log-standard-deviation per dimension continuous_visual_nav.py continuous_visual_nav.py.
9. Reward. The paper's reward has three parts: navigation, manipulation and efficiency §IV-D. The exact terms are in the pseudocode below, from app. Reward Function.
10. Augmentation. During training the RGB input gets colour jitter, Gaussian blur, random resized crop, posterisation and sharpness changes app. Visual Augmentation dinovit_aug_preprocessor.py. The code turns augmentation off at evaluation main.yaml.
11. Training. PPO-style updates with 20 parallel environments and Adam app. RL training Hyperparameters §V-A base.py. The paper gives 10^7 steps and three seeds §V-A. See Paper vs code.

Key equation (the paper gives it unnumbered §IV-D):

R = w_{nav} R_{nav} + w_{manip} R_{manip} + w_{eff} R_{eff} §IV-D

The equation is unnumbered in the source, so cite it as §IV-D. The weights are w_nav = 1, w_manip = 1 and w_eff = 1 app. Reward Function.

Policy step

# One policy step for HarmonicMM  [§IV-C] [app. Network Architecture]
# Input:  nav_rgb, manip_rgb (3, 224, 224); proprio (5,); hidden (512,)
# Output: action mean (5,), value (1,), next hidden (512,)

def policy_step(nav_rgb, manip_rgb, proprio, hidden):
    # Frozen DINOv2 ViT-S/14 features, pooled to 7x7 per camera  [app. NA]
    nav_tokens = pool_7x7(dino_vits14(nav_rgb))      # (384, 7, 7)
    manip_tokens = pool_7x7(dino_vits14(manip_rgb))  # (384, 7, 7)

    # Separate 1x1 conv compressors E_Nav and E_Manip, 384 -> 64 -> 16
    # [app. NA] [code: training/models/ncamera_dino.py#L538-L543]
    nav_feat = compress_nav(nav_tokens)              # (16, 7, 7)
    manip_feat = compress_manip(manip_tokens)        # (16, 7, 7)

    # Flatten both views and append the raw proprioception  [app. NA]
    joint = concat(flatten(nav_feat), flatten(manip_feat), proprio)
    # joint has 1573 features

    # One GRU layer with 512 units adds memory  [app. NA]
    hidden = gru(joint, hidden)                      # (512,)

    # Linear actor and critic heads  [code: continuous_visual_nav.py#L80-L110]
    action_mean = linear_actor(hidden)               # (5,)
    value = linear_critic(hidden)                    # (1,)
    return action_mean, value, hidden

Reward per step (door and fridge tasks)

The following restates the paper's terms §IV-D app. Reward Function. The constants shown are the paper's values; the Paper vs code section lists where the released configuration differs.

# Per-step reward for door and fridge tasks  [§IV-D] [app. Reward Function]
# Constants are the paper's. Shaping uses the L2 distance type in code.

def step_reward(state, action):
    # Step penalty and penalty on end-effector motion  [app. Reward Function]
    reward = -0.01 - w_ee_moved * ee_moved_distance
    # Navigation shaping: 2 per metre closed toward the target  [app. RF]
    # Only while the robot has not yet reached the target
    reward += 2.0 * max(closest_dist - dist_to_target, 0.0) * (1 - reached)
    # One-time bonus when the robot first reaches the target  [app. RF]
    if reaches_target_now and not reached:
        reward += 2.0
    # Manipulation shaping toward the handle, applied until the grasp  [app. RF]
    shaping = 0.02 * exp(-5.0 * dist_ee_to_knob) * 1000.0
    reward += shaping * max(closest_ee_dist - dist_ee_to_knob, 0.0)
    # Progress: change in openness, scaled by 80  [app. RF]
    reward += 80.0 * openness_delta
    # One-time grasp bonus  [app. RF]
    if grasp_succeeded_now and not grasped_before:
        reward += 2.0
    # Completion bonus  [app. RF]
    if task_complete_now:
        reward += 20.0
    return reward

Note that the paper's formulas for the cleaning table differ. Cleaning uses a per-dirt bonus and an arm-height gate in code. See Paper vs code.

Training loop

# PPO training loop  [§V-A] [app. RL training Hyperparameters]
# 20 parallel environments, Adam, 4 optimisation epochs per rollout

def train(policy, envs):
    # The first stage collects 32 steps per env  [code: opening_door_base.py#L520]
    # 20 envs x 32 steps = 640 samples per iteration [app. RL training]
    for iteration in count():
        rollout = collect(policy, envs, steps_per_env=32)
        # Advantages from GAE with gamma 0.99, lambda 0.95 [app. RL] [code]
        advantages = gae(rollout, gamma=0.99, lam=0.95)
        for epoch in range(4):
            # One mini-batch: the full rollout  [code: main.yaml#L76]
            loss = clipped_ppo_loss(rollout, advantages, clip=0.1)
            loss += -0.0025 * entropy(policy)
            # Learning rate 5e-5 for policy and value  [app. RL]
            adam_step(policy, loss, lr=5e-5, max_grad_norm=1.0)

Hyperparameters

Values are from the paper's appendix or main text unless the Notes column says otherwise. Each row cites the paper section or the code file that gives it.

Results

Author-reported. Setup: ProcTHOR-based simulation with a Stretch robot, 350 unseen houses, 300 episodes per task, and the baselines described in §V-B. The paper reports mean ± standard deviation over three seeds in its tables; the paper does not define the spread, and the source gives no seed count per cell.

Success and progress in simulation tab. I:

Efficiency in simulation tab. I, bottom:

Average success over the four tasks is 47% §V-B. The Two-Stage baseline is modelled on OVMM §V-B. The paper says its method is 17.6 points above ReLMoGen on average (37% relative) §V-B.

Real robot, one apartment with a novel layout, no fine-tuning tab. III:

Opening Fridge was not run on the real robot: the magnetic seal was too strong for the robot to pull §V-C.

Ablations:

• Proprioception tab. II. Push without proprioception: success 74.6 ± 1.1, progress 83.9 ± 1.4. With proprioception: 80.0 ± 19.9 and 89.7 ± 5.0. Pull without proprioception: success 5.2 ± 9.0, progress 11.0 ± 18.9. With proprioception: 51.2 ± 12.2 and 69.8 ± 2.7.
• Visual backbone and camera setup on Opening Door (Pull) tab. IV. No pretrained DINOv2: 0% success, 66.3% reached the door. Navigation camera only: 27.3% success, 47.9% progress. Manipulation camera only: 0% success, 0.235% progress. Adapted Skill Transformer: 0% success, 29.7% reached the door.
• Start distance fig. 6. The paper says the gain over ReLMoGen grows with the start distance for table and push tasks, from 1 to 4 m.

Paper vs code

Code: https://github.com/RchalYang/harmonic_mobile_manipulation at 76899cb5fa496fdcb430e0d62dfa89bf1fb01073. The project page links this repository. The README cites the paper and gives the venue as IROS 2024 README.md. Apache-2.0 LICENSE.

Search for code: the project page is the first hit. The GitHub search for the method name returned nothing.

Discrepancies, paper first:

1. Training length. Paper: 10^7 steps §V-A. The code's stage budgets are 10M, 10M and 980M, for a total of 10^9 opening_door_base.py. The config also sets ppo_steps: 10_000_000_000 main.yaml. The paper's number cannot be reproduced from the code's defaults.
2. Rollout length. Paper: 640 samples per iteration, which is 20 environments × 32 steps app. RL training. The code uses 32 steps per environment in the first stage only, then 64 and then 128 opening_door_base.py.
3. End-effector motion penalty. Paper: −0.01 app. Reward Function. Door and table runs use −0.05 in the config and in the scripts main.yaml open_door_pull.sh. The fridge script uses −0.01 open_fridge.sh.
4. Invalid-action penalty. Paper: −0.01 for all tasks app. Reward Function. Code: −0.002 for door and fridge main.yaml main.yaml, and −0.01 for table cleaning main.yaml.
5. Grasp bonus. Paper: 2 app. Reward Function. Code: 4.0 for door and fridge main.yaml, applied on the first grasp opening_door.py.
6. Completion bonus. Paper: 20 for all tasks app. Reward Function. Code: 20 for door and fridge main.yaml, 10 for table cleaning main.yaml.
7. Manipulation shaping weight and form. Paper: 0.02 for all tasks, with the exponential form app. Reward Function. Code: 0.02 for door and fridge with the exponential form main.yaml. For table cleaning, code uses weight 1 main.yaml and a different form: the moving reward is gated by arm height above 0.67 with no exponential cleaning_table.py.
8. Table-cleaning progress. Paper: progress weight 100 app. Reward Function. Code: 5 per dirt removed main.yaml, with 20 dirt points cleaning_table.yaml. That totals 100 only if every point is removed; the formula is not the paper's.
9. Door success threshold. Paper: over 90% §V-A. Code: pull at openness above 0.85 opening_door.py; push at openness above 0.90 opening_door.py.
10. Table success threshold. Paper: 75% §V-A. Code: progress above 0.9 cleaning_table.yaml, applied at cleaning_table.py.
11. Fridge success threshold. Paper: over 70% §V-A. Code has an openness threshold of 0.7 opening_fridge.py and also one of 0.90 opening_fridge.py. The paper does not say which branch its numbers use, and this card does not resolve it.
12. Fridge grasp distance. Paper: under 0.2 m app. Reward Function. The fridge script sets 0.3 open_fridge.sh.
13. Table-cleaning script overrides. clean_table.sh overrides mdp.reward.opening_door. The cleaning task reads mdp.reward.cleaning_table cleaning_table_base.py. So the overrides in the script do not reach the cleaning task.
14. Policy head. Paper: "1 FC with 512 units" for the policy and for the value app. Network Architecture. Code: the actor is a single linear layer from 512 to 5 outputs continuous_visual_nav.py. The critic is a single linear layer from 512 to 1 continuous_visual_nav.py.
15. Initial standard deviation. Paper: "standard Gaussian" app. RL training. Code: the standard deviation starts at 1.8 main.yaml and the log-std is clamped to [−3, 3] continuous_visual_nav.py.
16. Backbone width. Paper appendix: 768×16×16 features from DINOv2 app. Network Architecture, while its table names dinov2_vits14. ViT-S/14 has 384 channels; code uses 384 dinovit_aug_preprocessor.py. The 768 figure matches ViT-B, not the named model.
17. Input resolution. Paper table: 384×224 app. Network Architecture. Code resizes to 224×224 and asserts that shape main.yaml dinovit_aug_preprocessor.py.
18. Encoder layers. Paper table: 2 × Conv 1×1 with 16 channels. Code: 384→64→16 with ReLU ncamera_dino.py.
19. Optimiser and other PPO settings. Adam base.py, GAE λ = 0.95, gradient-norm clip 1.0 main.yaml and no value clipping opening_door_base.py are in the code only. The clip parameter 0.1 from the paper is not set in this repo, so it depends on allenact's default.
20. Seeds. Paper: three seeds §V-A. Scripts run seed 1 only open_door_pull.sh.
21. Navigation shaping form. Paper: 2 × max(closest − current, 0) app. Reward Function. Code uses the same L2 form with weight 2 base.py opening_door.py. This matches.
22. Door opening mechanics. Paper: openness is driven by the end-effector's motion relative to the door normal §IV-A. Code: openness is changed by an OpenObject call with a target openness opening_door.py. The simulator build with the paper's rule is not in this repo, so this card does not verify it.
23. Real-robot grasp heuristic. The paper says a heuristic grasp of the doorknob runs in the real world when the controller issues a grasp §V-C. This card did not find that code in the repo and did not check the robot folder for it.

Matches: the door and fridge manipulation shaping formula opening_door.py, the progress term as a per-step change in openness opening_door.py, the step penalty of −0.01, the 500-step cap main.yaml, the learning rate 5e-5 and entropy 0.0025 in the door and fridge scripts open_door_pull.sh, and the 20 parallel environments open_door_pull.sh.

Pitfalls

• Reward scale depends on which values you take. The paper's and the code's energy, invalid-action, grasp and completion values differ (see Paper vs code, items 3–6). A reward written from the paper will not match the released runs, and vice versa.
• Success thresholds differ by task and by source. Door, fridge and table values are not the ones in the paper's text (items 9–11). Compare success rates only at the thresholds used in each source.
• The paper's efficiency percentages do not recompute exactly from its table. From the printed values, progress speed is 32.7% higher for table cleaning, 112.8% for push and 28.1% for pull. The text says 32.2%, 113.4% and 27.6%. Use the table values.
• Standard deviations are undefined. The ± values do not say what the spread is over, and the seed count per cell is not stated.
• Real-robot success criteria are partly unstated. Door trials count "fully open". For table cleaning the paper reports the share of the surface cleaned in each trial, but does not give a pass mark for a trial to count as a success, so the 66.6% figure depends on an unstated threshold.
• The heuristic grasp is not described enough to implement. The paper says a doorknob heuristic is invoked for grasping in the real world §V-C. No details are given.
• Starting positions matter. The paper says start positions were tuned for exploration efficiency §V-A. The ranges are given, but not the tuning procedure.
• Action scale is a hidden parameter. The config multiplies each of the five action dimensions by 0.25 main.yaml. The paper does not state this.
• Scripts override the config. Learning rate, entropy and reward values in the shipped scripts replace the defaults in config/main.yaml. Read the script before copying a value from the config.
• Training data is not in the repo. The house counts (700, 2000, 1400) and the evaluation split (350 houses) cannot be checked from the repo.
• Frozen backbone. The code keeps the DINOv2 weights frozen (no_grad). The paper says only that a pretrained backbone is used, so an implementation must freeze it to match the code dinovit_aug_preprocessor.py.

Lineage

• Builds on: ProcTHOR (procedural houses, named in §I); DD-PPO and AllenAct (training, named in §V-A); DINOv2 (visual backbone, named in app. Network Architecture). No cards for these exist in this wiki, so nothing from them is used here.
• Compared against: ReLMoGen, adapted by the authors to the Stretch robot §V-B; a Two-Stage navigate-then-manipulate baseline modelled on OVMM §V-B; the Skill Transformer on the pull task tab. IV.
• Superseded by: not known from this wiki.