Object-Centric Dexterous Manipulation from Human Motion Data

In plain words

Teaching a robot to move an object into a target pose is hard when the robot has two multi-finger hands. Each hand has many joints, and a robot hand is not built like a human hand, so copying human finger motions does not carry over.

The paper splits the job in two. A first model learns, from recordings of people handling objects, where the hands should go as the object moves. A second controller, trained by trial and error in simulation, works out the finger motions that make the object actually follow the path, and it may nudge the hand a little when the robot's hand differs from a human's. Trial runs that succeed are fed back to improve the first model. The learned controllers were also run on a real two-arm robot with dexterous hands.

TL;DR

• Object-centric task: the goal is a sequence of object poses; success is the object following it §3.
• Hierarchy: a behavior-cloned transformer planner gives wrist poses; PPO learns fingers plus a bounded wrist residual §4.1 §4.2.
• A data augmentation loop alg. 1 widens the training data; the authors report it helps generalization §5.3.
• All headline numbers are author-reported. The released code does not contain the planner or the augmentation loop (see Paper vs code).

Problem

Dexterous manipulation with two multi-fingered hands means controlling both arms and both hands, a high-dimensional action space that makes plain RL hard §1. Human motion data could guide training, but the embodiment gap between human and robot hands means finger trajectories do not transfer well; the authors argue that wrist motion transfers better §1. Retargeting human hand poses by inverse kinematics does not guarantee task success across embodiments §2. Teleoperation data on a real robot is expensive to collect §2. The task is defined by the sequence of object poses, each an SE(3) transform plus a joint angle for articulated objects §3.

Method

The framework has a high-level planner, a low-level controller and a data augmentation loop, plus a sim-to-real step §4 fig. 2.

High-level planner §4.1

A Transformer generative model, called π^H here, takes an object category ID c and a window of the desired object trajectory G from time t to t+T. It outputs a sequence of 6-DoF wrist actions, one 6-DoF pose for the left and one for the right hand at each step. The horizon is T = 10 §4.1. It is trained with behavior cloning on human wrist motion from the ARCTIC dataset: 339 sequences, totalling 2.1 million steps §4.1.

Low-level controller §4.2

A PPO policy, π^L, controls the robot. Its inputs are the current observation s_t, the object goal window G, and the planner's wrist sequence. It outputs finger joint actions a^F_t. It also outputs a residual wrist correction Δa^W_t, which is limited to a fixed range of ±4 cm in translation and ±0.5 rad in rotation §4.2. The final robot action combines the planned wrist pose and the residual with the finger action §4.2.

The reward is the exponential of a weighted distance between the object's current pose and the goal pose, with a weight for rotation, translation and joint angle §4.2 eq. 1 app. B. The paper gives λ1 = 20 for rotation, λ2 = 1 for translation and λ3 = 5 for the joint angle app. B.

The observation vector in simulation has 527 entries, including the planned wrist actions and the goal window app. B.

# Reward per step  [§4.2] [eq. 1] [app. B]
# lambda_1 = 20 (rotation), lambda_2 = 1 (translation), lambda_3 = 5 (joint)
def reward(obj_rot, obj_trans, obj_joint, goal_rot, goal_trans, goal_joint):
    # Angular gap via the quaternion dot product  [app. B]
    rot_gap = angle_between(obj_rot, goal_rot)
    trans_gap = norm(obj_trans - goal_trans)
    joint_gap = abs(obj_joint - goal_joint)
    # Weighted sum of the three gaps  [eq. 1]
    distance = 20.0 * rot_gap + 1.0 * trans_gap + 5.0 * joint_gap
    # Exponential map shapes the distance into a reward  [app. B]
    return exp(-distance)

Data augmentation loop §4.3 alg. 1

Training only on ARCTIC limits the object shapes and motions the policy sees §4.3. The loop applies three augmentations during RL training of π^L: object size per axis, initial object pose, and goal-trajectory waypoints §4.3 app. A. Successful rollouts of π^L are added to the human data and used to fine-tune the planner, in rounds alg. 1 §4.3.

# Data Augmentation Loop  [alg. 1] [§4.3]
# Input:  human data (goals G, wrist actions), number of rounds L
# Output: planner pi_H and controller pi_L

def data_augmentation_loop(human_data, rounds):
    # Initialise the planner and the controller  [alg. 1]
    planner = init_planner()
    controller = init_controller()
    train_set = []
    for m in range(rounds + 1):
        # Fit the planner on human data plus augmented samples  [alg. 1]
        while not converged(planner):
            train_set += augment(human_data)  # [app. A]
            planner = train_planner(planner, train_set)
        # Fit the controller on the planner's wrist plans  [alg. 1]
        while not converged(controller):
            controller = train_controller(controller, planner, human_data)
        # Keep the successful rollouts  [alg. 1]
        successes = rollout_successes(controller, human_data)
        # Add them to the data for the next round  [alg. 1] [§4.3]
        human_data = human_data + successes
    return planner, controller

# Augmentations during RL  [app. A]
# Lengths in cm and angles in degrees, as stated in [app. A]
def sample_augmentation(rng):
    # Object size scale for width, length, height  [app. A]
    scale = rng.uniform(0.9, 1.1, size=3)
    # Initial object pose: x and y offset, and yaw about z  [app. A]
    offset_xy = rng.uniform(-0.02, 0.02, size=2)
    yaw_deg = rng.uniform(0.0, 30.0)
    # Goal trajectory: per-axis offset added to each waypoint  [app. A]
    goal_offset = rng.uniform(-0.02, 0.02, size=3)
    return scale, offset_xy, yaw_deg, goal_offset

Sim-to-real transfer §4.4 app. C

Some observations cannot be measured on the robot, such as joint velocity and object velocity. A teacher-student distillation removes them from the policy input §4.4. Distillation uses DAgger app. C. Four top-down cameras feed articulated-object pose estimation with FoundationPose, running at 15 Hz. Each object part's pose is estimated and the joint angle is computed from them §4.4. Each camera's object pose is compared with the expected pose; the text says a pose is removed "if the error is smaller than" 5 cm and 0.5 rad, which reads inverted app. C. The surviving poses are averaged, and the previous frame is reused if none survive app. C. The policy's finger commands go through an exponential moving average low-pass filter to reduce jitter §4.4; the smoothing factor is not stated.

Hyperparameters

Values from the paper. "Not stated" means the text, appendix and figures do not give the value.

Domain randomization in simulation, from the paper's table. Scaling multiplies the nominal value; additive adds noise app. E:

Randomization is refreshed every 1000 simulation steps app. E.

Simulation observation vector, index ranges app. B:

Real-world observation vector, index ranges app. C:

Results

All results are author-reported. Simulation numbers are means with standard deviation across 10 seeds. Real-world numbers are over 20 trials §5. "Completion" is the percentage of the goal trajectory the policy completes §5.2.

Planner accuracy tab. 1. TE is translation error in cm. The unit of OE is not stated. Errors are cumulative over the whole sequence §5.1. The baselines MLP and RNN are named only in this table:

Real-world transfer, one policy per object trained in simulation on one goal trajectory tab. 2 §5.4. Completion in %:

Simulation, one policy per object tab. 3 §5.2. Completion in %:

Generalization tab. 4 §5.3. Completion in %. Single-object policies are evaluated on trained and unseen trajectories; multi-object policies on trained and unseen objects:

Comparison with sample-based MPC app. H. Completion in %, same nine objects as tab. 3:

Goal representation app. F: the Box task and a Box task with a manually designed goal trajectory each reach 100 ± 0.0 completion.

Time-index robustness app. G, coffee maker, completion in %: Skip 2 85.7 ± 7.5; Skip 1 86.5 ± 3.4; Origin 86.1 ± 5.5; Inter 1 85.4 ± 8.1; Inter 2 87.2 ± 4.8; random sampling of time gaps 86.2 ± 6.2.

Figures: fig. 1 teaser; fig. 3 simulation and real workspaces, two Shadow Hands on separate UR10e arms; fig. 4 four hand types of different size and degrees of freedom, with over 50% completion on all four §5.2; fig. 5 four real-world tasks (coffee maker, laptop, mixer, notebook) §5.4.

Paper vs code

Repository checked: https://github.com/cypypccpy/ObjDexEnvs at 420c4b260456430e189074ac85cca95dedf678da. The paper's project page links it [project page]. It is a shallow clone. The repository holds the simulation environment, rl_games training configs and a trajectory loader. It does not hold the planner, the augmentation loop, the distillation code or the real-world stack. The README cites arXiv 2309.00987 as the paper; that id is not this paper.

• Planner absent. Paper trains a transformer planner by behavior cloning §4.1. The release has no planner model or training code. high_level_planner/data_utils.py only loads ARCTIC data data_utils.py.
• Reference is human data, not planner output. The paper's controller takes the planner's wrist sequence §4.2. The environment reads the ARCTIC human wrist trajectories as its reference dexterous_hand_arctic.py data_utils.py. The planned-wrist path is not in the release.
• Augmentation loop absent. use_p_c_impro_loop is read from the config dexterous_hand_arctic.py and set to False in dexterous_hand_arctic.yaml, but nothing else uses it.
• Reward weights differ. Paper: weight 20 on rotation, 1 on translation, 5 on joint angle app. B. Code: 20 on position, 2 on rotation, 2 on joint dexterous_hand_arctic.py.
• Reward has extra terms. Dead zones subtract 0.05 m, 0.1 rad and a joint tolerance from the object gaps, and 0.15 m and 0.5 rad from each hand gap dexterous_hand_arctic.py dexterous_hand_arctic.py. Reward adds an energy penalty of -1e-6 times squared torque-velocity products dexterous_hand_arctic.py dexterous_hand_arctic.py. The paper mentions neither.
• Jitter penalty unused. A 0.003 × squared-action term is computed but not added to the reward dexterous_hand_arctic.py.
• Hierarchy off by default. The shipped config has use_hierarchy: False dexterous_hand_arctic.yaml; the flag turns it on train_rlgames.py. With hierarchy on, the wrist action scales are 0.04 m and 0.5 rad, and the reward is multiplied by the hand-tracking terms dexterous_hand_arctic.py dexterous_hand_arctic.py. With hierarchy off, the scales are 0.02 m and 0.1 rad dexterous_hand_arctic.py. The paper says ±4 cm and ±0.5 rad §4.2. The code scales raw actions and does not show a clamp; any clamp inside rl_games was not checked.
• Trajectory density. The time-index study uses the original ARCTIC goal trajectory as its "Origin" reference app. G; the paper does not say which density its main results use. The task sets interpolate_time = 1 dexterous_hand_arctic.py, which inserts one linear midpoint between each pair of poses data_utils.py data_utils.py.
• Undocumented trajectory edits. The loader drops the first 30 frames data_utils.py. It adds fixed hand offsets (z -0.07 and x +0.07 for the right wrist, z -0.05 and x -0.07 for the left, y +0.04 for both) data_utils.py. It replaces object rotations that jump by more than 0.5 in Frobenius norm data_utils.py. None of this is in the paper.
• Episode length. Paper: 500 action steps §5.2. Code resets at start + 500 dexterous_hand_arctic.py dexterous_hand_arctic.py. The config's episodeLength: 200 is overwritten by the trajectory length dexterous_hand_arctic.yaml dexterous_hand_arctic.py.
• PPO settings differ. Config uses rl_games a2c_continuous arctic.yaml. Discount is 0.99, with the paper's 0.998 commented out arctic.yaml. Hidden sizes are [1024, 512, 256], not [1024, 1024, 512, 256] arctic.yaml. Mini-batch size is 4096 arctic.yaml, not 4 mini-batches. Initial sigma is a constant 0.0 with fixed sigma arctic.yaml; whether that equals the paper's 0.8 depends on rl_games and was not checked. The other values match: horizon 8 [#L51], 5 epochs [#L53], clip 0.2 [#L48], learning rate 3e-4 [#L40], KL 0.02 [#L56].
• Domain randomization disabled. The shipped config has randomize: False dexterous_hand_arctic.yaml. Its refresh interval is 600 steps dexterous_hand_arctic.yaml, not the paper's 1000 app. E. Object friction is 0.7 to 1.3 dexterous_hand_arctic.yaml, not 0.5 to 1.5. Joint damping and stiffness use 0.3 to 3.0 and 0.75 to 1.5 dexterous_hand_arctic.yaml dexterous_hand_arctic.yaml, not the paper's [0, 0.01]. Joint limits are additive, not scaled dexterous_hand_arctic.yaml. Object position noise is not in the config.
• Joint reset depends on object. The joint reset is 0.2 for the espresso machine and 0.5 otherwise dexterous_hand_arctic.py. The paper states one joint threshold of 0.5 rad §5.2.
• Completion metric absent. The release has no code for the paper's completion rule §5.2. It resets an episode when the object moves 5 cm, rotates 0.5 rad or exceeds the joint reset dexterous_hand_arctic.py. That is not the same as completion.
• Licence. The README says MIT and links a LICENSE file that is absent. Cite by path and line only; do not quote.

Pitfalls

• Planner and augmentation loop must be rebuilt. The release has the RL environment and training scripts, but not the planner or the loop. Anyone reproducing the full method must write both from the paper.
• Algorithm 1 has a typo. The final update reads D ← D_T + D_s^t alg. 1. D_T is undefined; the text says successes go into the training data for the planner §4.3, so the intended set is probably the human data D.
• Augmentation units conflict. Appendix A says lengths are in centimetres and angles in degrees app. A. Then the offset ±0.02 means 0.2 mm, which is implausible. The main text gives no units §4.3. Choose metres or centimetres deliberately; the code does not implement these augmentations, only object scale in domain randomization dexterous_hand_arctic.yaml.
• Residual indexing is open. Paper calls the planner output at time t the wrist action a^W_t §4.2, but does not say which entry of the T-step sequence the controller uses at each step. Whether the residual is clipped or scaled is not stated.
• Joint-space conversion is not described. The paper gives wrist and finger actions but not how they become joint targets. The release uses inverse kinematics with Jacobians for the arms dexterous_hand_arctic.py.
• Training data for the planner. The paper trains the planner on "the entire ARCTIC dataset" with 339 sequences §4.1. The RL set is 9 objects with 16 sequences each, and the rest is unseen §5. Whether the planner ever saw the unseen-trajectory test sequences is not stated, so the generalization numbers in tab. 1 may be optimistic.
• Object count. The data section says ARCTIC has 10 objects §5; the limitations say there are 11 §6. The abstract says 10 household objects [abstract].
• Real-world numbers are not multiples of 5%. With 20 trials per reported rate §5, a single completion rate should be a multiple of 5%. Values such as 69.8 and 76.7 in tab. 2 therefore come from some other aggregation (over trials, seeds or both), which is not stated.
• Inverted pose-rejection rule. The camera filter says a pose is removed "if the error is smaller than a threshold" app. C. Read literally, it keeps only the bad poses. The intended rule is probably removal when the error is larger.
• Text versus tables. "More than a 50% completion rate improvement" over prior methods §5.4 does not hold for Microwave: 100 versus 60.6 is a 39.4-point gap tab. 2. "Barely any success (<20%)" is not true for Vanilla RL on Box, at 20.4 tab. 2.
• Average over objects. The text says the method beats Vanilla RL by 47.3% on average §5.2. The mean of the nine Ours values in tab. 3 is 71.8 and the mean of Vanilla RL is 20.8, a 51.0-point gap. The averaging is not stated; these means were computed here from the table.
• Completion threshold wording. "2.5 centimeters in object's longest dimension multiplied by rotation angle" §5.2 does not say which unit the angle is in. The code does not implement it, so pick one reading and state it.
• Units of planner error. TE is in cm; the unit of OE is not stated §5.1. The MLP and RNN baselines in tab. 1 are not defined anywhere in the text.
• Simulation to real-world gap. Real-world episodes fail when the start pose is far from the goal, and when the hand occludes the object, which corrupts pose estimation §6.
• Joint order. The method needs a fixed joint order for articulated objects, which the authors say limits multi-joint objects §6.
• Cross-category generalization. The authors force a Box policy onto a Laptop by reusing the category ID §6, so this is not a category-level test.
• Observation table placement. The simulation observation table is referenced from app. B but typeset in app. C. Use the index ranges as written.

Lineage

• Builds on: PPO (schulman2017proximal); behavior cloning and DAgger (ross2011reduction); teacher-student distillation (chen2022visual, chen2021system, rusu2015policy); ARCTIC motion data (fan2023arctic); FoundationPose for pose estimation (wen2023foundationpose). Named by bibliography key only.
• Compared against: finger joint mapping, DexMV-style (qin2022dexmv); fingertip mapping by IK, DexCap-style (wang2024dexcap); vanilla PPO on arm and hand together; sampling-based MPC (pezzato2023samplingbased) app. H.
• Superseded by: not known from content already in this wiki.