Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge

In plain words

A robot hand has many joints, so teaching it to manipulate objects by trial and error takes a very long time. Most earlier methods start the hand from one fixed, hand-chosen pose and then let learning search the whole space around it, which is slow when the object sits somewhere new. This paper gives the learner a better starting point.

The method works in two stages. First, the robot looks at the object, finds the part a person would grab or push (such as a bucket handle or a laptop lid), and computes a grasp pose for the hand that targets that part. Second, a learning algorithm starts from that pose and adjusts the hand and arm to finish the task. The grasp stage does the hard search for where to stand and how to hold; the learning stage only has to refine the motion from there.

The paper reports better success and faster learning than the fixed-start baseline in simulation on four tasks, and some success on a real robot arm with two of the tasks. The real-robot test is small, about ten trials per task.

TL;DR

• Dexterous manipulation is split into an initial grasp stage and a PPO refinement stage §3.1 fig. 2.
• The initial grasp is built from AnyGrasp two-finger poses, mapped to a 16-DoF Allegro hand, filtered by collision checks, and chosen as the pose closest to the camera §3.1 eq. 2 eq. 5.
• In simulation on four DexArt tasks, the authors report higher success than DexArt at all four, e.g. 94.17% vs 79.17% on bucket tab. 1.
• The real UR5 experiments cover only the bucket and laptop tasks, with about 10 trials each §4.2 tab. 2.

Problem

Learning dexterous manipulation with reinforcement learning (RL) is slow and inaccurate because of the hand's many degrees of freedom §1 §3. Prior RL work often needs millions of time steps and usually fixes the grasp pose in advance §2.1. DexArt-style methods begin from a fixed, human-designed position for each task, which wastes learning time searching for a good position and viewpoint when the object is randomly placed §1. The paper uses one camera for the initial view and a separate RL camera for the observations during learning §3.1 §3.4.1.

Method

The pipeline has two stages fig. 2.

Stage 1: initial dexterous grasp.

1. Segment the functional part of the object. The initial point cloud P1 goes through a segmentation network S_e, giving P_f §3.1 eq. 1. The network is a PointNet trained on simulated renders; the paper uses GELU activations and cross-entropy loss §3.2.
2. Generate two-finger grasp candidates on P_f with AnyGrasp §3.1 eq. 2. Each candidate is G_hat = [R t w], with pose R and t and minimum grasp width w §3.3 eq. 4.
3. Map each candidate to dexterous grasps. The width axis is discretised into n values w_i. For each w_i, four hand configurations B_i1..B_i4 are designed manually §3.3 eq. 5 fig. 3. A dexterous grasp is G = [R t B], where B has 16 entries for the Allegro hand §3.3 eq. 3.
4. Remove invalid dexterous grasps with a collision check against P1 §3.1.
5. Keep the candidate closest to the camera as the initial grasp G §3.1.
6. Compute the initial joint angles of the arm with IKFast §3.1.

Stage 2: PPO refinement.

The problem is cast as an MDP with tuple <S, A, O, R', T, U> §3.1. The policy is trained with PPO §1 §3.1.

• Observation §3.4.1: (a) a point cloud P2 from the RL camera, encoded by a simplified PointNet; (b) robot proprioception S_r; (c) the object's goal position and rotation for the episode fig. 4.
• The feature extractor is pre-trained before PPO on four categories (functional part, non-functional part, robot arm, dexterous hand), with 6,000 point clouds per category. Its weights initialise the PPO extractor §3.4.1.
• Action §3.4.2: a 22-dimensional vector. The first 6 are the angular and linear velocity of the arm's end effector, integrated with the time step and converted to joint angles by IKFast. The last 16 are target joint angles for the Allegro hand, under PD control. The vector is scaled and clipped to its allowed range.
• Reward §3.4.3 eq. 6: R' = alpha r_in + beta r_co + eta r_re, with alpha, beta, eta as balance weights (values not stated).
  • Interaction r_in = r_finger + r_palm eq. 7. Finger term: a sum over fingers of min(-distance to the object's functional-part target, lambda) eq. 8. Palm term: min(distance between palm and object, gamma) eq. 9. lambda and gamma are regularisation caps (values not stated).
  • Completion r_co = Progress(task) + delta * IsCompletion * (H_s - H_c) eq. 10. Progress is task-specific; for the laptop it is the opening angle §3.4.3. H_s is the maximum step count, H_c the current step; both values are not stated.
  • Restriction r_re = r_robot + r_en eq. 11. Arm-motion term: -(norm of palm 6-D pose change + joint velocity) eq. 12. Collision term: minus the number of arm links in collision with the object eq. 13.

# Initial dexterous grasp  [§3.1] [eq. 1] [eq. 2] [eq. 5]
# Input:  initial point cloud P1 (camera view)
# Output: initial dexterous grasp G = [R, t, B]

def initial_grasp(P1):
    # Segment the functional part of the object  [§3.1] [eq. 1]
    P_f = segmentation_network(P1)

    # Generate two-finger grasp candidates on it  [§3.1] [eq. 2]
    two_finger = anygrasp(P_f)

    # Map each candidate to dexterous grasps, four hand types per width
    # [§3.3] [eq. 5] [fig. 3]
    candidates = []
    for grasp in two_finger:
        for hand_config in manual_hand_configs(grasp.width):
            candidates.append(make_dexterous(grasp.R, grasp.t, hand_config))

    # Drop grasps that collide, checked against P1  [§3.1]
    valid = [g for g in candidates if not collides(g, P1)]

    # Keep the grasp closest to the camera  [§3.1]
    return min(valid, key=distance_to_camera)

# Per-step reward  [§3.4.3] [eq. 6] to [eq. 13]
# Weights alpha, beta, eta and caps lambda, gamma, delta are not stated
def reward(state):
    # Weighted sum of the three reward groups  [§3.4.3] [eq. 6]
    r_in = interaction_reward(state)
    r_co = completion_reward(state)
    r_re = restriction_reward(state)
    return alpha * r_in + beta * r_co + eta * r_re

# Interaction reward  [§3.4.3] [eq. 7] [eq. 8] [eq. 9]
# finger_pos: (num_fingers, 3); obj_target: (3,)
# palm_rot, obj_rot: rotations, format not stated

def interaction_reward(state):
    # Fingers should approach the object's functional-part target  [eq. 8]
    r_finger = sum(
        min(-norm(finger - state.obj_target), lam)
        for finger in state.finger_pos
    )
    # Palm term as written: capped distance between palm and object  [eq. 9]
    r_palm = min(norm(state.palm_rot - state.obj_rot), gamma)
    return r_finger + r_palm

Hyperparameters

Values not in the paper or code are marked "not stated". Hyperparameters of PPO itself are not given anywhere in the paper.

Results

Author-reported. Simulation: XArm6 and Allegro hand, PartNet-Mobility objects, 3 seeds, 100 trials per object, DexArt as the baseline §4.1 tab. 1. Real world: UR5 with Allegro hand, two tasks, about 10 trials each §4.2 tab. 2.

Ablation on bucket and laptop, sim only §4.1.2 tab. 1 fig. 6:

The "Ours" rows are read as including the grasp B; §4.1.2 describes B as the additional prior on top of t and R, but no row label states it.

Other author claims §4.1.3: the method improves success by 15% to 29.31% over DexArt and improves learning efficiency by a factor of 80 to 150. These are not given as a table.

Paper vs code

No official code.

The paper, its arXiv metadata and its LaTeX source have no code link, and no project page is linked. GitHub searches were run for the title, for "dexterous grasp prior", "dexterous prior grasp", "dexterous grasp pose reinforcement", "dexterous hand PPO dexart", for the MVIG-SJTU organisation (searched by name; the paper does not name a lab), for the authors' names as GitHub user names, and for the public repositories of one author's account (Fang-Haoshu). Only unrelated repositories came back. None clearly belongs to the authors, so none is used.

Pitfalls

• Grasp selection is underspecified. The paper picks "the pose closest to the camera" after a collision filter §3.1. The distance measure, the collision model and whether the four hand configurations per width are all candidates are not stated. This choice fixes the initial state of every episode.
• Grasp width discretisation and hand configurations are unknown. The number of width bins n and the four hand configurations B_ij per bin are not stated §3.1 eq. 5. An implementation has to design these, and results will depend on them. Fig. 3 shows the mapping only as a diagram fig. 3.
• Rotation representation is ambiguous. Eq. 3 and Eq. 4 write R as a 3x1 matrix §3.3. The paper does not say whether it is axis-angle, Euler angles or another 3-vector form.
• The palm term may have the wrong sign for a "keep close" goal. As written, r_palm = min(distance, gamma) grows with distance up to the cap gamma eq. 9. The text says this term "ensures the proper amount of force" §3.4.3, which implies closeness is wanted. The paper also describes Y_palm and Y_object as rotations, yet eq. 9 takes a norm of their difference as if they were positions. Check the intended definition before using this term.
• The finger cap may be inactive. Eq. 8 caps each term at lambda, but the term is -distance, which is never above 0. For any lambda >= 0 the cap has no effect. The paper does not state lambda, so the intended form is not clear.
• Reward terms mix units with no weights. r_robot adds a 6-D pose change to a joint velocity eq. 12. The paper gives no scaling for either term. The balance weights alpha, beta, eta and the constants delta, H_s are not stated eq. 6 eq. 10.
• Progress(task) is only defined by one example. For the laptop it is the opening angle §3.4.3. Definitions for the bucket, faucet and toilet tasks are not given.
• Completion bonus depends on a step budget. The bonus delta * (H_s - H_c) is paid only when IsCompletion = 1 eq. 10, so it shrinks with time. Its size depends on the unstated H_s and delta.
• Text and table disagree on headline numbers. §4.1.3 gives 92% (bucket), 87% (laptop) and 87% (toilet) as the success rates at later steps, but tab. 1 reports 94.17, 90 and 87.88 for "Ours". The paper does not say whether the table shows the final checkpoint, the best checkpoint or a mean over seeds.
• The "15% to 29.31%" range does not match the table. The table differences against DexArt are 15.00 (bucket), 23.33 (laptop), 13.33 (faucet) and 29.31 (toilet) points tab. 1. The faucet gap is below the stated lower bound.
• Some ablation numbers are only in the text. Training-step values for the ablation appear only in §4.1.3 and fig. 6. The table has only final values. MGP and MGPR are not reported for faucet or toilet, marked "x" tab. 1; the paper does not say why.
• Training budget differs per task. The text quotes success at 0.4e6 to 6e6 steps, and the total number of training steps per run is not stated.
• Real-world results are small and partly reused. About 10 trials per task §4.2 give coarse percentages. The real-world policy is retrained in simulation with the real camera pose and rebuilt datasets §4.2; no real robot training is reported, and the sim-to-real method beyond this is not described. The claim that the method "is independent of robotic arm systems" rests on one extra arm and two tasks §4.2.
• Real-world setup needs extra scaffolding. Foam boards raise the workspace for the laptop, and a nail holds the bucket handle upright §4.2. These props are part of the reported setup and affect how the results transfer.
• Failure causes are described but not measured. The paper lists causes such as the palm getting stuck on the lid, or a bucket that is too heavy §4.2. No counts are given.

Lineage

• Builds on: DexArt (Bao et al., 2023), the benchmark and fixed-start baseline for articulated-object dexterous manipulation §3.1 §4.1.2. Not in this wiki.
• Uses: AnyGrasp for two-finger grasp generation (Fang et al., 2023) §3.1 eq. 2; PPO (Schulman et al., 2017) for refinement §1; PointNet (Qi et al., 2017) for segmentation and features §3.2 §3.4.1; SAPIEN with PartNet-Mobility objects §4.1.1. None of these is in this wiki.
• Compared against: DexArt, and its "big" workspace variant tab. 1; the paper's own MGP and MGPR ablations §4.1.2.
• Related: DexPoint (Qin et al., 2023) and "one-shot" imitation from teleoperation (Qin et al., 2022), cited in §1 and §2.1.
• Superseded by: not known from content already in this wiki.