GEX: Democratizing Dexterity with Fully-Actuated Dexterous Hand and Exoskeleton Glove

In plain words

Robot hands that can do fine work, such as turning an object in the fingers, are expensive, and the paper says the ones built for research are costly and hard to maintain. Simpler and cheaper grippers are common, but they cannot do much dexterous work. This paper presents a small robot hand with three fingers and a glove that you wear. When you move your fingers in the glove, the robot hand copies the motion in real time. Recording those movements is also a way to collect demonstrations for teaching robots new skills.

Every joint in both the hand and the glove has its own motor. That lets the system know the position of every joint and not just guess from a few sensors. The parts are 3D-printed plastic and the whole kit is meant to be cheap, so that more labs can afford to work on this kind of robot.

The software maps the glove's finger positions onto the hand's joints. The glove and the hand have different shapes, so this mapping is not a simple copy. The paper's pitch is cost and openness: the designs and software are released so others can build and improve the system.

TL;DR

• A teleoperation kit: a three-finger anthropomorphic hand (GX11, 11 DoF) and an exoskeleton glove (EX12, 12 DoF), joined by kinematic retargeting §1 §2 §3.2.
• Every joint has its own servo motor (23 DoF in total), chosen so joint positions are observable and the kinematics can be modelled exactly [abstract] §2.1 §2.2.
• Structural parts are 3D-printed ABS. Author-stated cost is $600 per device and under four hours of assembly §1.
• The glove can apply force feedback through the motors' current mode. This part is described in the paper but has no code in the released repository §3.3.
• Results are qualitative only: one real grasp of a paper cup, and three simulated tasks. No success rates, latency or baselines are reported §4 §4.1.

Problem

Dexterous hands are costly and often need a lot of maintenance because of their tendon or mechanical complexity. Direct-drive hands are cheaper, but the paper says even the Allegro hand retails above $16,000 §1. Teleoperation for collecting demonstrations is hard with many-DoF hands, since the operator needs precise motion capture and the robot needs a matching mapping §1. Underactuated or tendon-driven designs make the joint state hard to observe and the kinematics harder to model; the paper argues that full actuation avoids this [abstract] §5.

Method

GEX has three parts.

Hardware: the GX11 hand §2.1. A three-finger hand built on the LEAP Hand design with direct joint drive, no tendons. Thumb 3 DoF, index 4 DoF, middle 4 DoF, 11 DoF in total §2.1. Each joint is a Dynamixel XL330-M288-T servo with a 288:1 reduction and an absolute magnetic encoder §2.1.

Hardware: the EX12 glove §2.2. An exoskeleton glove worn on the human hand: a 4-DoF thumb and 4-DoF index and middle fingers, 12 DoF in total §2.2. Its servos (XL330-M077-T) have a smaller 77:1 reduction so that gear friction does not resist the wearer's finger motion §2.2.

Control SDK §3.1. A 48 MHz Cortex-M0+ control board (OpenRB-150) drives all 11 hand servos and the glove servos over a multidrop bus up to 1 Mbps. A pure-Python library, libgex, talks to the board. It runs on Linux and Windows §3.1. The library is not in the repository cloned for this card (see Paper vs code).

Dexterous retargeting §3.2. The glove's fingertips and the palm base are treated as keypoints. The paper frames retargeting as an optimization over the hand's joint trajectory eq. 1. The cost has two parts: a term that matches fingertip poses to scaled glove fingertip poses, and a smoothness term that penalizes changes from one step to the next. Joint limits are a hard bound eq. 1. The scaling factor α and the smoothing weight k are described as positive values but not given numerically eq. 1.

# Retargeting, one control step  [eq. 1] [§3.2]
# Input:  glove_keypoints (4, 3): palm base and three fingertips, m
#         previous_joints (n,): joint angles from the last step, rad
# Output: joint_angles (n,): new hand joint angles, rad

def retarget_step(glove_keypoints, previous_joints):
    # Minimise pose mismatch plus a smoothing penalty, within joint
    # limits  [eq. 1]. The paper gives no values for alpha or k.
    joint_angles = solve_constrained(
        pose_mismatch,       # matches scaled glove fingertip poses
        smoothing_weight,    # penalizes change from previous_joints
        previous_joints,
        lower_limits,
        upper_limits,
    )
    return joint_angles

Force feedback §3.3. When a hand finger touches an object, the paper says the matching glove finger switches to an impedance mode so the operator feels resistance. Contact is detected with tactile sensors, or by identifying the dynamics from the authors' earlier work §3.3. The paper gives no gains or thresholds.

# Force-feedback switch, per finger  [§3.3] [fig. 10]
# The paper's first sentence does not say which device's motors are meant;
# read here as the glove, and not confirmed by code.

def update_finger(contact_detected, glove_finger):
    if not contact_detected:
        # Free motion: only joint positions are recorded  [§3.3]
        glove_finger.mode = "record_position"
    else:
        # Contact: feedback through impedance control  [§3.3]
        glove_finger.mode = "impedance"
    return glove_finger

Hyperparameters

Results

Author-reported. The paper gives no numerical metrics for retargeting, teleoperation accuracy, latency or task success. Setup: a single GX11 hand with a robot arm, teleoperated from an EX12 glove for one real task §4; three simulated manipulation tasks §4.1. No baseline is reported.

The Shadow Hand price is not stated; the paper gives only "well over six figures" §1.

Paper vs code

Code: Democratizing-Dexterous/gex_retargeting_sim at commit df80783ba2146b8021988ce3f2348df7f779e565, MIT licence. The paper links the organisation's repository list [abstract] and the repository itself §3.2. The repository contains URDF models, a retargeting wrapper and two demos. It does not contain the firmware, libgex, or the force-feedback logic. libgex is linked from the paper §3.1 but was not cloned for this card. The repository was found through the abstract's organisation link; no GitHub search was needed. libgex and Democratizing-Dexterous/gex_retargeting_sim are the two repositories the paper links for the software.

• Retargeting method. The paper gives a generic constrained optimization with a smoothing term eq. 1. The code uses DexPilot from the dex-retargeting library, selected in the config [code: retargeting/configs/teleop/gx10_right_dexpilot.yml#L2] and built in gex_retarget.py. The solver itself is not in this repository, so the smoothing weight k and the scale α of eq. 1 have no direct counterpart in the released files.
• Keypoint vectors. The paper matches fingertip poses eq. 1 fig. 7. The code forms six vectors from four keypoints: three fingertip-to-fingertip and three fingertip-to-palm-base vectors gex_retarget.py. The retargeter receives relative vectors, not absolute poses.
• Smoothing. The paper's smoothing term has weight k eq. 1. The config sets a low-pass alpha of 1 and says smaller values filter more [code: retargeting/configs/teleop/gx10_right_dexpilot.yml#L11-L12], so no extra filtering is set in the config.
• Degrees of freedom. The paper says GX11 has 11 DoF with a 3-DoF thumb §2.1. In gx10.urdf there are 10 revolute joints: four in the chain joint1 to joint4 gx10.urdf, three in joint5 to joint7 gx10.urdf, and three in joint8 to joint10 gx10.urdf. The file names the robot gx10, not GX11. Code 1 calls the first glove finger the thumb [Code 1]. The thumb chain in the URDF therefore has four revolute joints, not three.
• Glove DoF. The EX12 URDF has 12 revolute joints, from joint1 at ex12.urdf to joint12 at ex12.urdf, which matches §2.2.
• Total DoF. The paper gives 23 DoF [abstract]. The two URDFs have 10 + 12 = 22 revolute joints.
• Listing vs demo. The paper listing imports libgex.libgx11 and libgex.libex12 and calls hand.connect(goal_pwm=600) and glove.connect(init=False) and glove.fk_finger1() [Code 1]. The demo imports from libgex import Hand, Glove after adding a libgex2 path, calls hand.connect(), glove.connect() and glove.fk(), which returns three fingers demo_retarget_real.py demo_retarget_real.py.
• Units. The real demo converts joint angles from radians to degrees before sending them to the hand demo_retarget_real.py. The paper states no units for the retargeting output. The listing's setj takes degrees [Code 1].
• Force feedback. No code in the repository for impedance mode, current control or contact detection. A search for impedance, current, torque and feedback terms in the Python, Markdown and YAML files found no matches. §3.3 describes it, and the design is not released here.
• Real loop timing. The real demo has no fixed delay [code: demo_retarget_real.py#L49, sleep commented out]. The simulation uses a 1/240 s sleep demo_retarget_pybullet.py.
• Sim keypoints. In simulation, glove fingertip positions come from PyBullet link states at fixed link indices 4, 9 and 14 [code: demo_retarget_pybullet.py#L33-L35] demo_retarget_pybullet.py. The real path uses the glove's own forward kinematics from libgex, which is not in the repository.

Pitfalls

• The formulation is not the code. Eq. 1 with α and k is the paper's framing. The working method is DexPilot from dex-retargeting, which is pinned only by name in the readme (pip install dex_retargeting, no version) [readme.md]. Reimplementations must use the library, not the equation. The config scaling_factor 1.05 is not the paper's α [code: retargeting/configs/teleop/gx10_right_dexpilot.yml#L9].
• Relative keypoint vectors. The code matches relative vectors among four keypoints gex_retarget.py. Matching absolute fingertip poses in the palm frame gives a different target.
• Joint limits. The paper states joint bounds q_l, q_u as hard constraints eq. 1. The numerical limits are not stated in the paper. The config loads the robot from gx10.urdf [code: retargeting/configs/teleop/gx10_right_dexpilot.yml#L3], so limits come from the URDF.
• Keypoint frame. The palm base is the zero vector in the glove frame in both demos demo_retarget_real.py [code: demo_retarget_pybullet.py#L52]. Its exact position on the glove is shown only in a figure fig. 7.
• Torque and power numbers. Rated torque is 0.53 N·m §2.1. No peak torque, no thermal limit and no maximum sustained current are stated.
• Sim and real loops differ. The simulation loop is paced at 240 Hz demo_retarget_pybullet.py. The real loop has no fixed pacing demo_retarget_real.py. Any timing that depends on loop rate, such as smoothing, will differ between the two.
• Serial setup. The paper's listing and the demo both need access to /dev/ttyACM* and /dev/ttyUSB* on Linux [Code 1]. The repo's helper runs chmod 777 on those devices with sudo using_serial.sh. Use narrower permissions in practice.
• Force feedback. The paper gives no gains, no thresholds and no description of the controller's stability. The one sentence that could be read as describing the hand's motors does not say which device is meant §3.3.
• Figure labels. The label fig:gx11_range is used twice in the LaTeX source, once for the GX11 range plot and once for the EX12 range plot. The text cross-references are ambiguous as a result §2.1 §2.2. This card numbers figures in source order.
• Cost claims. Cost and assembly time come from the authors and have no breakdown §1.
• Results. All results are qualitative. There is no accuracy, latency, success rate or repeated trial. Treat the three simulated tasks as a demonstration of range, not a benchmark §4.1.

Lineage

• Builds on: the LEAP Hand design for the GX11 hand's direct-drive joint layout, arXiv:2309.06440 §2.1. No card in this wiki.
• Retargeting builds on the dex-retargeting library (DexPilot). The paper names it in a footnote §3.2 and the repo's readme says "based on dex-retargeting" [readme.md].
• Cited for teleoperation difficulty: /papers/2307.04577 (AnyTeleop) §1.
• Compared against: none in the paper.
• Superseded by: not known from content already in this wiki.