Haptic Stiffness Perception Using Hand Exoskeletons in Tactile Robotic Telemanipulation

In plain words

A person wears a glove that controls a robot hand far away. The question is whether the person can feel how hard an object is while squeezing it through that robot, with no camera view, only the pressure the robot's fingertips feel pushed back through the glove.

The robot hand has sensors in its fingertips, and the glove pushes back on the person's fingers in proportion to what those sensors measure. The authors tried two ways of doing this. The simple one sends only the squeeze force. The second also takes into account how far the person's fingers moved compared with how far the robot fingers moved, because the glove and the robot hand do not move the same way.

Ten people tried to tell soft objects apart by feel. Their results were above chance, and the extra movement signal seemed to help mainly when two objects felt alike.

TL;DR

• Ten naive participants discriminated five soft silicone samples that differed only in stiffness, using haptic feedback alone §4.2 §4.1.
• Method I renders the follower's contact force; Method II multiplies it by a displacement ratio and a range factor §3.2.1 §3.2.2.
• ABX success averaged about 75% for both methods, above chance; the paper reports that the difference between the methods is not significant overall §5.2 tab. 2.
• The displacement term appears to help for similar-stiffness pairs, but the pair-level tests are mostly not significant §5.5.
• The paper has no released code, and several statistical numbers do not check out (see Pitfalls).

Problem

Telemanipulation with a remote robot often needs the operator to judge properties that cameras show poorly, such as stiffness §1. Prior work on stiffness perception through haptics is mostly about autonomous robot sensing or surgical systems with visual feedback; the authors report no earlier study of human stiffness perception through a hand exoskeleton with remote tactile sensing and no vision §1 §2.

A second problem is kinematic mismatch: the leader glove and the follower hand move their fingers over different ranges, so a force that is correct for the robot can feel wrong to the operator §3.2.2 §7.

Method

The setup has a leader side (HGlove exoskeleton glove on the human hand) and a follower side (Allegro robot hand with tactile fingertips) §3.1 fig. 1. Only three follower fingers are used (thumb, index, ring). They map to the glove's thumb, index and middle fingers §3.1.

Each follower fingertip has four magnetic force sensors that report forces in three directions. The fingertip force is the largest z-force of the four sensors eq. 1. Readings below a minimum threshold produce zero feedback for both methods §3.1 eq. 2 eq. 3.

Method I (contact force). Leader force is the follower force times a constant scale. The scale maps the sensor's maximum usable reading to the glove's maximum force of 5 N §3.2.1 eq. 4 eq. 5. The authors describe this as previously introduced in their earlier work §1 §3.1.

Method II (contact force times displacement factors). The authors first define a stiffness estimate from force over follower displacement eq. 6. The final rendered force does not use this estimate directly. Instead, it is Method I's force times two factors eq. 9:

• Δ_Z, the ratio of leader finger displacement to follower displacement during a squeeze eq. 7;
• β, the ratio of the maximum follower displacement to the maximum leader displacement, which normalises the two different motion ranges eq. 8.

The pseudocode below implements both methods as the paper describes them. Units of the sensor forces and displacements are not stated.

# Method I and Method II leader-side force  [§3.2] [eq. 1-9]
# Input:  fz (4,) z-forces of the four sensors in one fingertip
#         dz_leader, dz_follower: z-displacements in one finger pair
#         dz_leader_max, dz_follower_max: maximum travels (not stated)
# Output: leader_force1, leader_force2 for one glove finger (N)

def method_one(fz):
    # Largest of the four sensor z-forces  [§3.1] [eq. 1]
    follower_force = max(fz)
    # Readings below the minimum give zero feedback  [§3.1] [eq. 2-3]
    if follower_force < F_MIN:
        return 0
    # Above F_MAX the paper states no behaviour  [§3.1] [eq. 2]
    # Scale so the maximum reading maps to the glove limit  [§3.2.1] [eq. 5]
    alpha = F_L_MAX / F_MAX                  # 5 N / 1000 (units not stated)
    return alpha * follower_force

def method_two(fz, dz_leader, dz_follower, dz_leader_max, dz_follower_max):
    # Force from Method I, zero below the minimum  [§3.2.2] [eq. 9]
    force1 = method_one(fz)
    # Leader-to-follower displacement ratio  [§3.2.2] [eq. 7]
    delta_z = dz_leader / dz_follower        # undefined if dz_follower is 0
    # Ratio of the maximum follower to leader travel  [§3.2.2] [eq. 8]
    beta = dz_follower_max / dz_leader_max
    # Method I force times both factors  [§3.2.2] [eq. 9]
    return force1 * delta_z * beta

Constants: F_MIN = 30, F_MAX = 1000, F_L_MAX = 5 N §3.1 §3.2.1.

Hyperparameters

Results

Author-reported. Setup: 10 naive participants, five silicone samples, ABX and S tasks, glove-side haptic feedback only, no visual feedback §4 §5. Method I is the force-only baseline; Method II is the proposed method §3.2.

Per-group and per-day results (group and day means) §5.3 fig. 4:

Statistical tests §5.1 §5.4 §5.5:

• ABX: 16 of 24 correct gives the paper's stated 66.67% success; 17 of 24 (70.8%) is the paper's stated threshold for significance at 95% §5.1.
• Two-way ANOVA on Group, Day and Task: no significant main effects or interactions reported tab. 3 §5.4.
• Mann-Whitney U per pair and task: only the pair (1-US, 4-LH) in Task S differs (p = 0.048) §5.5.
• Mann-Whitney U per distance group: ABX D=1 p = 0.061 and D=3 p = 0.067; no group below 0.05 §5.5.
• Mean ABX success by distance: D=1 Method II 88% vs Method I 75%; D=3 Method I 80% vs Method II 65% §5.5.
• Mean Task S success by distance: D=1 Method I 73% vs Method II 68%; D=3 Method I 65% vs Method II 58% §5.5.
• The authors state that one further participant with 95% ABX and about 80% S success was excluded from the statistics §6.

Paper vs code

No official code. The paper and arXiv listing link no repository or project page §1 [arXiv abstract page]. GitHub repository searches for "haptic stiffness telemanipulation" and "HGlove allegro tactile" returned nothing, and a GitHub user search for "giudici qmul" returned nothing. The paper cites its earlier setup work, DexSkills (arXiv:2405.03476), and the tactile-fingertip work §3.1. Those are not treated as this paper's code.

Pitfalls

• Binomial numbers do not match. The paper gives P(X >= 16) ≈ 10.6% and P(X >= 17) ≈ 4.3% for 24 trials at p = 0.5 §5.1 eq. 10 eq. 11. Summing the same binomial terms gives about 7.6% and 3.2% (our recomputation, not in the paper). The conclusion that 17 of 24 is significant at 95% still holds; the printed confidence levels should not be copied.
• Stated ABX confidence depends on the numbers above. The Discussion's "confidence higher than 95%" for ~75% success §6 is not re-derived in the paper.
• Mean vs median. The text calls 67.91% a "median" §5.2, but Table II reports it as a mean tab. 2.
• Pair count. Section 4 says there are eight pairs, each tested three times §4.5. Section 5 says seven pairs §5.5. The eight entries include (1-US, 5-H) and (5-H, 1-US), which are the same unordered pair §4.5. This is probably why there are seven.
• ANOVA table looks copied. In Table III the Group:Task row repeats the Day row (SS 94.60, F 13.42, p 0.170), and Day:Task repeats Group:Day (SS 4.25, F 0.60, p 0.580). The residual has 1 degree of freedom tab. 3. The text repeats the same F and p for Group × Task §5.4. Treat this table as unreliable.
• Unit of analysis is unstated. It is not clear whether the Mann-Whitney and ANOVA tests use participants, pairs, trials, or sessions as observations, or whether the excluded outlier is out of Table II §5.2 §5.4 §5.5.
• Group sizes are unstated. Groups 1 and 2 are defined by starting method §5.3. Their sizes are not given.
• Trial structure is ambiguous. Section 4.5 says a session has 24 ABX and 24 S tests in three sets of eight, but also says sessions alternate methods over two days §4.5. Confirm whether 24 per task is per session or per day.
• Method II may exceed the 5 N limit. Method I is capped by its scale factor §3.2.1. The paper states no clamp for Method II, and Δ_Z·β can be greater than 1 §3.2.2. The paper gives no cap there.
• Division by zero. Δ_Z divides by follower displacement eq. 7, which is zero before contact. The paper does not say how this is handled.
• Displacement term is defined but absent. The stiffness estimate K eq. 6 is not used in the final rendered force eq. 9. The text describes Method II as based on stiffness estimation §3.2.2.
• Units missing. Sensor thresholds 30 and 1000 have no stated units §3.1. The 5 N limit is in newtons §3.2.1. The mapping between them is implied, not given.
• Maximum displacements unknown. The values ΔZ_L^max and ΔZ_F^max, which set β, are not reported §3.2.2 eq. 8.
• Objects are hard to reproduce. Stiffness is given only as Shore hardness labels and product names §4.1. No modulus or measured stiffness is reported.
• Human-subject study. Results depend on ten participants, with one outlier excluded §6. Training used visual feedback §4.3, and the ABX sequences were the same on both days, which participants did not know §4.5.
• Authors' own caution. The paper says Method II did not give consistently significant improvements and that force is the more important component §6.

Lineage

• Builds on: Method I comes from the authors' earlier tactile telemanipulation work §1 §3.1, cited as "Feeling good" (not linked to an arXiv id here). The teleoperation setup is partly from DexSkills, arXiv:2405.03476 §3.1.
• Compared against: Method I, the force-only feedback, within this paper. Related telesurgery and haptic teleoperation work is reviewed in §2.
• Superseded by: not known from content already in this wiki.