SoGraB: A Visual Method for Soft Grasping Benchmarking and Evaluation

In plain words

Soft robot grippers squeeze objects gently, but there is no shared test for how good one is. Existing tests count how often a grasp works, or how hard the gripper must pull to free the object. Neither says how much the object was squashed, which is what matters for fragile things.

SoGraB takes a 3D scan of the object before the grasp and another during it. It measures how far the surface moved and combines that with whether the object was held and how long it stayed held. The result is one number between 0 and 1 for each grasp. The authors use it to rank three soft finger designs and one rigid finger on a set of printed objects. The method needs only a robot arm, a depth camera and a printer, so other labs can repeat it.

TL;DR

• A scalar grasp score in [0, 1] built from success, drop time and a deformation distance between two point clouds eq. 1 eq. 2.
• Deformation is the Density-Aware Chamfer Distance (DCD) between the object before and after grasping, aligned with ICP §3.1 §3.2.
• Author-reported: 900 grasps (15 objects × 3 hardnesses × 4 grippers × 5 repeats), all successful, scores 0.517 to 0.940 §5.
• The released code is MIT-licensed, and its DCD and ICP details differ from the paper in several places. See Paper vs code.

Problem

Grasp success rate and retention force are the usual measures §1. Neither accounts for the force or deformation that the object experiences, and the paper argues that force sensors on the gripper or on the object are needed to get at force directly §1. Benchmarks for rigid grippers also assume the object stays still relative to the gripper during grasping §2. The paper sets out to build a benchmark that uses deformation as a non-contact proxy for stress, needs no instrumentation, and works with arbitrary gripper and object pairs §1 §6.

Method

Each grasp attempt is captured as one point cloud before grasping and one during grasping. The two clouds are aligned, and then the Density-Aware Chamfer Distance between them is computed. The score combines that distance with the grasp outcome §3.1.

Outcomes are grouped into three classes: unsuccessful (no grasp, or dropped before the grasped cloud was captured), partially successful (dropped after the grasped cloud was captured but before placing), and successful (held for a full pick-and-place cycle) §3.1. The score is given in eq. 1:

# Score for one grasp attempt  [eq. 1]
# Input:  outcome; distance d in [0, 1]; t_dropped, t_cycle in seconds
# Output: score in [0, 1]

def sograb_score(outcome, distance, t_dropped, t_cycle):
    # Unsuccessful grasps score zero  [eq. 1] [§3.1]
    if outcome == "unsuccessful":
        return 0
    # Partial success: half the range, weighted by hold time  [eq. 1]
    if outcome == "partial":
        return (1 - distance) * t_dropped / (2 * t_cycle)
    # Full success: the upper half, lowered by the deformation  [eq. 1]
    return 1 - distance / 2

Partial scores fall in [0, 0.5] and full-success scores in [0.5, 1] §3.1.

The distance is the average of the DCD in both directions, where each cloud takes a turn as the reference. A point's contribution is damped if its nearest neighbour in the other cloud is reused by many points, so that differences in point density do not dominate eq. 2 §3.1:

# Density-aware Chamfer distance between clouds S1 and S2  [eq. 2]
# Input:  S1 (N1, 3), S2 (N2, 3), sensitivity alpha
# Output: scalar in [0, 1]

def density_aware_chamfer(first, second, alpha):
    # Nearest neighbour of every point, in each direction  [eq. 2]
    nearest_in_second = nearest_index(first, second)    # (N1,)
    nearest_in_first = nearest_index(second, first)     # (N2,)
    # How many points picked each neighbour  [eq. 2]
    uses_second = count_uses(nearest_in_second, len(second))
    uses_first = count_uses(nearest_in_first, len(first))
    # Damped similarity term for each point, using the distance to its neighbour
    # [eq. 2]
    gap_first = distance(first, second[nearest_in_second])  # (N1,)
    gap_second = distance(second, first[nearest_in_first])  # (N2,)
    term_first = 1 - exp(-alpha * gap_first) / uses_second[nearest_in_second]
    term_second = 1 - exp(-alpha * gap_second) / uses_first[nearest_in_first]
    # Average the two directions  [eq. 2]
    return (mean(term_first) + mean(term_second)) / 2

Before the distance is computed, the two clouds are aligned. The grasped cloud is registered to the reference with ICP, starting from an estimate taken from the robot's kinematics. For symmetric objects with large deformation, the paper instead aligns the centre of mass and principal axes §3.2 fig. 2.

Point clouds are captured with two depth cameras on opposite sides of the object. The grasp is run with the arm rotated 90° about the vertical, so the fingers are vertical and the object is not occluded §4 fig. 3. The gripper closes at 0.1 m/s until it applies 0.5 N §4. The procedure is five steps: capture the ungrasped object, grasp, lift and capture the grasped object, return the arm to home, and release the object at its origin §4.

Objects are EGAD shapes scaled to a 55 mm maximum width, plus three custom symmetric shapes built from mirrored random splines and extruded §4.2 fig. 4. They are printed in white at Shore 40A, 60A and 85A and placed on a black background §4.2 §4.

Hyperparameters

Results

Author-reported. Setup: 15 objects, 3 hardnesses, 4 grippers (three Fin-Ray with 4, 6 and 8 ribs, one rigid PLA finger), 5 repeats each, 900 grasps, printed objects on a black background, Franka arm, two Zivid cameras §4 §5. The paper compares grippers against each other, not against prior benchmark methods, so there is no baseline column.

Because every grasp succeeded, all scores lie in [0.5, 1] §5. The text gives only the extremes; the per-cell means and standard deviations are in fig. 5, and the score distributions for selected objects are in fig. 7.

Paper vs code

The official repository is benjamin-greenland/SoGraB (MIT, copyright 2025 Ben Greenland; the first author). It is pinned to commit 2188842b9b4c2bbb12fd7f470ea06fe8dd721742. The DCD code is a vendored copy of the Density-Aware Chamfer Distance project under point_clouds/Density_aware_Chamfer_Distance/; that folder has no LICENSE file in this clone, so no licence is claimed for it here.

• α is not in the paper. The paper gives no value §3.1. The code sets ALPHA_DISTANCE = 0.005 dcd_distance_ply.py, but its own comment says the default is 1000 dcd_distance_ply.py, and the DCD function defaults to 1000 model_utils.py. Which value produced the paper's numbers is not stated.
• Exponent uses squared distance. Equation 2 writes the Euclidean norm ||x−y||₂. The Chamfer kernel computes squared distances chamfer3D.cu, and the DCD function feeds those into the exponential model_utils.py. A sensitivity that is correct for the paper's unsquared form will behave differently here.
• Extra density factor not in the equation. The code multiplies each exponential term by a ratio of point counts, n_other / n_this, and uses it again in the per-point weight model_utils.py model_utils.py model_utils.py. Equation 2 has no such factor. The default call does not switch it off point_cloud_utils.py.
• Exponential is exact, not approximated. The text describes a first-order Taylor bound §3.1. The code uses the exponential directly model_utils.py model_utils.py. Equation 2 matches the code on this point.
• ICP type is inconsistent in the code. A log message says point-to-plane point_cloud_utils.py, but the call uses point-to-point point_cloud_utils.py. The ICP distance threshold is 10000 dcd_distance_ply.py.
• No centre or principal-axis alignment. The paper describes this for symmetric objects with large deformation §3.2. A search of the repository for principal axes, centroid or centre of mass found nothing in the analysis code. The only centring is in the vendored EGAD mesh code mesh.py.
• Initial transforms are fixed matrices. The paper says the start point comes from kinematics §3.2. The code selects one of three hard-coded 4×4 matrices by the gripper angle in the file name: 45, 90 or 125 dcd_distance_ply.py dcd_distance_ply.py. The 45 and 125 cases are not in the paper.
• Partial-success branch. The text says the distance is halved to separate partial from full success §3.1. Equation 1 instead keeps (1−d) and divides the time ratio by 2. The code follows the equation dcd_distance_ply.py. Implement the equation.
• Outcome labels. The code uses the spreadsheet strings yes, no and dropped dcd_distance_ply.py dcd_distance_ply.py dcd_distance_ply.py. Unsuccessful grasps get score 0 and no DCD dcd_distance_ply.py, which matches the paper.
• Analysis uses only the 90° rows. The analysis script skips every row whose angle is not 90 analyse_scores.py. Its comment says the values are independent of gripper orientation. The interval is a normal approximation around the mean using the standard error analyse_scores.py. The paper does not state its interval method.
• Filter colour. The black-gripper filter uses a tolerance of 20% of √(3·255²) in 0–255 colour units dcd_distance_ply.py cloud_filter.py. The paper does not give a tolerance §4.

Pitfalls

• Alpha has no paper value. Its scale depends on whether the distance is squared or not (see above). Calibrate on your own data before comparing scores with the paper.
• n_ŷ is under-specified. The paper says the count of neighbour references damps repeated use §3.1, but does not say whether the count is per cloud, per direction, or in a particular order. The code counts references per cloud and direction, with a small epsilon in the inverse model_utils.py.
• Equation 2's min is an argmin. The notation writes ŷ as a minimum over distances, but then uses it as a point. Read it as the nearest point.
• Time inputs are unchecked. t_cycle is the "Max Time (seconds)" column and t_dropped the "Drop Time (seconds)" column dcd_distance_ply.py. The score divides by the max time with no zero check dcd_distance_ply.py. How the paper measured these times is not stated.
• GPU required. Point colours are moved to CUDA tensors point_cloud_utils.py dcd_distance_ply.py. The paper does not mention this.
• Colour assumptions. The filter removes dark points. This works with white objects and a black gripper on a black background §4. Other colour schemes need the tolerance retuned.
• Alignment can fail for large deformation. The paper says ICP approximates the transform reliably for moderate deformation, given a good initial estimate §3.2. The symmetric-object fallback is not in the code.
• Thin features give noisy scores. Objects with thin external features had large standard deviations and unexpected rankings §5.3. The paper says these need more repeats. Five repeats per cell is the design used §5.
• Ranking is not tested statistically. The rankings are read from overlapping distributions and means §5.3 fig. 7. The paper reports no significance test.
• Results depend on the gripper set. Several objects converge at the hardest setting, so the method cannot separate grippers there §5.3. The authors say softer or fewer-rib designs would be needed §5.3.
• Scores are not comparable across setups. The paper uses one camera pair, one arm and one printer §4. The authors say other hardware can be substituted, but do not report such a test §4.

Lineage

• Builds on: the Density-Aware Chamfer Distance from Wu et al. (2021), used as the deformation metric §3.1; ICP for alignment §3.2; the EGAD object set §4.2.
• Compared against: Chamfer distance and Earth Mover's distance, which the paper rejects for this use §3.1; grasp success rate, retention force, and the NIST/IEEE grasp-quality measures §1 §2.
• Superseded by: not stated in this card.