T-DOM: A Taxonomy for Robotic Manipulation of Deformable Objects

In plain words

Robots that handle cloth, rope, bags or foam have to deal with objects that change shape as they are moved. Most existing ways of classifying robot grasps and motions assume the object stays rigid, so they miss the thing that matters most here: how the object is bending, twisting or stretching at each step.

This paper proposes a labelling scheme, called T-DOM. Every step of a task gets tags for three things: what kind of deformation the object is under, how the robot arm is moving (inertia-dominated, or slow and driven by weight or stretch), and how the gripper or some other part touches the object. The authors recorded ten deformable-object tasks, labelled every step by hand, and compared the labels with two older schemes. The deformation tags separate steps that the older schemes group together, such as sliding under a folded object and sliding under a flat one.

TL;DR

• T-DOM is a tag set for robot manipulation of deformable objects, not an algorithm or a trained model. §4
• Its tags cover deformation type and level, motion energy regime, prehensile grasp, non-prehensile contact and contact sliding, with separate tags per arm. §4
• On ten recorded tasks, labelled by hand, the deformation tags split actions that the Bullock 2012 and Paulius 2020 tags merge. §5.3 fig. 5
• Results are cluster counts on the authors' own labels. There is no inter-rater check and no downstream robot experiment.

Problem

Existing grasp and manipulation taxonomies assume rigid objects, so they do not describe how a deformable object's shape constrains the next step §1 §2.1. Measuring a deformable object's state precisely is hard, so the authors use qualitative labels tied to the forces that cause the deformation instead of a quantitative metric §1. Some earlier taxonomies for deformable objects are narrower: one covers only textile grasps §2.1, and another classifies deformation only as temporary or permanent §5.3.

Method

Structure. The taxonomy has three branches: object deformation (D), robot motion (M), and interactions. Interactions are split into prehensile grasps (G), non-prehensile interactions (NP) and contact sliding (CS), which can attach to both prehensile and non-prehensile contacts §4 fig. 3.

Deformation (D). The deformation types are compression, tension, bending, torsion and shear. They are defined by the direction of the applied force §3.1. Compression and tension are described by the force direction relative to a fixed point on the object. Torsion comes from a torque about the long axis. Shear comes from two opposite forces along two parallel lines §3.1. The authors state that 1D and 2D objects are usually modelled as incompressible, so compression is mainly relevant to folded, stacked or volumetric objects §3.1.

Bending is split into two qualitative levels:

• Structured bending counts ordered bends: removable crossings or loops for 1D objects, and g-folds for 2D objects §3.1 fig. 2.
• Unstructured bending counts how much manipulation is needed to remove the bend: the linking number of knots for 1D objects, and the number of accessible keypoints (for example, the shoulders of a T-shirt or the corners of a napkin) for 2D objects §3.1.

Both can occur together, for example loops and knots on a rope §3.1.

Tags in use are compression C, tension TN, torsion TR, shear S, structured levels S L0 onward, unstructured levels US L0 onward, and combinations such as C+B, TN+B and TN+TR §4.1. Permanent or breaking deformations (tearing, cutting, plastic change) are left out §4.1 §6.1.

Robot motion (M). Motion follows the dynamic versus quasi-static split of Mason, 2001 §3.2. The tags are:

• MK, kinetic: inertia-dominated motion, such as swinging a cloth §4.2;
• ME, elastic: quasi-static, the task depends on elastic forces §4.2;
• MG, gravitational: quasi-static, elasticity can be neglected, such as carrying a cloth §4.2;
• MGE, gravitational-elastic: both energy terms matter §4.2.

Kinematic and static categories are dropped because the authors judge them insufficient for deformable objects §3.2.

Prehensile grasps (G). A grasp is prehensile when the contact forces hold the object without external support §3.3. The tags classify the geometry the grasp constrains, not the contact patches. GP constrains a point (pinch grasp). GL constrains a line §4.3. The set is limited to geometries that parallel-jaw grippers can make §4.3.

Non-prehensile interactions (NP). A non-prehensile interaction involves gravity or a reaction force from the environment §3.3. Agent contacts are rigid (N-P.AR) or soft (N-P.AS). Environment contacts are rigid (N-P.ER), soft (N-P.ES), or both (N-P.ERS). Tools held by a robot count as agent contacts §4.4.

Contact sliding (CS). CSA (active): the object stays still while the end-effector slides across it. CSP (passive): the end-effector stays still while the object slides, for example under gravity §4.5.

Bimanual tasks. Each arm gets its own copy of the tags, instead of separate bimanual categories §4.6. Unimanual tasks keep both tag slots and set the second arm to a none tag §5.2.

Labelling. The authors label the time instants at which a robot action changes any category. Each action segment then receives one tag set §5.2 tab. 2. The paper does not describe a labelling protocol beyond these rules.

Clustering. Each action gets an action-ID, which is its ordered tag set. A graph links two action-IDs when their tag sets are equal, and the clusters are then compared across taxonomies §5.3 fig. 5.

# Group actions whose full tag sets are equal  [§5.3] [fig. 5]
# Input:  actions: list of (action_id, code), code = tuple of tags
# Output: clusters: list of lists of action-IDs, each with more than one member

def cluster_actions(actions):
    # Equal tag sets form an equivalence relation, so the connected
    # components of the edge rule are exactly these groups.
    # Inferred: the text does not define "cluster" explicitly.  [§5.3]
    groups = {}
    for action_id, code in actions:
        groups.setdefault(code, []).append(action_id)
    clusters = [ids for ids in groups.values() if len(ids) > 1]
    return clusters

Category boundaries as the text defines them:

Hyperparameters

Not applicable. The taxonomy has no learned model, no training and no numerical settings. Category boundaries are in the Method table above.

Results

Author-reported, on the authors' own labels. Setup: 10 recorded tasks with the 60 action-IDs of tab. 2, labelled by hand §5.1 §5.2. The comparisons are qualitative cluster counts. No accuracy or success rate is reported.

The text uses these numbers to argue that deformation tags separate actions that look the same, such as slides under a folded object and a flat one §5.3. It also shows, in transition plots for tasks 1, 2 and 4, where the segments of the T-DOM labels and the baselines differ §5.2 fig. 4.

Paper vs code

No official code. The paper links only a dataset project page (sites.google.com/view/t-dom), which has no code link §5.1. A GitHub search for "T-DOM deformable" returned no repositories. A search for "deformable manipulation taxonomy" returned gawtamcr/vlm-taxonomy, which is a third-party project and not by the authors, so it is not used.

Pitfalls

• No protocol for labelling. Labels are assigned by hand at time instants where a robot action changes a category §5.2. The paper reports no second annotator and no agreement score, so the labels cannot be reproduced exactly.
• Cluster counts are not a benchmark. The comparison is cluster count and largest-cluster size on one set of actions. The paper does not give a score or a test for the difference §5.3. Treat the result as an illustration of the taxonomy's granularity.
• The same letter means different things. In tab. 2, the environment column uses S for soft contact, while the bending column uses S for structured bending. The text uses N-P.ES for soft environment contact §4.4.
• Motion and grasp tags differ between text and table. The text defines MK, ME, MG, MGE and GP, GL. tab. 2 uses K, G, GE and P, and never uses a bare E or the GL line grasp. The mapping is implied, not stated.
• Three-token contact-sliding entries. In tab. 2 the CS column holds three tokens, such as "N A A". The caption lists left arm, right arm and environment, but does not say which position is which.
• Unstructured levels are defined by a figure. The US levels beyond L0 "follow fig. 2" §4.1. The text gives no rule for them, so an implementer must read the figure.
• Boundary between none and L0. "No bending" (N) and "S L0" are separated only by the verbal description of a transition §4.1. No threshold is given.
• Dataset licence is not stated. The project page says the dataset has depth and RGB data, and the paper says it is open-sourced §5.1. The page states no licence, so do not redistribute it.
• Minor text errors. The edge-tracing task is called "T5-2" and "T5-3" in the discussion, but it is task 4 in tab. 2 §5.2. Table 1 marks each baseline's coverage with the authors' own judgement tab. 1.
• Figures are described, not reproduced. Figure content was taken from captions and text, since the page does not include images.
• Numbering. Figure and table numbers follow the order of the LaTeX source, since only the source was fetched and no compiled PDF was checked.

Lineage

• Builds on: the rigid-object hand taxonomy of Bullock et al., 2012 §2.1; the cloth grasp framework of Borras et al., 2020, whose geometry-based grasp classes are adapted §4.3; the manipulation taxonomy of Paulius et al., 2020 §2.1; and the dynamic/quasi-static split of Mason, 2001 §3.2.
• Compared against: Bullock et al., 2012 and Paulius et al., 2020, on the same actions §5.3.
• Superseded by: not known from this wiki.