wikillm

Concept · 1 paper · updated 2026-11-15

Tactile sensing

Getting contact information at a robot fingertip from a soft, camera-based sensor. Pins under a skin move marker tips that a camera tracks. A review of one such sensor argues that sideways shear carries shape, and reports a shift from marker features with classical learners to CNNs on whole tactile images. Results are author-reported and come from separate studies.

tactile-sensingrobot-manipulationsoft-robotics

Markdown4 min read

AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

Overview

Tactile sensing gives a robot information about contact at its fingertips. The problem is that many optical tactile sensors measure only normal pressure, or use reflection or marker schemes whose output is hard to interpret 2105.14455 §II 2105.14455 §IV.A.

The variants share one design. A soft skin covers a transparent gel, and stiff pins run from the skin into the gel. Each pin tip carries a marker, and a camera inside the sensor images the markers through the gel 2105.14455 §III. When the skin is indented, the pins tilt and the markers shift sideways. The camera image of the marker pattern is therefore the signal, and everything downstream is a function of marker motion or of the whole image. The review's central claim is that the sensor responds mainly to shear strain, not to normal strain alone 2105.14455 §IV.A.

Variants

Marker features with classical learners (about 2015 to 2019)

Each frame is processed by tracking every pin marker. Two signals come out of the tracking: displacement from the resting position, and velocity between frames 2105.14455 §VI 2105.14455 §VI.B. Probabilistic classifiers work on displacements, and support vector machines on velocities for slip detection 2105.14455 tab. IV. Gaussian process and polynomial regressors serve sliding and grasp control 2105.14455 §VI.B. The review reports a slip-detection accuracy of more than 95% on a UR5 arm in 2018 2105.14455 tab. IV, and a single-axis position estimate of 0.7 mm in 2015 2105.14455 tab. IV. It judges these approaches to transfer poorly once the task is continuous and moving 2105.14455 §VI.B.

Card: TacTip review.

Whole-image CNN on tactile images (2019 onward)

A convolutional network reads the whole tactile image. The only preprocessing is cropping, thresholding or concatenation, and there is no marker-tracking step. The network predicts a quantity directly, such as pose, grasp success or object class 2105.14455 §VII 2105.14455 tab. IV. Reported results include object recognition above 95% and grasp success above 90% on the Tactile Model O hand, from 60 x 120 tactile images 2105.14455 tab. IV, and 3D surface localisation of 0.1 mm and 0.3 degrees from 128 x 128 images 2105.14455 tab. IV. The review does not state the network architecture, training set size, loss or optimiser for these tasks 2105.14455 §VII.

Card: TacTip review.

Hardware variants: printed sensors and hands

The sensor has been rebuilt many times with 3D printing. The review lists 40 mm tips with 127 to 532 pins across versions, and a 20 mm version with 276 pins that is not integrated with a camera 2105.14455 tab. II. The same design has been fitted to several hands, including the Tactile Model M2, GR2 and O, the Shadow Modular Grasper and the Pisa/IIT SoftHand 2105.14455 tab. III. Multi-material printing enabled rapid prototyping of these probes and hands 2105.14455 §VI.A.

Card: TacTip review.

When to use which

SituationUseWhy
Recover edges or shape from contact with a soft camera sensorShear-based marker sensingThe shear-sensing hypothesis, which the review reports, holds that zero crossings of the shear field mark edges without a gradient step 2105.14455 §IV.B. The review's two shear models are approximations and are not validated against measurements 2105.14455 §IV.B.
Detect slip from a short, mostly static eventMarker velocity with an SVMThe review reports more than 95% slip detection on one arm and sensor 2105.14455 tab. IV.
Continuous, moving contact with a tactile image streamWhole-image CNNThe review judges marker-feature methods to transfer poorly to continuous, moving tasks 2105.14455 §VI.B, and reports CNN results for servo control and grasping 2105.14455 §VII.
Need a sensor of a new shape or size quickly3D-printed TacTip familyPrinting enabled rapid prototyping of these probes and hands 2105.14455 §VI.A.
Need deep reinforcement learning with touchTrain mostly in simulationThe review says deep reinforcement learning on physical robots takes too long, and expects most learning in simulation 2105.14455 §VII.
Need to reproduce a learnerRead the cited worksThe review gives no architecture, dataset size, loss or optimiser for its learning tasks 2105.14455 §VII.

The review does not compare its methods on a common benchmark, and each row of its results table comes from a different robot, sensor and metric 2105.14455 tab. IV. Compare the rows only within their own setup.

Implementation sketch

The two learning paths share the same front end: the camera frames go in, and the question is whether the markers are tracked first or the whole image is used. Shapes are shown in the comments where they matter.

Pseudocode
# Front end: track each pin marker through the frames  [§VI] [§III]
# Input:  camera frames, resting marker positions
# Output: displacement and velocity per pin

def marker_features(frames, rest_markers):
    # 1. Track markers; shape (T, N_pins, 2)  [§VI]
    markers = track_markers(frames)
    # 2. Displacement from rest, the slow-adapting signal  [§VI.B]
    displacement = markers - rest_markers
    # 3. Frame-to-frame velocity, the fast-adapting signal  [§III]
    velocity = diff(markers, axis=0)
    return displacement, velocity
Pseudocode
# Whole-image path: no marker tracking  [§VII]
# Input:  tactile image, cropped or thresholded
# Output: quantity of interest, e.g. pose or grasp success

def predict_from_tactile_image(tactile_image, network):
    # 1. Feed the image directly  [§VII]
    # Architecture and sizes are not stated in the review.
    return network(tactile_image)

Where the variants differ is the front end and the learner. The review gives no training recipe for either path 2105.14455 §VI 2105.14455 §VII.

Contradictions

Within the review, the text and Table IV give different numbers for the Tactile Model O hand. The text reports 93% object classification and 95% lifting success over 26 objects 2105.14455 §VI.B. The table reports more than 95% for object recognition and more than 90% for grasp success 2105.14455 tab. IV. The review does not say which is correct. Cite the source you use.

Papers

  • TacTip review: a review of a soft, camera-based, biomimetic tactile sensor, covering shear sensing, the printed sensor family, and the move from marker features to CNNs on tactile images.

Source and license

Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.