--- kind: concept slug: robotic-grasp-generation title: Robotic grasp generation summary: >- Predicting where a robot gripper should close on an object, and how it should approach and open. Nine papers: sample-and-classify, per-point contact, latent diffusion, Gaussian-scene grasping, dexterous RL priors and geometric grasp points. Author-reported, on different setups, so the numbers are not comparable. tags: [grasping, point-cloud, robot-manipulation, 6-dof, collision-avoidance, grasp-detection, generative-model, dexterous-manipulation] papers: ["1603.01564", "1706.09911", "2103.14127", "2312.11243", "2411.14786", "2412.02140", "2412.15587", "2411.08169", "2411.06244"] related: [sim-to-real-transfer, robot-simulation, local-reference-frame, convolutional-neural-network, diffusion-models, latent-diffusion, variational-autoencoder, gaussian-splatting, proximal-policy-optimization, reward-design, haptic-feedback-rendering] status: draft updated_at: 2026-11-01 --- ## Overview A grasp is a gripper pose plus an opening width. The pose has six degrees of freedom, and most of those poses collide with something in a cluttered scene. The problem is to find poses that close on the object, stay clear of its neighbours, and can be reached. The wiki has nine papers on it. Most of them work from a depth point cloud. Grasp Pose Detection and Contact-GraspNet both avoid fitting a model to a known object [arxiv:1706.09911 §1] [arxiv:2103.14127 §1]. The others change the input or the output. GraspLDM samples grasps from a learned latent space [arxiv:2312.11243 §3]. SparseGrasp works from a few RGB views through a Gaussian scene [arxiv:2412.02140 §3]. A wearable prototype uses fixed geometric rules and no learning [arxiv:2411.08169 §II.C]. The papers differ in where candidates come from. Grasp Pose Detection draws hand poses on the visible surface, then scores each pose with a classifier [arxiv:1706.09911 §4] [§5]. Its 2016 conference precursor, which shares the sample-then-classify design, is covered under the same variant below [arxiv:1603.01564 §3.2]. Contact-GraspNet asks, for every point in the scene, whether a grasp touching it would succeed, and predicts the grasp along with the answer [arxiv:2103.14127 §3.1]. In both, the candidate is anchored at a point the depth sensor already sees, which keeps the search away from unobserved space. ## Variants Seven method families are covered, plus one survey. The sources also name other families, such as template methods, learned three-degree-of-freedom detectors and kernel-density methods [arxiv:1706.09911 §1.2], but no card exists for them yet. ### Sample, then classify each candidate Grasp Pose Detection samples a point from the cloud, builds a local frame from the surface normals around it, and places a two-finger hand over a grid of offsets and rotations. It slides each hand along its approach direction until it touches the cloud [arxiv:1706.09911 §4]. A candidate is kept only when the hand body is collision-free and the closing region holds at least one point [arxiv:1706.09911 §4]. Each survivor becomes a 15-channel 60 x 60 image of the points in the closing region, including estimated occluded space, and a four-layer CNN scores it [arxiv:1706.09911 §5.1] [§5]. The authors train it on candidates labelled by a frictionless antipodal test against meshes of 55 BigBird objects, with 20 scanned views each [arxiv:1706.09911 §6]. The executed grasp is chosen by a cost over height, approach direction and arm travel [arxiv:1706.09911 §8.5] [eq. 1]. Card: [Grasp Pose Detection in Point Clouds](/papers/1706.09911). The 2016 precursor, [High precision grasp pose detection in dense clutter](/papers/1603.01564), uses the same two steps with a shorter description. It classifies each candidate with a four-layer CNN over 15 channels from three projections of the space between the fingers [arxiv:1603.01564 §3.2] [§3.3]. Its labels are frictionless antipodal grasps, and it reports that occlusion channels and CAD pretraining raise recall at 99% precision [arxiv:1603.01564 §4.2] [§4.3]. Its real-robot clutter result is 93% (20 failures of 288) with an active wrist sensor, against 73% for the authors' prior work [arxiv:1603.01564 tab. 2] [§6]. ### Per-point contact prediction on a scene point cloud A network reads a point cloud of the scene and gives each point three outputs: a contact score, an approach direction and a width [arxiv:2103.14127 §3.3]. The approach and baseline vectors are made orthonormal with two Gram-Schmidt steps [arxiv:2103.14127 eq. 6]. A grasp is kept when its score passes a threshold, and farthest-point sampling spreads the kept contacts over the surface [arxiv:2103.14127 §4.1]. Training uses simulated grasps from ACRONYM, placed on ShapeNet meshes and rendered as point clouds [arxiv:2103.14127 §3.2]. Card: [Contact-GraspNet](/papers/2103.14127). ### Generative sampling in a learned latent space GraspLDM trains a conditional VAE. A grasp encoder, conditioned on a point-cloud encoding, maps each grasp to a short latent, and a decoder maps it back to a pose [arxiv:2312.11243 §3.1]. A DDPM is then trained on the latents of the frozen VAE, which the authors describe as a prior that repairs the gap between the VAE prior and its posterior [arxiv:2312.11243 §3.2]. Sampling starts from noise in the latent space, denoises it, and decodes the result. The paper also reports DDIM with 100 steps as a drop-in sampler with no retraining [arxiv:2312.11243 §4.3]. For a new downstream task, only a small latent denoiser is trained, with a task label as an extra input [arxiv:2312.11243 §4.2]. Card: [GraspLDM](/papers/2312.11243). ### Language-queried grasps on a Gaussian scene SparseGrasp fits a 3D Gaussian Splatting scene to three RGB views, initialised from dense DUSt3R points [arxiv:2412.02140 §3.1] [§4]. Each Gaussian carries a semantic feature distilled from MaskCLIP region features after PCA compression to 16 dimensions [arxiv:2412.02140 §3.2]. When an object moves, the Gaussians that match the moved pixels are re-posed by render-and-compare, which the paper reports at about 200 ms [arxiv:2412.02140 §3.3] [tab. 2]. A retrained GraspNet reads each Gaussian's centre, scale and rotation to propose grasps [arxiv:2412.02140 §3.4]. Card: [SparseGrasp](/papers/2412.02140). ### Geometric grasp points from a wearable depth camera This variant has no learned model. A wrist-mounted depth camera and an IMU classify the scene as tabletop, doorknob or pot handle. The classification uses RANSAC plane detection, a plane-hull extrusion and clustering [arxiv:2411.08169 §II.C]. For each object, the grasp points are the two points nearest the minor principal axis on opposite sides [arxiv:2411.08169 §II.C]. The pot-handle mode uses a height test, and the authors list the RANSAC threshold and the cluster size as values the implementer must choose [arxiv:2411.08169 §II.C]. Card: [Point Cloud Context Analysis for Rehabilitation Grasping Assistance](/papers/2411.08169). ### Dexterous grasp as the start state for reinforcement learning For a 16-DoF Allegro hand, this method first segments the functional part of the object, runs AnyGrasp on it, and maps each two-finger candidate to dexterous hand configurations [arxiv:2412.15587 §3.1]. Collision-free candidates are kept, and the one closest to the camera becomes the start state. A PPO policy then refines the arm and hand from there [arxiv:2412.15587 §3.1] [§3.4]. The paper compares this with a fixed, human-designed start, and reports higher success in simulation on four articulated-object tasks [arxiv:2412.15587 tab. 1]. Card: [Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge](/papers/2412.15587). ### Human-hand grasp synthesis (not a gripper) FastGrasp produces a MANO human hand mesh for an object point cloud. It uses a latent diffusion model over a hand-pose autoencoder, then an MLP that adds an offset trained with contact and penetration losses [arxiv:2411.14786 §3.1] [§3.3]. It answers a different question from the gripper variants. Its output is a hand, so it fits hand-object animation or avatars, and it does not select a parallel-jaw pose. Card: [FastGrasp](/papers/2411.14786). ### Survey A survey of grasping in virtual reality, set against robotic grasping, groups methods into physics-based, visual and data-driven families and reports no experiments of its own [arxiv:2411.06244 §2.2] [§4]. Card: [Grasping Object: Challenges and Innovations in Robotics and Virtual Reality](/papers/2411.06244). ## When to use which The rows below rest on the constraints each source reports. No paper compares all of them directly, so the table says what each one supports. | Situation | Use | Why | |---|---|---| | Latency under one second per scene | Per-point contact prediction | Reported 0.28 s for a full scene, hardware not stated [arxiv:2103.14127 §4.1]. Grasp Pose Detection reports one to eight seconds per grasp [arxiv:1706.09911 §8.4] on an Intel i7-4770K with a GTX 970 [arxiv:1706.09911 tab. 2] | | Sample-then-classify must run faster | Use the three-channel encoding | Classifying 1k candidates at 39k points takes 4.3 s with 15 channels and 0.3 s with 3 channels [arxiv:1706.09911 tab. 2]. The paper's robot clutter results used the 3-channel form [arxiv:1706.09911 §8.6] | | One depth camera, one view per scene | Per-point contact prediction | Evaluated with one depth camera on a Franka [arxiv:2103.14127 §4.3] | | Sensor moves or several views are available | Sample-then-classify | Its accumulated-cloud run reports 93% (268 of 288), against 84% with two body-mounted sensors [arxiv:1706.09911 tab. 3] | | Reimplementing the 2016 classifier from its paper | Check the training settings before copying them | The paper gives a learning rate of 0.00025 with no other optimiser detail, while the released Caffe solver uses 0.01 with inverse decay [arxiv:1603.01564 §3.2] [code: models/caffe/15channels/lenet_solver_15_channels.prototxt#L10-L15] | | Training data must be real scans | Sample-then-classify | Labels come from registered meshes and real depth scans of 55 BigBird objects [arxiv:1706.09911 §6] | | Training data is simulated only | Per-point contact prediction | Trained on simulated ACRONYM grasps, about 40 h on one V100 [arxiv:2103.14127 §3.5] | | Simulated pre-training is an option | Sample-then-classify, with its caveat | 207K simulated exemplars alone reach 83% on the real test set, against about 90% for real data; pre-training cut fine-tuning from 20000 to 4000 iterations [arxiv:1706.09911 §7.3] | | Many distinct grasps per object, and a few hundred ms to seconds per batch | GraspLDM with DDIM, 100 steps | 100 grasps in 0.75 s on an RTX 3080 Ti, median success 0.756, against 7.39 s and 0.792 for 1000 DDPM steps [arxiv:2312.11243 tab. 1]. These timings are for the 63-category full-point-cloud model | | Speed matters more than success | GraspLDM, VAE only | Same table: 100 grasps in 0.02 s, median success 0.660 [arxiv:2312.11243 tab. 1] | | New task, and the grasp model must stay frozen | GraspLDM | Only the latent denoiser is trained for the new task, under two hours on one V100, with a mean label precision of 0.703 over 200 unseen objects [arxiv:2312.11243 §4.2] | | Only RGB cameras, a few views, and objects move during the task | SparseGrasp | Three views give 55 of 70 successful grasps on a real arm, against 38 of 70 for F3RM with 17 views [arxiv:2412.02140 tab. 1]. The update needs a 200 ms budget: 22 of 25 scene updates succeed, against 1 of 25 at 50 ms [arxiv:2412.02140 tab. 2]. Adding or removing objects needs a rebuild [arxiv:2412.02140 §4.3] | | Language-named target, and the scene can be re-initialised | SparseGrasp, with calibrated poses | The method feeds calibrated poses to DUSt3R [arxiv:2412.02140 §4]. The paper does not spell out how a text query is matched to the Gaussian features, so that step needs its own design [arxiv:2412.02140 §3.4] [fig. 2] | | Object with curved parts where one contact admits several approach directions | Check the output carefully | The paper says per-point approach directions are ambiguous at curved surfaces such as mug rims and handles [arxiv:2103.14127 §2]. The method picks one approach per contact [arxiv:2103.14127 §3.1] | | Need to pick a named object in clutter | Sample-then-classify with a separate detector | Object-specific use needs a separate detector [arxiv:1706.09911 §9]. In nine cluttered scenes, 85.4% of grasp-derived proposals were classified correctly [arxiv:1706.09911 tab. 4] | | Dexterous hand, RL, articulated objects | Dexterous grasp prior | Simulated success above DexArt on four tasks, for example 94.17% against 79.17% on bucket [arxiv:2412.15587 tab. 1]. Real UR5 tests use about 10 trials per task [arxiv:2412.15587 §4.2] | | Wearable device, no training data, feasibility only | Geometric grasp points | Tabletop and doorknob modes correct in 100% of crowded-kitchen and computer-desk scenes, and pot-handle mode correct in 85% [arxiv:2411.08169 §III.A]. Grasp points within 5 to 6 mm on one object [arxiv:2411.08169 §III.B]. No baseline and few scenes | | Hand-object animation or avatars, not a robot | FastGrasp | Lowest penetration volume in its out-of-domain table, 5.23 against 7.37 for the next method [arxiv:2411.14786 tab. 2]. Reported 0.14 s on an NVIDIA A40 [arxiv:2411.14786 §4.2] | | Product that must ship commercially | Sample-then-classify, or GraspLDM, after checking the code licence | Grasp Pose Detection code is BSD-2-Clause and GraspLDM code is Apache-2.0, per their cards. The Contact-GraspNet code is non-commercial, so do not use it without a licence from the authors. FastGrasp and the RL prior have no licence file or no code | | Need results beyond one robot and one camera | Gather your own data | Contact-GraspNet's real-world test covers nine scenes, 51 objects, one robot and one depth camera [arxiv:2103.14127 §4.3]. The sample-then-classify robot study covers one Baxter and 27 household objects [arxiv:1706.09911 §8.1] | | Background reading before choosing | Survey | Lists the challenges and families for VR and robotic grasping, with no experiments [arxiv:2411.06244 §2.1] [§3] | Two caveats on the figures. The success rates are author-reported and come from different robots, grippers, object sets, simulators and protocols. The Contact-GraspNet numbers are against 6-DoF GraspNet and CollisionNet baselines [arxiv:2103.14127 tab. 1]. The sample-then-classify numbers come from its own ablations [arxiv:1706.09911 tab. 3]. The paper's simulation ablations report their values only in plots, so this page does not quote them [arxiv:2103.14127 fig. 5]. ## Implementation sketch Language-agnostic pseudocode of the per-point variant's common core. The other variants have different steps. Their cards carry their own sketches, for example [GraspLDM](/papers/2312.11243) for latent sampling and [SparseGrasp](/papers/2412.02140) for the update step. ```pseudocode # Build one 6-DoF grasp from a contact point [arxiv:2103.14127 §3.1] # contact (3,): point on the observed cloud # base, approach (3,): unit vectors from the network # width: metres; d: gripper base distance (paper gives no value) def build_grasp(contact, base, approach, width, d): # Translation: half the width along the baseline, plus d along # the approach [arxiv:2103.14127 eq. 1] translation = contact + (width / 2) * base + d * approach # Rotation columns: base, approach x base, approach [eq. 2] rotation = column_stack(base, cross(approach, base), approach) return pose(rotation, translation) # Select grasps for execution [arxiv:2103.14127 §4.1] def select_grasps(points, scores, threshold=0.23): # Keep contacts whose score passes the threshold kept = points[scores > threshold] # Spread the choice over the surface with farthest point sampling chosen = farthest_point_sample(kept, n=150) # Execute the most confident grasp that is reachable and collision-free return sort_by_score(chosen) ``` The sign of the approach term differs between the paper and its code. See the Contradictions section. ## Contradictions No two papers report the same quantity, so none of their numbers conflict directly. The success rates and timings above are not comparable: the robots, grippers, object sets and trial protocols differ [arxiv:1706.09911 §8.1] [arxiv:1706.09911 §8.6] [arxiv:2103.14127 §4.2] [arxiv:2103.14127 §4.3] [arxiv:2412.02140 §4] [arxiv:2411.14786 §4.2]. Within each source, paper and code disagree on points that matter for a reimplementation: - Contact-GraspNet, the sign of the approach offset: the paper writes plus, and the code subtracts [arxiv:2103.14127 eq. 1]. - Contact-GraspNet, the pose-loss normalisation: the paper divides by the number of positive contacts, and the code does not [arxiv:2103.14127 eq. 8] [code: contact_graspnet/contact_graspnet.py#L290-L298]. - Sample-then-classify, the grasp labels in the 2016 precursor: the paper labels frictionless antipodal grasps as positive, and the released configuration uses a 20-degree friction cone [arxiv:1603.01564 §3.4] [code: cfg/generate_data.cfg#L54]. The two give different positive sets. - Sample-then-classify, the normal of the local frame: the paper's text takes the smallest-eigenvalue axis as the normal, and the code takes the largest [arxiv:1706.09911 §4] [code: src/gpd/candidate/local_frame.cpp#L22-L29]. - Sample-then-classify, the score: the paper says softmax probability, and the code outputs a logit difference [arxiv:1706.09911 §5] [code: src/gpd/net/eigen_classifier.cpp#L74]. Within a single paper, text and table disagree on some headline numbers: - Geometric grasp points: the abstract says grasp points fall within 1 cm on average, while the body reports 5 to 6 mm for one object with a caliper ground truth [arxiv:2411.08169 §III.B]. - GraspLDM: the text puts the partial-point-cloud gain over its VAE baseline at about 4.75%, and the table gives a difference of 3.75 points [arxiv:2312.11243 §4.5] [tab. 2]. - RL grasp prior: the text gives 92%, 87% and 87% for bucket, laptop and toilet, and the table gives 94.17, 90 and 87.88 for the same method [arxiv:2412.15587 §4.1.3] [tab. 1]. The paper does not say which checkpoint each figure comes from. - SparseGrasp: the paper says it is strong on novel views, but its table has the 17-view F3RM baseline higher on novel views, 0.75 against 0.71 [arxiv:2412.02140 §4.1] [tab. 3]. It names FastSAM for masks in one section and SAM ViT-H in another [arxiv:2412.02140 §3.1] [§4]. ## Papers - [High precision grasp pose detection in dense clutter](/papers/1603.01564): the 2016 precursor of grasp pose detection. It samples 6-DoF candidates from a point cloud and scores each with a 15-channel CNN. Author-reported 93% active-sensor success on a Baxter. - [Grasp Pose Detection in Point Clouds](/papers/1706.09911): samples two-finger 6-DoF hand candidates from a depth point cloud, encodes each as a 15-channel image, and scores it with a CNN. Author-reported 93% in dense clutter on a Baxter. - [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](/papers/2103.14127): a per-point contact network that predicts a collision-aware 6-DoF grasp from one depth point cloud, trained on simulated ACRONYM grasps. - [GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Models](/papers/2312.11243): a conditional VAE with a DDPM prior in its latent space, so a new task only trains a small denoiser. Author-reported 80% and 78.75% real-robot success on two arms. - [FastGrasp: Efficient Grasp Synthesis with Diffusion](/papers/2411.14786): a latent diffusion model and an adaptation MLP that produce a human hand mesh for an object point cloud in one pass. Human hand, not a gripper. - [SparseGrasp: Robotic Grasping via 3D Semantic Gaussian Splatting from Sparse Multi-View RGB Images](/papers/2412.02140): a 3DGS scene from three RGB views with distilled semantic features, updated by render-and-compare after objects move. Author-reported 55 of 70 grasps. - [Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge](/papers/2412.15587): an AnyGrasp-based dexterous grasp as the start state for PPO on an Allegro hand. Author-reported gains over DexArt in simulation. - [Point Cloud Context Analysis for Rehabilitation Grasping Assistance](/papers/2411.08169): a wrist-mounted depth camera and IMU classify the scene and pick two opposing grasp points with geometric rules. No learning and no baseline. - [Grasping Object: Challenges and Innovations in Robotics and Virtual Reality](/papers/2411.06244): a survey of grasping in VR and robotics. No method and no experiments.