wikillm

Concept · 1 paper · updated 2026-11-15

Set abstraction

The set abstraction layer of point-based 3D detectors: pick key points, group the points around each one, and pool one feature per key point. The variable part is how key points are chosen. Plain FPS, feature-distance FPS and foreground-weighted FPS (SASA) are compared. Gains are author-reported on KITTI and nuScenes.

point-cloud3d-object-detectionsampling

Markdown5 min read

AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

Overview

A set abstraction (SA) layer is the down-sampling stage of a PointNet++-style point network. It selects a smaller set of key points from its input, groups the points around each key point, runs a shared MLP and max-pools, so every key point gets one feature 2201.01976 §3. The layers are stacked. In 3DSSD, level 1 keeps 4096 key points and levels 2 and 3 keep 512 + 512 and 256 + 256, where each sum is an S-FPS half and an FPS half 2201.01976 tab. 5 2201.01976 §3.3.1. In PointRCNN the key-point counts are 4096, 1024, 256 and 64 2201.01976 tab. 6.

The key-point rule decides which points survive each level. Plain farthest-point sampling (FPS) spreads the picks over the scene, and in outdoor scans most of those picks land on background such as ground 2201.01976 §1. Earlier detectors recovered the dropped points with feature propagation (FP) layers, which the paper says add memory and compute cost 2201.01976 §1. The variants below change the sampling rule. The MLP and the pooling stay the same.

Only one card is linked to this slug so far, so the variants below are the ones that paper describes.

Variants

Plain FPS sampling

The sampler picks points by spread alone, with no per-point scores or labels. SASA's first SA level uses it, because that level's input is the raw point cloud and has no foreground scores yet 2201.01976 §3.3.1.

Feature-distance FPS (F-FPS, 3DSSD)

3DSSD samples its levels 2 and 3 with F-FPS, which measures distance between feature vectors rather than between coordinates 2201.01976 §3.3.1. The paper says this costs quadratically in the number of points, because it needs pairwise feature distances 2201.01976 §4.3.3.

Score-weighted FPS (S-FPS, SASA)

SASA adds a segmentation head to each SA layer and uses its output in the sampler. The steps, as the paper defines them:

  • Score. A two-layer MLP reads each point's features at that layer and outputs a foreground score p in [0, 1] through a sigmoid [arxiv:2201.01976 §3.1] eq. 1.
  • Supervision. A point is foreground if it lies inside any ground-truth 3D box, and background otherwise. Each SA layer has a cross-entropy loss, and the total is a weighted sum over layers 2201.01976 §3.1 eq. 2. This variant therefore needs labelled boxes.
  • Sampling. The first key point is the highest-scoring point. Each later key point maximises p raised to γ, times its distance to the nearest key point already chosen 2201.01976 §3.2 alg. 1 eq. 3. γ = 0 gives plain FPS for the later picks. A very large γ approaches a top-K selection on scores 2201.01976 §3.2.
  • Weights. The paper's ablation favours γ = 1 2201.01976 §4.4 tab. 3. Loss weights for 3DSSD are 0.01 at level 2 and 0.1 at level 3, and level 1 is not supervised 2201.01976 §3.3.1. For PointRCNN they are 0.001, 0.01 and 0.1 at levels 2 to 4 2201.01976 §3.3.2.
  • Placement. In 3DSSD, S-FPS replaces F-FPS for half of the key points at levels 2 and 3, and plain FPS samples the other half. The other sampling settings stay as they were 2201.01976 §3.3.1 fig. 3. In PointRCNN it is used at levels 2 to 4 2201.01976 §3.3.2.
  • Reported gains (author-reported). On the KITTI validation split, moderate car AP for 3DSSD with fusion sampling rises from 83.46 (F-FPS) to 85.76 (S-FPS, γ = 1), and point recall from 96.65 to 97.65 [arxiv:2201.01976 tab. 3]. On the KITTI test set, 3DSSD moderate car AP rises from 79.57 to 82.16 2201.01976 tab. 1. On nuScenes validation, NDS rises from 56.4 to 61.0 2201.01976 tab. 2.

When to use which

SituationUseWhy
Raw points at the first level, or no per-point labelsPlain FPSThe first level has no scores 2201.01976 §3.3.1, and the score head is trained from ground-truth boxes 2201.01976 §3.1.
Labelled 3D boxes, and FPS would keep mostly backgroundS-FPSThis is the case the paper targets 2201.01976 §1. Its gains are reported on KITTI and nuScenes 2201.01976 tab. 1 tab. 2 tab. 3.
Keeping the 3DSSD layout with feature-distance samplingF-FPSIt is the baseline that S-FPS replaces at levels 2 and 3 2201.01976 §3.3.1 tab. 3. The paper's cost argument against it is the quadratic growth 2201.01976 §4.3.3.
Latency is a constraintMeasure on your own hardwareThe paper reports 36 ms for 3DSSD with SASA against 38 ms without, on one V100 with OpenPCDet 2201.01976 tab. 1 §4.3.3. These numbers come from the authors' setup.
Trying γ above 1Check numerical precisionThe paper reports γ = 100 2201.01976 tab. 3. At that γ, p^γ underflows in float32 for most scores, which the card's Pitfalls section explains.

Implementation sketch

The layer core is the same whichever sampler is used. Only the call to the sampler changes.

Pseudocode
# Set abstraction layer  [arxiv:2201.01976 §3]
# Input:  coords (N, 3), feats (N, C), num_keys M
# Output: key_xyz (M, 3), key_feats (M, C_out)

def set_abstraction(coords, feats, num_keys, sampler):
    # Choose key points. The sampler may read per-point scores  [§3.2]
    key_idx = sampler(coords, feats, num_keys)     # (M,)
    key_xyz = coords[key_idx]                      # (M, 3)
    # Group the points around each key point  [§3]
    groups = group(coords, feats, key_xyz)         # (M, K, 3 + C)
    # Shared MLP, then max-pool over each group  [§3]
    key_feats = max_pool(mlp(groups), axis=1)      # (M, C_out)
    return key_xyz, key_feats

For S-FPS, the sampler first computes a score per point, then scores the candidates 2201.01976 eq. 1 eq. 3:

Pseudocode
# Score-weighted sampler  [arxiv:2201.01976 alg. 1] [eq. 3]
# Input:  scores (N,) in (0, 1); gamma; num_keys M
# Output: key_idx (M,)

def s_fps_sampler(coords, scores, gamma, num_keys):
    # First key point: highest foreground score  [alg. 1]
    first = argmax(scores)
    key_idx = [first]
    chosen = zeros(N, dtype=bool)
    chosen[first] = True
    min_dist = norm(coords - coords[first], axis=1)
    for _ in range(num_keys - 1):
        # Weighted distance, chosen points excluded  [eq. 3]
        weighted = scores ** gamma * min_dist
        weighted[chosen] = -inf
        nxt = argmax(weighted)
        key_idx.append(nxt)
        chosen[nxt] = True
        min_dist = minimum(
            min_dist, norm(coords - coords[nxt], axis=1))
    return key_idx

The released code and the paper differ on the distance used here. See Contradictions.

Contradictions

  • Distance in S-FPS. Algorithm 1 and eq. 3 use the Euclidean distance 2201.01976 alg. 1 eq. 3. The released CUDA kernel squares it, so at γ = 1 the code ranks by p times d², not p times d. The card's Paper vs code section records this.
  • Labels around boxes. The paper gives two labels, foreground inside any ground-truth box and background otherwise 2201.01976 §3.1. The released code adds an ignored band around each box, which its loss leaves out. The card's Paper vs code section records this.

Papers

  • SASA: adds a foreground head and score-weighted FPS to the SA layers of 3DSSD and PointRCNN, with author-reported gains on KITTI and nuScenes.

Source and license

Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.