Surfel mapping

Overview

A surfel is a small oriented disc with a position, a normal and a radius. A surfel map stores the scene as a set of these, and each one also carries timestamps and a stability score 2105.11320 §III-B. A new LiDAR scan is aligned to the map by ICP, and the scan is then merged into the map 2105.11320 §III. SuMa adds loop closure and pose-graph optimisation on top of this loop 2105.11320 §III-B.

The variants differ in what each map element carries and in whether a new frame is aligned to the map before it is merged. SuMa and SuMa++ align each LiDAR scan to the surfel map and keep a stability score per surfel. ConceptFusion takes camera poses as given, merges per-pixel features into its map, and keeps only an accumulated confidence per point 2302.07241 §3.1 2302.07241 eq. 2. Two cards list this concept. The LiDAR systems come from one paper; the point map comes from a second paper, whose card does not call its map surfels. Other surface-based maps are not covered here.

Variants

Geometric surfel mapping (SuMa)

The baseline keeps the surfel map and the align-then-merge loop, with no semantic labels 2105.11320 §III-B. Each surfel's stability is a log-odds value updated by a binary Bayes filter, so a surfel can be trusted more or less over time 2105.11320 §III-B.

The paper reports this baseline as a clear failure on moving-object scenes. On KITTI highway sequence 35 the relative translational error is 26.8%, author-reported 2105.11320 tab. I. On the odometry training sequences 00 to 10, the average is 0.83% 2105.11320 tab. II. The baseline has no card of its own; it is compared through the SuMa++ card.

Naive removal of movable surfels (SuMa_nomovable)

This baseline deletes every surface of a movable class from the map. The paper uses it to show that removal alone is not enough. Parked cars help scan registration, and wrong labels delete static surfels 2105.11320 §IV-B. On the odometry sequences the average relative translational error is 9.24%, and the paper says it diverges on the urban sequences 2105.11320 tab. II. On the highway sequence it does well, at 1.22% 2105.11320 tab. I. Card: SuMa++: Efficient LiDAR-based Semantic SLAM.

Semantic surfel mapping with dynamic filtering (SuMa++)

SuMa++ puts semantics into three steps of the loop 2105.11320 §III:

1. Label refinement. Each scan gets per-point labels from a segmenter on its spherical projection. The label image is then eroded at object boundaries and refilled from consistent neighbours 2105.11320 §III-C 2105.11320 §III-D 2105.11320 alg. 1.
2. Stability update. When a surfel's stored label disagrees with the new label and its class is movable, the stability update is penalised. Static classes such as vegetation are not penalised 2105.11320 §III-E 2105.11320 §IV-C.
3. Semantic ICP. Each residual gets a semantic compatibility weight on top of the robust weight, so moving objects count less in the pose estimate 2105.11320 §III-F 2105.11320 eq. 6.

On the odometry training sequences the average relative translational error is 0.70%, author-reported, against 0.83% for SuMa 2105.11320 tab. II. On highway sequence 35 it is 1.11%, against 26.8% for SuMa 2105.11320 tab. I. The paper reports these for the full system and does not isolate the semantic ICP weights 2105.11320 §IV. Card: SuMa++: Efficient LiDAR-based Semantic SLAM.

Point-map feature fusion without alignment (ConceptFusion)

This variant builds its map from posed RGB-D frames and fuses a per-pixel feature into it, where SuMa fuses a label. Each pixel's feature goes into the map point it projects to, weighted by a confidence that falls off with radial distance from the camera 2302.07241 §3.1 2302.07241 eq. 1. That confidence accumulates over observations 2302.07241 eq. 2. Points are merged when their positions and normals are close, using PointFusion's defaults of 5 cm and 20 degrees 2302.07241 app. A3. Outdoors the paper uses LeGO-LOAM odometry on lidar 2302.07241 app. A3. The paper describes no step that aligns each frame to the map, and no stability score per point 2302.07241 §3.1.

The headline result is open-vocabulary retrieval, not mapping error. On UnCoCo structured text queries it reports 0.446 3D mIoU, against 0.289 for supervised OpenSeg-3D 2302.07241 tab. 1. These numbers score retrieval of map regions from queries, so they cannot be set against the KITTI trajectory errors above 2105.11320 tab. II. Feature extraction is a separate cost: the paper reports 10 to 15 s per image on an RTX 3090 2302.07241 §3. Card: ConceptFusion: Open-set Multimodal 3D Mapping.

When to use which

The LiDAR results are KITTI only 2105.11320 §IV. For other sensors or environments they do not transfer by themselves. The ConceptFusion results come from tabletop scenes and are not on the same benchmark 2302.07241 §4.1.

Implementation sketch

The loop below is the common core. The marked lines are where the variants differ. The semantic steps follow the paper's Algorithm 1 and Equations 1, 6 and 8, with the code's details noted below.

# One scan of surfel mapping  [arxiv:2105.11320 §III]
# Input:  scan points, surfel map, previous pose
#         (semantic variant also takes per-point labels)
# Output: pose of the scan, updated surfel map

def process_scan(scan, surfel_map, pose_prev, labels=None):
    # Spherical projection to a vertex map and a normal map  [arxiv:2105.11320 §III-B]
    vertices, normals = project_and_normals(scan)
    # Semantic variant only: clean the labels  [arxiv:2105.11320 §III-D]
    if labels is not None:
        labels = refine_labels(labels, vertices)
    # Frame-to-model ICP; semantic variant adds label weights  [arxiv:2105.11320 §III-F]
    pose = icp_to_surfels(vertices, normals, surfel_map, pose_prev, labels)
    # Merge the scan and update each surfel's stability  [arxiv:2105.11320 §III-E]
    surfel_map.integrate(scan, labels, pose)
    return pose

The stability update is a log-odds step. Its penalty term applies only when the label disagrees and the class is movable 2105.11320 §III-E 2105.11320 eq. 1.

# Stability log-odds update for one surfel  [arxiv:2105.11320 eq. 1]
# odds(p) = log(p / (1 - p))

def update_stability(log_odds, compat, prior, penalty_applies):
    # compat = p_stable times two exponential terms in angle and distance  [eq. 1]
    log_odds += odds(compat) - odds(prior)
    # Movable surfel whose label conflicts with the scan  [arxiv:2105.11320 §III-E]
    if penalty_applies:
        log_odds -= penalty  # value not stated in the paper
    return log_odds

For ConceptFusion the same merge step runs with the pose given and a feature vector in place of the label. The alignment line is absent 2302.07241 §3.1.

Implementation notes, from comparing the paper with the released code (MIT, PRBonn/semantic_suma):

• The paper leaves the penalty value p_penalty unstated. The code subtracts a constant of 1.0 in log-odds, and only for movable classes whose label differs from the scan's label 2105.11320 §III-E.
• The paper's angle term uses the angle between normals. The code uses the sine of that angle, which is the length of the cross product of the two normals 2105.11320 §III-E.
• The code clamps the compatibility probability and caps log-odds, and it invalidates surfels below a threshold. The paper does not describe these steps 2105.11320 §III-E.
• Most numeric settings are not stated in the paper, including the stability priors, the Huber threshold, the ICP limits and the flood-fill tolerance 2105.11320 §III-D 2105.11320 §III-F. The released configuration gives defaults for the KITTI setting. Treat them as defaults, not as tuned values.

Contradictions

None known. The paper reports the baseline's effect differently by sequence set: naive removal of movable surfels does well on highway sequence 35 and diverges on the urban odometry sequences 2105.11320 tab. I 2105.11320 tab. II. This is one paper's result, not a disagreement between papers. The two families report different tasks, query mIoU and KITTI trajectory error, so their numbers are not comparable 2302.07241 tab. 1 2105.11320 tab. II.

Papers

• SuMa++: Efficient LiDAR-based Semantic SLAM: surfel LiDAR mapping with semantic labels that remove moving-object surfels from the map and weight the scan alignment; reports author-reported KITTI results and compares against SuMa and naive removal.
• ConceptFusion: Open-set Multimodal 3D Mapping: fuses CLIP-aligned per-pixel features into a posed point map that is queried by text, image, click and audio; reports author-reported UnCoCo tabletop query results against supervised baselines.