Loop closure detection

Overview

Odometry drifts. Loop closure is the step that recognises a revisited place and turns that recognition into a constraint, so the trajectory can be pulled back into line 2105.11344 §1. OverlapNet and CT-ICP both treat it in three stages: propose earlier places that may match, verify the proposal geometrically, then add the result to the SLAM back end. Each accepted loop triggers a global optimisation of the pose graph 2109.12979 §4.

OverlapTransformer covers only the first stage. It returns the stored scans whose descriptors are nearest to the query 2203.03397 §III-C. Its card describes no geometric check, so one must be added by the SLAM system.

The CT-ICP paper argues that registration-based loop closure only suits small, low-drift trajectories, and that per-scan place recognition is sensitive to change in the environment 2109.12979 §2. The approaches below are the three answers the cards describe for LiDAR.

Variants

Learned overlap and yaw estimator

OverlapNet is a siamese network. Each LiDAR scan is projected to a 64 × 900 range image with depth, normal, intensity and semantic cues [arxiv:2105.11344 §3.2] 2105.11344 §3.3. A shared 11-layer leg maps each scan to a 360 × 128 feature volume 2105.11344 tab. 1. One head predicts the overlap of the two scans in [0, 1]. A second head predicts relative yaw in 1-degree bins, because a yaw change appears as a circular shift of the feature volume 2105.11344 §3.3.1 2105.11344 §3.3.3. Training needs poses to label overlap and yaw, but the network itself takes no poses as input 2105.11344 §3.4. Card: OverlapNet: Loop Closing for LiDAR-based SLAM.

The SLAM integration matters as much as the network. Pose covariance is propagated along odometry, and candidates are gated by Mahalanobis distance inside a 3σ region. The most recent 100 scans are skipped [arxiv:2105.11344 §3.6] 2105.11344 §4.1. Among the 10 candidates with the highest predicted overlap, the one nearest in Mahalanobis distance becomes the loop candidate, and the predicted yaw seeds ICP 2105.11344 §4.1 2105.11344 §4.6.

Author-reported on KITTI sequence 00, AUC/F1 is 0.87/0.88 with all cues, and 0.96/0.96 for the covariance-gated variant 2105.11344 tab. 2 2105.11344 tab. 3. On Ford Campus 00, which was not used for training, the geometry-only model reports 0.85/0.84 2105.11344 tab. 2. Inference takes about 17 ms per pair with depth and normals only. An exhaustive search over orientations takes about 1.2 s per pair, and scoring every candidate takes up to 630 ms 2105.11344 §3.2 2105.11344 §4.7. The paper says online use depends on the pose-uncertainty gate for this reason.

Training-free elevation-image matching with pose-graph correction

CT-ICP's loop closure needs no learned model. It merges a window of 100 odometry scans into one cloud, projects that cloud to a 2D grid keeping the highest point per cell, clips heights, and extracts rotation-invariant 2D features 2109.12979 §4 2109.12979 §5.3. The newest elevation image is compared with the 10 stored images closest to it. A 2D rigid fit by RANSAC is validated by its inlier count, and ICP refines it to a 6-DoF loop constraint 2109.12979 §4. A pose graph is optimised with g2o, and the global optimisation runs only when a new loop constraint is found 2109.12979 §4. Card: CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closure.

The method assumes mostly planar motion and a known vertical axis, and the authors rotate one dataset so that z points up 2109.12979 §4 2109.12979 §5.3. On the paper's own tests, loop closure reduces absolute trajectory error from 6.22 m to 0.66 m on KITTI-raw 00, and from 29.87 m to 1.07 m on KITTI-360 00 2109.12979 tab. 2. It raises error on two sequences, KITTI-CARLA Town01 (0.21 m to 0.26 m) and NCD 01_short_experiment (0.22 m to 0.36 m) 2109.12979 tab. 2. The cost is about 1.1 s per elevation grid and match, plus about 1.2 s per pose-graph optimisation. The hardware behind these times is not given [arxiv:2109.12979 §5.3].

Yaw-invariant transformer descriptor for retrieval

OverlapTransformer turns one LiDAR range image into a 256-d global descriptor. The input is depth only, on a 64 × 900 grid for KITTI 2203.03397 §III 2203.03397 §V-A. Yaw invariance comes from the architecture rather than from augmentation. A yaw change is a circular column shift of the range image. The encoder convolves only along the height, with no padding, so the shift passes through it. One transformer block mixes the columns, and NetVLAD pools over them, which is permutation-invariant 2203.03397 §III-A 2203.03397 §III-B 2203.03397 §III-C. Retrieval is a Euclidean nearest-neighbour search over stored descriptors 2203.03397 §III-C.

Training pairs are labelled with ground-truth poses. A pair is positive when the overlap of the reprojected range images exceeds 0.3, and the loss is a triplet loss with margin 0.5 2203.03397 §III-D [arxiv:2203.03397 §V-A]. The printed loss in eq. 16 has no hinge on the negatives, which would leave it unbounded. The card's Paper vs code section records how the released code differs 2203.03397 eq. 16. Card: OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition.

Author-reported: AUC 0.907 and F1max 0.877 on KITTI 00, the highest of the methods in its table 2203.03397 tab. 2. The model was trained on KITTI only, and on Ford Campus 00, which it did not see in training, the reported AUC is 0.923 and F1max 0.856 2203.03397 §V-B [arxiv:2203.03397 tab. 2]. Descriptor extraction takes 1.37 ms and search 0.44 ms per scan, against 2000 reference scans 2203.03397 tab. 4. These timings were measured by the authors on an i7-11700K with an RTX 3070 [arxiv:2203.03397 §V-F]. The yaw-invariance test is reported only as a plot [arxiv:2203.03397 fig. 7]. On the authors' own Haomo sequences, one transformer block raises Recall@1 from 0.642 to 0.776 and adds about 0.5 ms 2203.03397 tab. 3.

When to use which

Implementation sketch

The common core is the same for all variants. Only the proposal and verification steps differ. Values are the papers' own, and the acceptance threshold is not stated in the cards.

# Common loop-closure step  [arxiv:2105.11344 §4.1] [arxiv:2109.12979 §4]
# Input:  new keyframe (scan, pose, covariance); history of past keyframes
# Output: loop constraints added to the pose graph

def on_new_keyframe(scan, pose, cov, history, graph):
    # Skip the most recent scans, so a scan cannot match its neighbours
    # (OverlapNet and OverlapTransformer use 100)  [arxiv:2105.11344 §4.1]
    past = history.older_than(n_recent=100)
    # Optional gate from odometry covariance: 3-sigma Mahalanobis region
    # (the elevation method and OverlapTransformer do not use it)
    cands = [h for h in past if mahalanobis(pose, h.pose, cov) < 3.0]
    # Rank by a learned overlap score, or by descriptor distance
    # (OverlapTransformer: Euclidean nearest neighbour)
    # [arxiv:2203.03397 §III-C]
    ranked = sorted(cands, key=lambda h: score(scan, h.scan), reverse=True)
    # Keep 10 candidates  [arxiv:2105.11344 §4.1] [arxiv:2109.12979 §4]
    for cand in ranked[:10]:
        # Initial relative pose: predicted yaw, or a RANSAC 2D fit
        # [arxiv:2105.11344 §4.6] [arxiv:2109.12979 §4]
        init = estimate_relative_pose(scan, cand.scan)
        # ICP refines it to a 6-DoF constraint  [arxiv:2109.12979 §4]
        rel, fitness = icp(scan, cand.scan, init)
        # Acceptance threshold: not stated in the cards
        if accept(fitness):
            graph.add_loop_constraint(cand.pose, rel)
            # Global optimisation only when a loop is found
            # [arxiv:2109.12979 §4]
            graph.optimise()
    history.add(scan, pose)

Contradictions

• Metrics differ. OverlapNet reports AUC and F1 per pair [arxiv:2105.11344 tab. 2]. OverlapTransformer reports AUC, F1max and recall [arxiv:2203.03397 tab. 2]. CT-ICP reports trajectory error, ATE in metres, before and after loop closure 2109.12979 tab. 2. The numbers cannot be ranked against each other.
• OverlapNet's KITTI numbers differ between papers. OverlapNet's card reports AUC 0.87 and F1 0.88 on KITTI 00 2105.11344 tab. 2. The OverlapTransformer table lists OverlapNet at AUC 0.867 and F1max 0.865 on the same sequence 2203.03397 tab. 2. The Ford Campus 00 figures agree closely: 0.85/0.84 in one and 0.854/0.843 in the other. The papers do not say which OverlapNet variant or F1 definition each uses, so neither number is adopted here.
• Dataset labels differ. OverlapNet's KITTI 00 is an odometry sequence 2105.11344 §4. CT-ICP's KITTI-raw 00 is a raw-data sequence 2109.12979 tab. 2. Check the source before comparing.
• Loop closure helps and hurts in CT-ICP. ATE drops strongly on KITTI-raw 00, KITTI-360 00 and ParisLuco, and only modestly on NCLT (2.97 to 2.58). It gets worse on KITTI-CARLA and NCD 2109.12979 tab. 2. The paper's text describes the CARLA change as almost unchanged 2109.12979 §5.3.
• Planar motion. CT-ICP's loop closure requires mostly planar motion 2109.12979 §4. The OverlapNet and OverlapTransformer cards record no comparable requirement, and no source in these cards tests the methods on the same non-planar data.

Papers

• OverlapNet: Loop Closing for LiDAR-based SLAM: a siamese network for overlap and yaw between two scans, gated by pose covariance inside SuMa-style SLAM.
• CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closure: elevation-image loop closure on local maps, with a pose graph, and no learned component.
• OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition: a yaw-invariant range-image transformer that returns a 256-d descriptor for nearest-neighbour retrieval, trained with overlap labels that follow OverlapNet.