TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenes

In plain words

A self-driving car has to know which lane leads into which. At a busy intersection, a lane that ends in one place must line up with the lane that starts there. Earlier systems learned this with a generic neural layer that looked at each lane on its own. Small errors in where a detected lane ends could make two lanes that really connect look disconnected.

TopoLogic checks the connection in two ways that can be read by a person. The first looks at geometry: how far apart are the end of one lane and the start of the next? A small gap suggests a connection. The second compares the internal descriptions the model has of each pair of lanes, and asks whether they look similar. The two answers are combined and passed to a graph layer, so each lane can borrow information from the lanes it connects to. The paper also shows that the geometric check can be added after a model has already been trained, without retraining it.

TL;DR

• Lane topology (which lane connects to which) is predicted from the distance between lane endpoints and from the similarity of lane queries, then fused §3.3-§3.5.
• The distance-to-score function is learnable and has a wider tolerance for endpoint shift than Gaussian, sigmoid and tanh alternatives §3.3 fig. 3 tab. 3.
• Author-reported on OpenLane-V2 subset_A with a ResNet-50 backbone: TOP_ll 23.9 and OLS 44.1 (v2.1.0), against TopoNet's 10.9 and 39.8 tab. 1.
• The geometric step can run as post-processing on a trained model, raising TopoNet's TOP_ll from 10.9 to 22.3 tab. 5.

Problem

Lane detectors predict lane centerlines. Topology reasoning then predicts which lanes connect. Prior methods mostly improved the detection side and used an MLP on each lane query to predict the connections §1 §2.2. The authors argue that each lane is encoded independently, so the endpoints of two connected lanes are unlikely to overlap exactly. A small endpoint shift can then turn a true connection into a missed one, so the model predicts too few connections §1 fig. 1. The paper's experiments use OpenLane-V2 and assume the seven-view subset_A or six-view subset_B input §4.1.

Method

The pipeline is an image encoder followed by a lane decoder §3.2 fig. 2. Multi-view images go through a backbone and FPN, a BEVFormer-style view transformer produces BEV features, and a Deformable DETR-style lane decoder produces lane queries and lane points §3.2 §4.2.

Notation. A lane is an ordered list of 3-D points. The topology is a directed graph over lane instances, and an entry (i, j) is positive when the end of lane i meets the start of lane j §3.1.

Step 1. Geometric distance. For each pair of lanes, take the distance from the end point of lane i to the start point of lane j. This gives a distance matrix D §3.3 eq. 1-3. The paper's notation uses an absolute value. The code uses the L1 norm over x, y and z sgnn_decoder.py.

Step 2. Distance to score. Map each distance to a score between 0 and 1, where a distance near zero gives a score near 1 §3.3. The paper's learnable function is

f(x) = \exp\!\left(-\frac{x^{\alpha}}{\lambda\,\sigma}\right) eq. 4

where sigma is the standard deviation of D, and alpha and lambda are learnable §3.3. The alternatives compared are exp(-x^2/2), 2/(1+e^x) and a tanh form eq. 6 fig. 3 tab. 3. The code does not use sigma in the mapping. See the Paper vs code section.

Step 3. Similarity. Encode the lane queries with two separate three-layer MLPs. Take the dot product of the two encodings to form S, and apply a sigmoid to get a connection score in (0, 1) §3.4 eq. 7-9.

Step 4. Fusion. Combine the two matrices with two learnable scalars, G = lambda1 * G_dis + lambda2 * G_sim §3.5 eq. 10. Both start at 1 §4.2.

Step 5. Graph update. The fused topology feeds a graph layer that mixes each lane's features with those of its connected lanes. This runs in the decoder so that the next layer sees the updated lanes §3.2 fig. 2. The paper does not write the graph update as an equation. The code's version is in the Pitfalls section.

Step 6. Learning. The total loss is L = L_det + L_top eq. 11 §3.6. L_det is the lane detection loss: a focal loss for classification and an L1 loss for the point offsets §3.6 §4.2. L_top is the loss on the similarity matrix G_sim only §3.6. The geometric term has no loss of its own. Its parameters are updated through L_det §3.6. Supervision is applied to every decoder layer §3.6.

# Lane-topology step after one decoder layer  [§3.3-§3.5] [eq. 4, 7-10]
# Input:  queries (N, C) with N = 200, C = 256; lanes (N, 11, 3)
# Output: queries (N, C) and topology adjacency (N, N) for the next layer

def topology_step(queries, lanes, w, P, lam_1, lam_2):
    # Gap from the end of lane i to the start of lane j  [§3.3] [eq. 2]
    # Code uses the L1 norm over x, y, z  [code: sgnn_decoder.py#L93]
    gap = abs(lanes[:, None, -1, :] - lanes[None, :, 0, :]).sum(-1)  # (N, N)

    # Map the gap to a score; the diagonal is masked  [§3.3] [eq. 4]
    # Paper: exp(-x^alpha / (lambda * sigma)); code: exp(-x^P / w)
    geo = exp(-gap ** P / w) * (1 - identity(N))  # (N, N)

    # Similarity: two MLPs, dot product, sigmoid  [§3.4] [eq. 7-9]
    emb_a = mlp_a(queries)                   # (N, C)
    emb_b = mlp_b(queries)                   # (N, C)
    sim = sigmoid(emb_a @ emb_b.T)           # (N, N)

    # Fuse the two matrices with learnable scalars  [§3.5] [eq. 10]
    adj = lam_1 * sim + lam_2 * geo          # (N, N)
    return adj

The graph update that uses adj is described in the Pitfalls section. The loss is described in Step 6 above eq. 11.

Hyperparameters

Values marked "code" come from the official configuration or code, not from the paper text. Values marked "not stated" are absent from both.

Mapping-function hyperparameters in the code are learnable scalars with initial values P = 2 and w = 10, and are not stated as such in the paper sgnn_decoder.py.

Results

Author-reported. Benchmark: OpenLane-V2 §4.1. Table 1 covers centerline perception, Table 2 lane-segment perception, Tables 3 and 4 ablations and Table 5 the post-processing use. Numbers are copied from the paper's tables; the paper says results for existing methods come from TopoNet and SMERF tab. 1.

Centerline, subset_A, no SDMap tab. 1

Centerline with SDMap, subset_A tab. 1: SMERF gives TOP_ll 15.4 and OLS 42.9 (v2.1.0); TopoLogic with SDMap gives TOP_ll 28.9 and OLS 47.5 (v2.1.0).

Centerline, subset_B, no SDMap tab. 1: TopoNet gives TOP_ll 6.7 and OLS 36.8 (v2.1.0); TopoLogic gives TOP_ll 21.6 and OLS 42.3 (v2.1.0).

Lane segment, subset_A tab. 2

Ablations on subset_A, v2.1.0 metrics

Post-processing on trained models tab. 5: adding the geometric distance step without retraining raises TOP_ll from 10.9 to 22.3 for TopoNet, from 15.4 to 26.2 for SMERF, and TOP_lsls from 25.4 to 29.6 for LaneSegNet.

Paper vs code

Checked at commit c7b37f73bc1b92ac8646351a9c63e86a5d74e8c8 (raw/code/2405.14747). The repository has no LICENSE file, so no code is quoted verbatim in this card; all code references are citations only.

• Mapping function. The paper's alpha and lambda are learnable and start at 0.2 and 2 §3.3 §4.2. The code's learnable scalars are P (starts at 2) and w (starts at 10), and the mapping is exp(-d^P / w) sgnn_decoder.py sgnn_decoder.py. The paper's sigma does not appear in the mapping. The code computes sigma but never uses it sgnn_decoder.py topologic_head.py.
• Inference path. The post-processing path in get_lanes hard-codes P = 2, w = 11.5275 and lambda1 = lambda2 = 1. The lines that would read the learned values are commented out topologic_head.py. Results from this path will not reflect the learned parameters.
• Distance norm. The paper writes an absolute value. The code uses the L1 norm over x, y and z sgnn_decoder.py.
• Lane-to-traffic-element topology. The code has a second relation head between lanes and traffic elements, with its own focal loss (weight 5) topologic_r50_8x1_24e_olv2_subset_A.py topologic_head.py. The paper does not describe it, and its L_top covers only G_sim §3.6.
• Loss weights. The paper states focal weight 1.5 and L1 weight 0.025 §4.2. The code matches. The paper does not state the topology loss weight, which the code sets to 5 topologic_r50_8x1_24e_olv2_subset_A.py.
• Batch size. The paper says batch size 2 on 8 GPUs §4.2. The code uses 1 sample per GPU, so 8 in total topologic_r50_8x1_24e_olv2_subset_A.py.
• Input size. The paper says all images are resized to 1550 x 2048 §4.2. The code crops the front view to rows 356-1906 and then applies a random scale from the list [0.5] to all views transform_3d.py transform_3d.py topologic_r50_8x1_24e_olv2_subset_A.py.
• BEV size. The paper says 200 x 100 §4.2; the code uses 100 rows and 200 columns topologic_r50_8x1_24e_olv2_subset_A.py. This is the same grid, with the axes named differently.
• Graph update. The code uses a graph layer with a self term, a forward term and a backward term, weighted by 0.6 sgnn_decoder.py topologic_r50_8x1_24e_olv2_subset_A.py. The paper does not give this equation.
• Schedule. The code sets a linear warmup of 500 iterations and gradient clipping at 35. The paper states neither topologic_r50_8x1_24e_olv2_subset_A.py topologic_r50_8x1_24e_olv2_subset_A.py.

Pitfalls

• Text and table disagree in places. The text gives TopoNet's subset_A TOP_ll (v2.1.0) as 10.8 §4.2, while Table 1 and the abstract give 10.9. The text gives TopoNet's subset_B DET_l as 24.4, while Table 1 gives 24.3 tab. 1. Use the table values.
• OLS formula. The paper writes OLS = 1/4 [DET_l + DET_t + f(TOP_ll) + f(TOP_lt)] with f the square root §4.1. Using the table values, this reproduces 44.1 only if TOP is taken as a fraction and multiplied by 100 before the square root. Our check: sqrt(0.239) x 100 is about 48.9, and the mean with DET_l 29.9 and DET_t 47.2 is about 44.1. Read the metric definitions in the OpenLane-V2 devkit before reimplementing it.
• Ambiguous graph timing. The adjacency that the layer-l graph update uses comes from layer l-1. The first layer receives zeros sgnn_decoder.py sgnn_decoder.py. The paper does not say this.
• Detachment. The geometric distance is computed on detached lane points sgnn_decoder.py. Gradients reach the mapping parameters and features, but not the lane point coordinates through this term. The similarity term is detached before it feeds the graph layer sgnn_decoder.py, so it is trained only by its own loss.
• Label inversion. The topology targets are inverted (1 minus the adjacency) before the focal loss relationship_head.py. Unmatched lane pairs are labelled as no connection. The paper does not say how unmatched queries are treated. Keep the same convention when reimplementing.
• Unscaled dot product. The similarity is a raw dot product of 256-dimensional embeddings, with no scaling relationship_head.py. The paper does not say whether it should be scaled.
• Numerically sensitive exponent. The mapping raises the gap to a learnable power (alpha in the paper, P in the code). Its value after training is not reported, and neither paper nor code bounds it. Because the gap is raised to that power and then divided by w or lambda times sigma, a learned P far from its start can change the score sharply. Check the learned values before fixing them. The point-cloud range is set in topologic_r50_8x1_24e_olv2_subset_A.py.
• Post-processing needs a trained model. The paper's post-processing gains are reported for TopoNet, SMERF and LaneSegNet tab. 5. The repository's get_lanes is the TopoLogic head. We did not find code for applying the step to those other models in this repository.
• Costs the abstract hides. The paper trains on 8 RTX 3090 GPUs for 24 epochs §4.2. The lane-to-lane graph is N x N with N = 200 queries §4.2, so memory grows with the square of the query count.
• Scope. The paper says it is for research only and should not be deployed in autonomous driving. It makes no safety guarantee §5. The graph layer improves topology more than detection §5, Limitations.

Lineage

• Builds on: TopoNet (Li et al., 2023) for centerline topology with a GNN §2.2; Deformable DETR for lane queries §4.2; BEVFormer for the view transformer §4.2. These works are not compiled here.
• Compared against: TopoNet, SMERF, LaneSegNet, VectorMapNet, MapTR, MapTRv2, STSU tab. 1 tab. 2.
• Superseded by: not stated in the paper. The code README links a later work from the same group, TopoPoint README.md.