TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Driving

In plain words

At a junction, one lane's end is supposed to sit exactly where the next lane begins. A model that predicts each lane on its own often gets those two ends slightly apart, so it cannot tell which lanes connect. The error comes from the lane predictions, not from the connection step.

TopoPoint treats the meeting points as objects of their own. It predicts them separately from the lanes, then lets the points and the lanes share information while the model learns. The connection checks then use these points as well. When the model runs, the predicted points pull nearby lane ends toward one shared spot, so lanes that meet there line up. The authors report better scores than earlier lane-topology methods on the OpenLane-V2 benchmark.

TL;DR

TopoPoint adds an explicit lane-endpoint detector to a bird's-eye-view lane-topology model. Points and lanes have separate queries that interact through a merged self-attention and a point-lane graph network. At inference the lane ends are averaged with nearby detected points. On OpenLane-V2 subset_A the authors report OLS 48.8 and DET_p 52.6. The released code differs from the paper in its inference step and in several training details, see Paper vs code.

Problem

Topology reasoning means predicting which lanes connect to which, and which lanes connect to traffic elements. Its quality depends on lane detection, especially at the ends of connected lanes §1. In query-based models, the lane ends are supervised together with the whole lane, so the ends of connected lanes do not coincide in the final output. The authors call this endpoint deviation §1 fig. 1.

Earlier work tackled the problem in other ways. STSU moves whole lanes to align their ends, and LaneGAP merges lane pieces into paths. The authors note that these methods were later replaced, because lane detection was not yet accurate enough for them §1. TopoLogic reduces the effect of the deviation by combining lane-lane distance and semantic similarity, but it does not fix the lane predictions, so the deviation remains §1.

Method

TopoPoint has five parts: a traffic-element detector, a point-lane detector, a topology head, a training objective, and a matching step at inference §3.2 fig. 2.

Step 1. Inputs and BEV features. The multi-view images go through a ResNet-50 backbone with an FPN neck, then a BEVFormer encoder produces the BEV features §3.2. The front view is passed at its original resolution §4.2.

Step 2. Traffic detector. Traffic-element queries are refined against the front-view features by Deformable DETR, which gives Q̂_t eq. 1. A traffic head turns these into 2D boxes eq. 2 §3.3.

Step 3. Point and lane queries. Point queries and lane queries are initialised independently §3.4. They first pass through the Point-Lane Merge Self-Attention (PLMSA). The two sets are concatenated into one set of queries eq. 3. The paper defines the geometric terms used in this step from the previous layer's predictions. Lane-lane distances D_ll are the L1 distances from each lane's end to each lane's start eq. 4. Point-lane distances D_pl are the smaller of the L1 distances from a point to the start and to the end of a lane eq. 5. Both are mapped through f_map, which is taken from TopoLogic, giving M_pl and M_ll eq. 6 §3.4. In f_map, λ and α are learnable, and σ is the standard deviation of the distance matrix D §3.4. The paper states that M_pl and M_ll are added as a mask to the attention logits eq. 7, followed by layer normalisation eq. 8.

Step 4. Deformable cross-attention. Points and lanes each have their own learnable reference points, R_p and R_l §3.4. Each attends to the BEV features with deformable attention eq. 9, then goes through its own FFN eq. 10.

Step 5. Unified scene graph network. The point-lane graph convolution (PLGCN) uses the adjacency A_pl, built from the point-lane topology G_pl and M_pl with learnable weights λ1 and λ2 eq. 11. Points receive messages from lanes through A_pl, and lanes receive messages from points through its transpose, with residual connections eq. 12 §3.4.

The scene graph then runs in four parts eq. 13-16 fig. 3:

1. PLGCN_1 exchanges features between points and lanes eq. 13.
2. The lane branch runs two graph convolutions. One is over the lane-lane graph, with adjacency I + M_ll + M_llᵀ. The other pulls in traffic queries through G_lt. Both are added to the lane features and concatenated, then a linear layer reduces them back to the lane width eq. 14.
3. PLGCN_2 runs again on the new lane features eq. 15.
4. The output queries are the points and lanes after step 3 eq. 16.

Step 6. Heads. The point head regresses one 3D endpoint and a score per point query. The lane head regresses a lane of k = 11 points, each in 3D, and a score per lane query eq. 17 §3.4. Each head has two separate MLP branches, one for regression and one for classification §3.4.

Step 7. Topology head. Three matrices come from pairwise dot products of MLP embeddings, each passed through a sigmoid: point-lane G_pl, lane-lane G_ll, and lane-traffic G_lt eq. 18-20 §3.5. The matrices of the previous decoder layer feed the next layer §3.4.

Step 8. Training. The total loss is a weighted sum of the traffic, point and lane detection losses and the three topology losses eq. 21 §3.6. The point and lane losses use a focal classification term and an L1 regression term. The traffic loss adds a GIoU term. All three topology losses are focal losses §3.6.

Step 9. Point-Lane Geometry Matching at inference. The paper keeps points and lanes whose scores exceed τ_p and τ_l alg. 1 step 1. For each kept point, it finds the kept lanes whose endpoint lies within a distance δ of the point alg. 1 step 2. The point and those matched endpoints are averaged, and every matched lane takes the averaged value as its endpoint alg. 1 step 2 §3.7. The released code does this differently; see Paper vs code.

# Point-lane detector, one decoder layer  [§3.4] [eq. 3-16]
# Input:  points (Np, C), lanes (Nl, C), traffic (Nt, C), bev (HW, C)
# Output: points (Np, C), lanes (Nl, C); C = 256, Np = 200, Nl = 300 [§4.2]

def point_lane_layer(points, lanes, traffic, bev, prev):
    # Geometric biases from the previous layer's predictions  [eq. 4-6]
    # prev.lane_start, prev.lane_end: (Nl, 3); prev.points: (Np, 3)
    d_ll = pairwise_l1(prev.lane_end, prev.lane_start)   # (Nl, Nl)
    d_pl = point_to_lane_min_l1(prev.points, prev.lane_start, prev.lane_end)
    m_ll = f_map(d_ll)                                   # (Nl, Nl)
    m_pl = f_map(d_pl)                                   # (Np, Nl)

    # Merged self-attention over points and lanes  [eq. 3, 7, 8]
    joint = concat(points, lanes, dim=0)                 # (Np+Nl, C)
    bias = block([[zeros(Np, Np), m_pl], [m_pl.T, m_ll]])
    attn = softmax(joint @ joint.T / sqrt(C) + bias)
    joint = layer_norm(attn @ joint)
    points, lanes = split(joint, [Np, Nl], dim=0)

    # Deformable cross-attention to BEV, separate reference points  [eq. 9-10]
    points = ffn_p(layer_norm(deform_attn(points, ref_p, bev)))
    lanes = ffn_l(layer_norm(deform_attn(lanes, ref_l, bev)))

    # PLGCN: points and lanes exchange features  [eq. 11-13]
    a_pl = lam_1 * g_pl + lam_2 * m_pl                   # learnable lam
    points, lanes = plgcn_1(points, lanes, a_pl)

    # Lane-lane and lane-traffic branches, then downsample  [eq. 14]
    lane_branch = gcn_ll(lanes, eye(Nl) + m_ll + m_ll.T)
    traffic_branch = gcn_lt(traffic, g_lt)
    lanes = linear(concat(lane_branch + lanes, traffic_branch + lanes))

    # Second PLGCN on the updated features  [eq. 15-16]
    points, lanes = plgcn_2(points, lanes, a_pl)
    return points, lanes

# Point-Lane Geometry Matching at inference  [§3.7] [alg. 1]
# Input:  points P, point scores, lanes L, lane scores
# Output: lanes with refined endpoints
# tau_p = tau_l = 0.3; delta = 1.5 m [§4.2]

def refine_endpoints(points, point_scores, lanes, lane_scores):
    # Keep only confident points and lanes  [alg. 1 step 1]
    kept_points = points[point_scores > 0.3]
    kept_lanes = lanes[lane_scores > 0.3]
    for point in kept_points:
        # Lanes whose endpoint lies within delta of this point  [alg. 1 step 2]
        matches = [lane for lane in kept_lanes
                   if l1(point, lane.endpoint) < 1.5]
        if matches:
            # Average the point with the matched endpoints  [alg. 1 step 2]
            refined = (point + sum(m.endpoint for m in matches))
            refined = refined / (len(matches) + 1)
            for lane in matches:
                lane.endpoint = refined
    return kept_lanes

Note that the paper's algorithm does not say which endpoint of a lane is matched; the released code uses different rules (see Paper vs code).

Hyperparameters

Values from the paper are cited to the paper. Values marked "code" come from the official configuration or code, not from the paper text. Values marked "not stated" are absent from both.

Results

Author-reported. Setup: OpenLane-V2 v2.1.0 metrics. TopoPoint is trained for 24 epochs with a ResNet-50 backbone §4.2. The other rows are copied from earlier papers tab. 1.

Metric definitions: DET_l uses the Fréchet distance at thresholds 1.0, 2.0 and 3.0 §4.1. OLS averages DET_l, DET_t, √TOP_ll and √TOP_lt eq. 22. DET_p is the authors' own endpoint metric: a point-wise Fréchet-based AP averaged over the same three thresholds eq. 23 §4.1.

Main comparison tab. 1. TopoFormer* is marked in the paper as using a pretrained lane detector. TopoNet, TopoMLP and TopoLogic DET_p values were computed by the authors with the official codebases. Other values are copied from the TopoLogic and TopoFormer papers tab. 1.

Module ablation on subset_A tab. 2. The baseline is TopoLogic code re-run by the authors.

GCN ablation on subset_A tab. 3. "w/o GCN" removes the unified graph network.

Image scales on subset_A tab. 4. S_fv is the front-view scale and S_mv the multi-view scale.

Query counts on subset_A tab. 5.

Paper vs code

The official repository is Franpin/TopoPoint at 30f4bf2f3a0de1a87ca3c8c452debf8c0ee0e9b7. The owner, Franpin, also owns the TopoLogic repository, whose first author is Yanping Fu. The README names Yanping Fu as primary contact README.md, so the repository belongs to the authors.

• The point-to-lane matching is not active. The paper's matching fuses detected points into lane ends. The released code's version that does this is commented out lane_coder.py. The point predictions are decoded and output separately, and they do not change the lanes lane_coder.py.
• The active refinement only uses lane ends. get_lanes takes the lane scores above 0.3 and matches each lane's end to other lanes' starts. Matching uses a geometric similarity, not the paper's direct distance threshold topopoint_head.py.
• The matching threshold is not 1.5 m. A similarity threshold of 0.7 with scale w = 11.5275 and exponent 2 is used topopoint_head.py. Solving exp(-d² / 11.5275) > 0.7 gives an L1 distance of about 2.03, not 1.5.
• Two passes, one distance matrix. The code refines end-to-start, then start-to-end. The distances are computed once, before either pass topopoint_head.py topopoint_head.py topopoint_head.py. The second pass therefore uses distances from before the first pass changed the lanes.
• Lane-lane output adds a geometric term. The reported lane-lane topology is the sigmoid of G_ll plus the geometric matrix, with weights of one, and it is computed after refinement topopoint_head.py. The paper's G_ll is the sigmoid alone §3.5.
• f_map parameters differ between training and inference. Training uses learnable parameters, initialised to 10 for the scale and 2 for the exponent sgnn_decoder.py. Inference uses the constant 11.5275 topopoint_head.py topopoint_head.py. The σ normalisation from the paper is computed but not used topopoint_head.py topopoint_head.py.
• No geometric attention mask. The paper's PLMSA adds M_pl and M_ll to the attention logits. The code builds no mask and passes none to the decoder, so the self-attention is plain sgnn_decoder.py transformer_decoder_only.py. The geometric matrices are used only as graph adjacencies sgnn_decoder.py sgnn_decoder.py.
• Graph convolutions differ from the equations. The code uses the raw adjacency, with no degree normalisation and no sigmoid. Neighbour messages are scaled by 0.6, and forward and backward directions have separate weights sgnn_decoder.py [config: projects/configs/topopoint_r50_8x1_24e_olv2_subset_A.py#L214]. The lane-lane branch uses M_ll directly, not I + M_ll + M_llᵀ. A ReLU is applied before the linear downsample sgnn_decoder.py.
• Lane-traffic features are class-weighted. Each traffic query is scaled by its sigmoid class scores, which are detached sgnn_decoder.py. The paper does not describe this.
• Loss weights differ from the paper. The paper sets all detection weights to 1.0 §4.2. The code uses the weights listed in Hyperparameters, and points and lanes share the same lane-head loss weights topopoint_head.py.
• Front view. The paper says the front view is cropped and padded §4.2. The code crops a fixed row range and does not pad it there transform_3d.py.
• BEV orientation. The paper writes 200 x 100. The code uses 100 rows and 200 columns [config: projects/configs/topopoint_r50_8x1_24e_olv2_subset_A.py#L42-L43].
• README links. The README badge shows arXiv 2405.17771 and its update line links 2405.14747 (TopoLogic). The paper is 2505.17771 README.md.
• Licence. The repository has no LICENSE file, so code_license is recorded as none. The openlanev2/ directory carries Apache-2.0 headers and projects/bevformer/ carries OpenMMLab headers. Neither is the authors' licence. No code is quoted in this card.

Pitfalls

• Matching in the paper is not what ships. Implement the rules in Paper vs code, or the reported inference gain is not reproduced.
• The point detector's output does not reach the lanes. In the released code, point predictions are emitted separately, so a point-driven refinement needs the commented-out block rebuilt lane_coder.py.
• Sequential, order-dependent updates. Each matched lane is overwritten by the average computed for the current point, so later points can overwrite earlier ones topopoint_head.py. The paper does not say how overlapping matches are resolved.
• Which endpoint is matched. The paper writes "the endpoint" of a lane without saying which end. The code matches the end to starts in one pass and starts to ends in the other topopoint_head.py.
• Gradients stop at the geometry. The geometric matrices are built from detached coordinates, so the topology losses do not move the endpoints through them sgnn_decoder.py. The learned point-lane relation is detached before it enters the graph update sgnn_decoder.py. The paper does not say this.
• Reference points are 2D for sampling. The paper's reference points have three coordinates §3.4. The decoder samples with the first two only sgnn_decoder.py.
• Query projections are not stated. Equation 7 has no Q, K or V projections, and the code uses multi-head attention with its own projections (8 heads) [config: projects/configs/topopoint_r50_8x1_24e_olv2_subset_A.py#L188-L193].
• Equation 8 has a typo. It writes LN(Q_p) twice; the second should be LN(Q_l) §3.4 eq. 8.
• Endpoint de-duplication is exact. Ground-truth endpoints are merged only when their coordinates are identical, with no tolerance. Nearly identical endpoints remain separate candidates openlanev2_subset_A_dataset.py.
• Topology targets are inverted for the focal loss. The targets are passed as 1 minus the adjacency, which matches the focal loss's class-index convention. A reimplementation using binary cross-entropy should use the adjacency directly relationship_head.py.
• Fixed thresholds. Output filtering (0.3), the refinement filter (0.3) and the matching threshold (similarity 0.7) are fixed numbers in the code. The paper does not report how sensitive results are to them.
• Sensitivity to query counts. N_p = 300 is worse than 200 for endpoint detection tab. 5 §4.4.
• Front-view resolution matters for traffic elements. Native front-view scale raises DET_t from 48.6 to 55.3 in the ablation tab. 4. The cost in compute and memory is not stated.
• Stated limitations. TopoPoint relies on accurate calibration. It may underperform in bad conditions or dense traffic, because the number of queries is fixed §5.
• Environment. The README pins Python 3.8, PyTorch 1.9.1 and CUDA 11.1 README.md. The config has a hard-coded data path [config: projects/configs/topopoint_r50_8x1_24e_olv2_subset_A.py#L23].

Lineage

• Builds on: TopoLogic for the f_map distance function and the baseline code §3.4 tab. 2. Also Deformable DETR, BEVFormer and ResNet-FPN, which are not compiled here §3.2 §4.2.
• Compared against: STSU, VectorMapNet, MapTR, TopoNet, TopoMLP, TopoLogic and TopoFormer tab. 1.
• Superseded by: not stated in the paper.