TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoning

In plain words

A self-driving car has to know which lanes are on the road, how they join at junctions, and which traffic light or sign belongs to which lane. This paper treats that as two steps: first find the lanes and the traffic elements, then decide for every pair whether they connect. Each lane is drawn as a smooth curve described by a few points, which keeps the output compact.

The main claim is that the second step is easy once the first is good. If the lanes and signs are found accurately, a plain small neural network can read their connections from simple features, without a complicated graph model. The authors show that the connection score rises with detection quality, and they note that the common scoring metric can be gamed by raising confidences of false detections. They propose a corrected metric for that case.

TL;DR

• Driving-scene topology: detect 3D lane centerlines and traffic elements, then predict lane-lane and lane-traffic connections §1, §3.
• Lanes are learned queries that output Bezier control points; a 2D traffic detector can use optional YOLOv8 boxes as proposals §3.1, §3.2.
• Both topology heads are MLPs over pairs of query embeddings §3.3, §3.4.
• Author-reported: 38.2 OLS with ResNet-50 on OpenLane-V2 subset A, 41.2 with extra YOLOv8 proposals tab. 1.
• The official code is Apache-2.0 and differs from the paper in several hyperparameters and in an unstated auxiliary training branch (see Paper vs code).

Problem

Topology reasoning needs four outputs: lane detection, traffic-element detection, lane-lane links and lane-traffic links. These tasks depend on each other: a missed lane or sign means a missed connection §1, fig. 1. The authors measured this on OpenLane-V2 with a fixed detector: when the topology predictions were replaced by ground truth, the topology scores rose only a little (Swin-B: TOP_ll 9.5 to 10.0, TOP_lt 28.3 to 30.9), so detection quality was the main limit §1, fig. 1.

Earlier topology methods used graph modelling or MLPs on image-space features. The paper's view is that a 3D lane representation plus position embedding is enough §2.2, §1. The OpenLane-V2 metric is an mAP that depends on ordering. False positives with confidence 1.0 can be pushed down, and this inflates the score §1, §4.6.

Assumptions: multi-view camera input, per-frame prediction, and OpenLane-V2 annotations §4.1. Two subsets exist, based on Argoverse 2 (subset A, seven views) and nuScenes (subset B, six views) §4.1.

Method

The model has one shared image backbone with four heads: a 3D lane decoder, a 2D traffic decoder, a lane-lane topology head and a lane-traffic topology head. Topology heads read the decoder outputs and the predicted lane points §3, fig. 2.

1. Image features. A backbone (ResNet-50, VoV or Swin-B) yields multi-view feature maps. The 3D position embedding of PETR is added to the features §3.1. The lane branch uses the upsampled C5 level fused with C4 through an FPN; the traffic branch uses C3, C4 and C5 §4.2.

2. Lane queries. There are N_L learnable 3D anchor points, one per lane query, mapped to C channels by a position encoding and a linear layer. A transformer decoder updates them, which gives the query features §3.1, eq. 1.

3. Bezier control points. Two independent MLPs predict, for every query, a control-point offset and a class score. Each query is one lane, given by M control points. The final control points are the anchors plus the offsets §3.1. The lane points used for loss and topology are obtained from the control points by evaluating the Bezier curve §3.1.

4. Traffic elements. The traffic branch follows the Deformable DETR head. It builds reference boxes (centre, height, width) from randomised queries. Optional YOLOv8 detections are added as extra proposals. Two MLPs predict class and box from the decoded features §3.2, eq. 2.

5. Lane-lane topology. Lane coordinates are embedded by an MLP and added to the lane query features §3.3. The N_L lane features are repeated along rows and columns and concatenated, giving an N_L x N_L x 2C tensor. An MLP scores each pair §3.3, eq. 3.

6. Lane-traffic topology. A view transformation matrix A (3x3, built from camera intrinsics and extrinsics §3.4) is embedded by an MLP and added to the lane query features. Lane and traffic features are repeated and concatenated into an N_L x N_T x 2C tensor, and an MLP scores each pair §3.4, eq. 4.

7. Matching. Bipartite matching is used only for the lane and traffic detection losses. The same assignment is reused for the topology losses §3.5.

8. Loss. The total loss is the sum of the two detection losses and the two topology losses §3.5, eq. 5. The per-term weights are given in Hyperparameters.

Equations, as in the paper:

\hat{\bm{Q}}^L = \textbf{LaneDecoder}(\bm{F}, \textbf{Linear}(\bm{Q}^L)) eq. 1

\hat{\bm{Q}}^T = \textbf{TrafficDecoder}(\bm{F}^0, \bm{Q}^T, \bm{R}^T) eq. 2

\bm{G}^{LL} = \textbf{MLP}(\hat{\bm{Q}}^{LL}) \in \mathbb{R}^{N_L \times N_L} eq. 3

\bm{G}^{LT} = \textbf{MLP}(\hat{\bm{Q}}^{LT}) \in \mathbb{R}^{N_L \times N_T} eq. 4

\mathcal{L} = \mathcal{L}_{det_{l}} + \mathcal{L}_{det_{t}} + \mathcal{L}_{top_{ll}} + \mathcal{L}_{top_{lt}} eq. 5

Metric, as in the paper:

\text{OLS} = \frac{1}{4}\left[\text{DET}_l + \text{DET}_t + f(\text{TOP}_{ll}) + f(\text{TOP}_{lt})\right], \quad f = \sqrt{\cdot} eq. 6

\text{TOP} = \frac{1}{|V|}\sum_{v \in V} \frac{\sum_{\hat{n}' \in \hat{N}'(v)} P(\hat{n}')\, \mathbb{1}_{\text{cond}}(\hat{n}' \in N(v))}{|N(v)|} eq. 7

\text{TOP}^{\dagger} = \frac{1}{|V|}\sum_{v \in V} \frac{\sum_{\hat{n}' \in \hat{N}'(v)} P(\hat{n}')\, \mathbb{1}_{\text{cond}}(\hat{n}' \in N(v)) \frac{N_{TP}}{N_{TP} + N_{FP}}}{|N(v)|} eq. 8

Forward pass, our wording of §3 and §4.2 (one view per call; shapes are the paper's, with N_L = 300 and C = 256):

# Lane branch: N_L queries, each one a Bezier curve  [§3.1] [eq. 1]
# Input:  images (V, 3, H, W) for all views; anchors (N_L, 3) learned
# Output: control_points (N_L, M, 3) in BEV metres
def predict_lanes(images, anchors):
    # Multi-view features with 3-D position embedding  [§3.1]
    features = encode_with_3d_position(images)    # (V, C, H', W')
    # Same learned point per control point  [code: lane_head.py#L430-L431]
    anchor_points = repeat_per_control_point(anchors, M)  # (N_L, M, 3)
    lane_queries = lane_decoder(features, linear(anchor_points))  # (N_L, C)
    # Offsets are added in logit space, then squashed into the BEV range
    offsets = lane_reg_head(lane_queries)        # (N_L, M * 3)
    control_points = sigmoid(logit(anchor_points) + offsets)
    return scale_to_bev_range(control_points)

# Lane-lane topology  [§3.3] [eq. 3]
# Input:  lane_queries (N_L, C); control_points (N_L, M, 3)
# Output: logits (N_L, N_L); one score per ordered lane pair
def lane_lane_logits(lane_queries, control_points):
    # Embed lane features, then add an embedding of lane coordinates  [§3.3]
    # The code detaches both inputs  [code: topo_ll_head.py#L105-L109]
    embeds = mlp_embed(lane_queries)             # (N_L, 128)
    embeds = embeds + mlp_coords(flatten(control_points))  # (N_L, 128)
    # Pair every row lane with every column lane  [§3.3]
    rows = repeat_rows(embeds, N_L)              # (N_L, N_L, 128)
    cols = repeat_cols(embeds, N_L)              # (N_L, N_L, 128)
    pairs = concat(rows, cols, axis=-1)          # (N_L, N_L, 256)
    return mlp_classify(pairs)                   # (N_L, N_L, 1)

# Lane-lane topology loss  [§3.5] [code: topo_ll_head.py#L174-L192]
# Input:  logits (N_L, N_L); gt_adj (N_L, N_L), 1 if connected;
#         lanes (N_L, 11, 3) sampled from the Bezier curves
# Output: scalar loss over matched lane pairs
def lane_lane_loss(logits, gt_adj, lanes):
    # Focal loss on the binary connection label  [§3.5] [eq. 5]
    # Code inverts the label: 0 means connected  [code: topo_ll_head.py#L157]
    # Weight 5 on this term  [§4.2]
    loss_rel = 5 * focal_loss(logits, 1 - gt_adj)
    # Distance from the end of lane i to the start of lane j, for connected
    # pairs only; the paper calls these "intersection points" [§4.4]
    # [code: topo_ll_head.py#L185-L186]
    ends = lanes[:, -1]                          # (N_L, 3)
    starts = lanes[:, 0]                         # (N_L, 3)
    gap = abs(ends[:, None] - starts[None, :]).mean(-1)  # (N_L, N_L)
    loss_l1 = mean(gap * gt_adj)
    return loss_rel + 0.1 * loss_l1              # weight 0.1  [§4.2]

Hyperparameters

Values as stated in the paper unless marked "code". Where paper and code disagree, both are shown; see Paper vs code.

Results

Author-reported. Dataset OpenLane-V2, subsets A and B. Setup: 24 epochs unless stated, batch size 8 §4.2. Metric values as in the paper. The paper's TopoMLP rows marked * use extra YOLOv8 proposals tab. 1.

Subset A tab. 1

Subset B tab. 2

Ablations on subset A, all with ResNet-50 and 24 epochs tab. 3

Lane queries tab. 3a:

Control points tab. 3b:

LL topology tab. 3c:

LT topology tab. 3d:

The ablations are tied to a single backbone and one training run each; no variance is reported tab. 3.

Metric analysis tab. 4, §4.6. TopoNet and TopoMLP were trained by the authors (TopoNet reimplemented with the same ResNet-50 backbone). "Enhanced" means confidences were pushed to 1.0 or 0.0 around 0.5, which raises the original metric. The adjusted metric TOP† (eq. 8) lowers the enhanced score.

Figure-only results: the qualitative lane and traffic visualisations and the motivation study fig. 1, fig. 3, fig. 4 are described in §1 and §4.5 and are not reproduced here.

Paper vs code

Repository: wudongming97/TopoMLP, owned by the first author, linked from the abstract and the arXiv page. Commit faab5b6be99a951832a50923ebbd4a8adbf586d2. Code is Apache-2.0 LICENSE. The repo's configs cover ResNet-50, VoV and Swin-B on subset A, and ResNet-50 on subset B [code: projects/configs/]. The 48-epoch Swin-B run in Table 1 has no config here.

• Auxiliary one-to-many lane branch. Paper: 300 lanes, no extra branch §3.1, §4.2. Code: 300 one-to-one lanes plus 1500 one-to-many lanes (k = 5), with weight 2.0 on their loss, and an attention mask that stops the two groups from attending to each other topomlp_setA_r50_wo_yolov8.py lane_head.py lane_head.py. Topology heads use only the one-to-one outputs lane_head.py topomlp.py.
• Lane regression weight. Paper: 0.2 §4.2. Code: 0.02 topomlp_setA_r50_wo_yolov8.py. The assigner asserts that each loss weight equals its matching-cost weight (cls 1.5, L1 0.02), so the code's weight is the one that matches training lane_head.py.
• LT topology weight. Paper: 0.5 §4.2. Code: 5 topomlp_setA_r50_wo_yolov8.py.
• Lane regression target. Paper: "L1 loss for lane regression" §3.1. Code: L1 on 11 sampled points of the Bezier curve, not on control points lane_head.py lane_head.py.
• Lane-lane L1 target. Paper: L1 between matched lane points §3.5; the ablation calls it supervision of "intersection points" §4.4. Code: L1 between the last sampled point of lane i and the first of lane j, for connected pairs topo_ll_head.py.
• Gradient flow into topology heads. Paper: silent. Code: lane features and lane points are detached before the topology heads, so topology losses do not update the lane decoder or lane points topo_ll_head.py topo_lt_head.py topomlp_setA_r50_wo_yolov8.py topomlp_setA_r50_wo_yolov8.py.
• View matrix A. Paper: 3x3 matrix from intrinsics and extrinsics, added to the lane query before concatenation §3.4. Code: the upper-left 3x3 of the first entry of lidar2img, flattened to nine values, through a three-layer MLP, added to the lane features after the lane MLP topo_lt_head.py topo_lt_head.py. The code does not confirm which camera is index 0.
• YOLOv8 proposals. Paper: boxes are encoded with sine-cosine embedding and concatenated with random queries §3.2. Code: normalised centre-width-height boxes are concatenated directly with the learned reference embeddings as reference boxes; no sine-cosine step found in the traffic head traffic_head_yolov8.py. The YOLOv8 detections come from pre-computed pickle files topomlp_setA_r50_w_yolov8.py topomlp_setA_r50_w_yolov8.py.
• Metric. The paper's adjusted TOP† (eq. 8) is not implemented in this repo. The repo only builds the topology submission, after sorting lanes by an index openlane_v2_dataset_subset_a.py openlane_v2_dataset_subset_a.py. The scoring is in the external OpenLane-V2 devkit, which is not included here.
• Reproduced numbers. The README reports ResNet-50 subset A with devkit v1.0.0: DET_l 28.5, TOP_ll 7.1, DET_t 49.5, TOP_lt 23.4, OLS 38.3 README.md. The same checkpoint with devkit v2.1.0 gives OLS 44.1 README.md. Subset B with ResNet-50 gives OLS 38.0 under v1.0.0 README.md, against 38.7 in the paper tab. 2. The paper does not say which devkit it used.
• Pre-test filter. The test code keeps lanes where the sigmoid score is greater than 0.0, which keeps every query topomlp.py. Paper: no post-processing §4.2.
• Extra augmentation and schedule. Code: photometric distortion, GridMask on 70% of training images topomlp.py; warmup and minimum learning rate topomlp_setA_r50_wo_yolov8.py. Paper: HSV augmentation and grid mask §4.2; warmup not stated.

Pitfalls

• Missing training branch. An implementation that follows only the paper trains 300 lanes without the one-to-many branch. The code trains 1800 queries with an auxiliary loss, and the topology heads see only the 300 one-to-one lanes. Results depend on this, so the code's branch matters for reproduction topomlp_setA_r50_wo_yolov8.py.
• Ambiguous loss weights. Eq. 5 is an unweighted sum §3.5. The weights come from §4.2 and disagree with the code for the lane regression and LT terms. The §4.2 text gives the LT coefficient as "L_top_ll", a typo; read it as the LT term §4.2.
• Edge label. The lane-lane focal loss uses the inverted label (0 means connected). A reimplementation that uses 1 for connected will train the wrong sign topo_ll_head.py.
• Endpoint L1. The lane-lane L1 is direction-aware: it aligns the end of the first lane with the start of the second. A reimplementation based on "intersection points" will differ topo_ll_head.py.
• Gradient stop. The topology losses do not reach the lane decoder in the code. Expect the topology loss to act only on the heads if you copy this topo_ll_head.py.
• Pair-tensor cost. The lane-lane tensor is N_L x N_L x 2C. With N_L = 300 and C = 256, this is 300 x 300 x 512, about 46 million values per image before the MLP. The lane-traffic tensor with N_T = 100 is 300 x 100 x 512, about 15 million §3.3, §3.4, §4.2. These are our arithmetic from the stated sizes.
• Variable traffic count. The traffic decoder concatenates YOLOv8 boxes with the 100 learned queries. The number of boxes per image is not fixed, so N_T changes between frames traffic_head_yolov8.py. The paper does not say how many boxes are used.
• Threshold scale. The metric treats an edge as present above 0.5 §4.6. The code returns raw sigmoid scores and leaves the threshold out topo_ll_head.py. Apply the threshold where the metric is computed.
• Metric sensitivity. Table 4 shows OLS can rise by about 4 points from confidence changes alone tab. 4. Compare numbers only under the same devkit version; the README shows the same checkpoint at 38.3 and 44.1 under two versions README.md README.md.
• Position range. The 3D position embedding range is [-61.2, 61.2] m in X and Y and [-10, 10] m in Z. The paper states the BEV range [-51.2, 51.2] m and [-25.6, 25.6] m for X and Y, but not this embedding range topomlp_setA_r50_wo_yolov8.py §4.2.
• Assignment reuse. Topology losses reuse the detection matching §3.5. Only matched query pairs enter the topology loss; unmatched queries get no topology supervision at all, neither positive nor negative topo_ll_head.py.

Lineage

• Builds on: PETR and PETRv2 for 3D position embedding and query-based lanes §3.1, refs liu2022petr, liu2022petrv2; Deformable DETR for the traffic head §3.2, ref zhu2020deformable; the pairwise representation used in human-object interaction detection §1.
• Compared against: STSU, VectorMapNet, MapTR and TopoNet on OpenLane-V2 tab. 1, tab. 2. TopoNet models centerlines as graph vertices §2.2. TopoNet is reimplemented by the authors for tab. 4 tab. 4.
• Superseded by: not known from this wiki.