---
kind: concept
slug: bird-eye-view-perception
title: Bird's-eye-view perception
summary: >-
  Fill one top-down grid around the vehicle from cameras, LiDAR or shared agent features, then read map elements, 3D objects, lane links or off-road costs from it. Variants: HDMapNet, BEVFormer, V2X-ViT, TopoLogic, TopoFG, TopoPoint, SALON, multi-task BEV (2606.02979). Author-reported on different datasets.
tags: [autonomous-driving, bird-eye-view, sensor-fusion, mapping, 3d-object-detection, cooperative-perception, lane-topology, multi-task-learning, off-road-navigation]
papers: ["2107.06307", "2203.10638", "2203.17270", "2405.14747", "2511.12590", "2505.17771", "2504.01989", "2606.02979", "2412.07826"]
related: [bev-representation, object-detection, object-queries, semantic-segmentation, instance-segmentation, monocular-depth-estimation, collaborative-perception, graph-neural-network, bipartite-matching, multi-task-learning, multimodal-fusion, adaptive-loss-weighting, gaussian-process, foundation-model]
status: draft
updated_at: 2026-10-25
---

## Overview

A bird's-eye-view (BEV) model turns what the vehicle senses, and what connected
agents share, into one top-down grid centred on the vehicle, then reads its
outputs from that grid. The grid is the shared interface. The cards differ
in how the grid is filled and in what is read from it [arxiv:2107.06307 §3.1]
[arxiv:2203.17270 §3.1] [arxiv:2203.10638 §3.1] [arxiv:2405.14747 §3.2].

- **Per-camera lift.** Each camera's image features reach the grid through a
  learned per-camera network and the camera extrinsics. HDMapNet uses this and
  outputs vectorized lane, boundary and crossing elements [arxiv:2107.06307 §3.1].
- **Query-based attention.** A fixed set of learnable grid queries samples the
  camera features where each query's 3D points project, and reads the previous
  grid after ego-motion alignment. BEVFormer uses this and outputs 3D boxes and
  map segmentation [arxiv:2203.17270 §3.3] [arxiv:2203.17270 §3.4].
- **Cross-agent fusion.** Connected vehicles and roadside units each send a
  LiDAR feature map. The ego vehicle warps every received map into its own frame
  and fuses them with a Transformer. V2X-ViT uses this and outputs 3D vehicle
  boxes [arxiv:2203.10638 §3.1] [arxiv:2203.10638 §3.2].
- **Lane topology on a BEV encoder.** A BEVFormer-style view transformer fills
  the grid, and a lane decoder reads lane centrelines and the connections
  between them. TopoLogic uses this and outputs which lane connects to which at
  intersections [arxiv:2405.14747 §3.2] [arxiv:2405.14747 §1].
- **Fine-grained lane queries.** Each lane gets one query per ordered point,
  seeded from a predicted BEV lane mask and point-order embeddings. Only the
  first and last query of each lane feed a pairwise connection head. TopoFG uses
  this and outputs the same kind of connection matrix as TopoLogic, with a
  learnt score plus TopoLogic's geometric term [arxiv:2511.12590 §3.2]
  [arxiv:2511.12590 §3.4].
- **Endpoint detection on a query-based lane model.** Lane endpoints get their
  own queries, which share attention and graph layers with the lane queries.
  At inference the detected endpoints pull nearby lane ends together. TopoPoint
  uses this and outputs lane-lane, point-lane and lane-traffic connection
  matrices [arxiv:2505.17771 §3.4] [arxiv:2505.17771 §3.7]
  [arxiv:2505.17771 eq. 18-20].
- **Top-view LiDAR input inside a multi-task network.** LiDAR is already encoded
  as a fixed top-view stack of height bins. A BEV decoder reads it together with
  the LiDAR segmentation output, next to depth and segmentation heads for four
  cameras. The network is small and trained with adaptive loss weights.
  Natan and Miura use this and output a BEV map with the other three tasks
  [arxiv:2606.02979 §3.1] [arxiv:2606.02979 §4.3].
- **Costmap from foundation-model features.** Dense features from a visual
  foundation model are reduced to a few descriptors and projected onto the grid
  with lidar points. A Gaussian process then reads roughness and a speed limit
  for each cell, instead of objects or lanes. SALON uses this and outputs a
  traversability cost map and a speed map [arxiv:2412.07826 §3.1.3]
  [arxiv:2412.07826 §3.4].

A survey of lane topology describes the same onboard pipeline: a backbone, a
perspective transform into bird's-eye view, a centreline decoder and a topology
decoder. It calls onboard methods the current state of the art
[arxiv:2504.01989 §2.3] [arxiv:2504.01989 §1].

The hard part for cameras is depth. A single image does not say how far away a
pixel lies. The per-camera lift has to learn where image content falls on the
ground plane [arxiv:2107.06307 §1]. BEVFormer does not predict depth. It samples
features at the projections of a pillar of assumed heights above each cell
[arxiv:2203.17270 §3.3]. LiDAR does not face the problem: its points go into
pillars directly [arxiv:2107.06307 §3.1].

For cross-agent fusion the hard part is the link between vehicles. GPS errors
corrupt the coordinate transforms, and features arrive late. The authors model
both as noise and delay during training [arxiv:2203.10638 §1] [arxiv:2203.10638 §3].

The cards measure different things. HDMapNet reports IoU and AP for map
elements. BEVFormer reports NDS and mAP for 3D detection, plus BEV IoU for
segmentation. V2X-ViT reports AP at IoU 0.5 and 0.7 for vehicles, on a simulated
dataset. Their numbers cannot be compared with each other, except that TopoLogic
and TopoFG share the OpenLane-V2 setup.

## Variants

### Per-camera lift, cameras only (HDMapNet, HDMapNet(Surr))

Each of the six cameras is embedded by one shared image encoder. A learned
per-camera MLP maps every top-down cell from the perspective pixels, and the
camera extrinsics move the result into the ego frame [arxiv:2107.06307 §3.1]
[arxiv:2107.06307 eq. 1]. The paper says the camera features are averaged; the
official code takes an element-wise max [code: model/homography.py#L292].

Under the same encoder and decoder, the paper's camera-only baselines differ
only in their view transform. HDMapNet(Surr) scores 32.9 IoU and 22.7 mAP,
against 32.4 IoU and 19.7 mAP for IPM(CB), 30.8 and 17.4 for Lift-Splat-Shoot
(named, no card), and 29.3 and 17.5 for VPN [arxiv:2107.06307 §4.2]
[arxiv:2107.06307 tab. I] [arxiv:2107.06307 tab. II]. Against IPM(CB) the IoU
gap is small (0.5 points), while the mAP gap is 3.0 points.

Temporal stacking helps this variant. Fusing 1, 2 and 4 frames by max pooling
of ego-aligned features gives 32.9, 35.8 and 36.4 IoU. Only the camera-only
model was tested this way [arxiv:2107.06307 tab. IV].

### Query-based attention, cameras only (BEVFormer)

BEVFormer keeps one learnable query per grid cell (200 × 200 on nuScenes) and
updates them through six encoder layers. Each layer has three parts: temporal
self-attention over the current queries and the previous BEV, spatial
cross-attention over the six camera feature maps, and a feed-forward block
[arxiv:2203.17270 §3.1] [arxiv:2203.17270 §4.2].

- **Spatial cross-attention.** A query takes a pillar of four heights above its
  cell, projects each 3D point into every camera, and samples image features
  around the hits with deformable attention. It averages over the cameras that
  were hit [arxiv:2203.17270 §3.3] [arxiv:2203.17270 eq. 2]
  [arxiv:2203.17270 eq. 4]. The deformable attention is adapted from Deformable
  DETR, which is named here without a card [arxiv:2203.17270 §2].
- **Temporal self-attention.** The previous BEV is moved to the current ego
  frame, then each query attends to the current and the aligned previous grid
  [arxiv:2203.17270 §3.4] [arxiv:2203.17270 eq. 5]. On the first frame of a
  sequence the history is a copy of the current queries.

The single-frame model (BEVFormer-S) scores 44.8 NDS on nuScenes val, against
51.7 for the four-frame model, both with ResNet-101 [arxiv:2203.17270 tab. 2].
BEVFormer-S is the paper's static variant, so the gap comes from the temporal
history [arxiv:2203.17270 §4.2]. The gain levels off after four frames, with
five giving the same 51.7 [arxiv:2203.17270 tab. 7].

### LiDAR pillars (HDMapNet, HDMapNet(LiDAR))

Points are grouped into pillars, a PointNet with a 64-dimensional layer encodes
each pillar, and a convolutional network turns the pillars into a BEV map
[arxiv:2107.06307 §3.1] [arxiv:2107.06307 §4.1]. The LiDAR-only model is better
than the camera-only model on boundaries (44.6 against 39.5 IoU) and worse on
dividers (26.7 against 40.6) and crossings (17.3 against 18.7)
[arxiv:2107.06307 tab. I]. Its mAP is the lowest of the three (11.6)
[arxiv:2107.06307 tab. II].

### Camera-LiDAR fusion (HDMapNet, HDMapNet(Fusion))

The image BEV and LiDAR BEV features are concatenated along the channel axis
after upsampling. The official code has no fusion weights or gates
[code: model/hdmapnet.py#L94-L97]. This variant is best on every class in the
paper's tables [arxiv:2107.06307 tab. I] [arxiv:2107.06307 tab. II]. The paper
reports IoU by weather for this variant only: 44.9 in normal conditions, 39.3
at night and 38.7 in rain [arxiv:2107.06307 tab. III]. The authors say
performance drops in extreme weather but stays reasonable [arxiv:2107.06307 §4.3].

### Cross-agent fusion, LiDAR from vehicles and roadside units (V2X-ViT)

Each agent turns its LiDAR into a BEV feature map with a PointPillar encoder,
and a compression step is applied before sending [arxiv:2203.10638 §3.1]. The
ego vehicle warps each received map from its capture pose to its own pose, then
runs three Transformer layers. Each layer applies heterogeneous multi-agent
self-attention, which keeps vehicles and infrastructure as separate node types,
then multi-scale window attention, then an MLP [arxiv:2203.10638 §3.2]
[arxiv:2203.10638 §4.2]. A time-delay encoding is added to each agent's features
to handle transmission delay [arxiv:2203.10638 §3.2.3].

The paper's default compression rate is 32, but the released config turns
compression off [arxiv:2203.10638 §4.2]
[code: v2xvit/hypes_yaml/point_pillar_v2xvit.yaml#L84].

### Lane topology on a BEV encoder (TopoLogic)

TopoLogic uses a BEVFormer-style view transformer over the surround cameras
(seven views on subset_A, six on subset_B) [arxiv:2405.14747 §3.2]
[arxiv:2405.14747 §4.1]. Its lane decoder predicts which lane ends where
another begins. Two matrices are fused with two learnable scalars. One is a
score from the gap between the end of lane i and the start of lane j, passed
through a learnable mapping. The other is a sigmoid of the dot product of two
MLP encodings of the lane queries [arxiv:2405.14747 §3.3]
[arxiv:2405.14747 §3.4] [arxiv:2405.14747 §3.5]. The fused topology then feeds
a graph layer that mixes each lane's features with those of its connected
lanes [arxiv:2405.14747 §3.2].

The paper's ablation on subset_A separates the two sources. An MLP on the
queries gives TOP_ll 10.8, similarity alone 12.9, geometry alone 20.1 and
both 23.9 [arxiv:2405.14747 tab. 4]. The learnable mapping beats fixed
Gaussian, sigmoid and tanh forms on the same metric [arxiv:2405.14747 tab. 3].
The geometric step also works as post-processing on an already trained model,
without retraining [arxiv:2405.14747 tab. 5]. The card records that the
released post-processing path does not read the learned values (see
Contradictions).

### Lane topology with fine-grained queries (TopoFG)

TopoFG builds on TopoLogic [arxiv:2511.12590 §3.4]. It changes what a lane is in
the decoder. Each lane gets 11 queries, one per ordered point, instead of one.
Each query starts from two priors. The spatial prior is a predicted BEV lane
mask, from a Mask2Former-style head over learnable lane queries. Cells with mask
probability above tau = 0.3 get the constant alpha = 1.0. The sequential prior is
a learnable point-order embedding [arxiv:2511.12590 §3.2] [arxiv:2511.12590 eq.
1-4] [arxiv:2511.12590 §4.2]. The deformable attention in its six decoder layers
takes reference points from the predicted mask [arxiv:2511.12590 §3.3].

Connections use only the first and last query of each lane. A shared MLP scores
each ordered pair, the end of lane i with the start of lane j [arxiv:2511.12590
§3.4] [arxiv:2511.12590 eq. 6-7]. At inference the learnt score is added to a
geometric term that follows TopoLogic's mapping [arxiv:2511.12590 §3.4]. Training
uses Hungarian matching plus five noisy copies of each ground-truth lane. Their
connection targets stay fixed, so the labels do not move with the matching
[arxiv:2511.12590 §3.4] [arxiv:2511.12590 §4.2].

On subset_A the paper adds the three modules to TopoLogic one at a time. The
spatial and sequential priors reach OLS 45.4, the reference sampling 45.8, and
the full boundary-point module 48.0 [arxiv:2511.12590 tab. 3]. Within the last
module, the denoising group alone reaches 47.3 and boundary-point reasoning alone
46.6, both over the 45.8 before them [arxiv:2511.12590 tab. 6]. The paper reports
one number per row, with no seeds or spread [arxiv:2511.12590 tab. 6]. The
linked repository has no LICENSE file, so this page quotes none of its code.

### Endpoint detection with point-lane matching (TopoPoint)

TopoPoint keeps the lane decoder of a TopoLogic-style model and adds a second
query set for lane endpoints. Each point query regresses one 3D endpoint, and
the lane queries regress 11-point lanes [arxiv:2505.17771 §3.4]
[arxiv:2505.17771 §4.2]. Points and lanes first share one self-attention. The
attention bias comes from distances between lane ends and starts, and between
points and lanes, passed through the same mapping TopoLogic uses
[arxiv:2505.17771 §3.4] [arxiv:2505.17771 eq. 3-8]. A point-lane graph layer
then passes messages in both directions [arxiv:2505.17771 eq. 11-12].

At inference the paper keeps confident points and lanes. Each kept point is
averaged with the kept lane ends within 1.5 m, and every matched lane takes the
average as its endpoint [arxiv:2505.17771 §3.7] [arxiv:2505.17771 alg. 1].

The ablation on subset_A starts from a TopoLogic baseline that the authors re-ran.
Adding the merged self-attention on top of the native front view raises TOP_ll
from 23.8 to 27.2 and OLS from 46.0 to 47.6. The graph layers and the point matching bring OLS to 48.8
[arxiv:2505.17771 tab. 2]. The largest single step is the native-scale front
view, which takes OLS from 43.4 to 46.0. In the scale table, native front-view
scale raises DET_t from 48.6 to 55.3 [arxiv:2505.17771 tab. 4]. The paper does
not state the compute cost of the native scale.

The released code does not run the point-to-lane matching as the paper
describes it (see Contradictions). The point-matching gain in the table cannot be
checked against the released code without rebuilding that step
[arxiv:2505.17771 §3.7] [code: projects/topopoint/core/lane/coders/lane_coder.py#L90-L201].

### Multi-task BEV map from LiDAR height stack (compact network)

This variant does no view transform. LiDAR points are binned into a fixed
15-channel top-view tensor: 14 height bands from −2 m to 11 m, and one flat
top-view channel, at 128 × 128 [arxiv:2606.02979 §4.3] [arxiv:2606.02979 eq. 22]
[arxiv:2606.02979 fig. 6]. The LiDAR encoder feeds a LiDAR segmentation decoder,
and that output feeds the BEV decoder [arxiv:2606.02979 §3.1]. The BEV decoder
is one of four outputs. The other three are depth for four views, which takes
DVS input, segmentation for four views, which takes RGB input, and the LiDAR
segmentation itself [arxiv:2606.02979 §3.1] [arxiv:2606.02979 §4.3].

The paper's BEV output has 9 classes on the CARLA sets [arxiv:2606.02979 tab. 1]
and 32 on nuScenes-lidarseg [arxiv:2606.02979 §4.2]. The nuScenes variant has no
DVS input, because that dataset has none [arxiv:2606.02979 tab. 2].

Training weights the four task losses with a modified GradNorm. The weights
are updated once per epoch, using the gradient norms on one chosen layer per
bottleneck [arxiv:2606.02979 §3.3] [arxiv:2606.02979 alg. 1]. The card gives
no training-time cost for this step. Inference speed, averaged over the three
simulated sets on a GTX 1080 Ti, is 56.0 FPS for 15L and 54.4 FPS for 15L+MGN
[arxiv:2606.02979 tab. 3].

### Costmap from foundation-model features (SALON)

SALON does not output objects or lanes. Its BEV grid holds features from a
visual foundation model, DINOv2 on the ATV [arxiv:2412.07826 §4.1.1]. The
features are reduced to a k-dimensional descriptor: each pixel gets its L1
distance to each of k k-means centres [arxiv:2412.07826 §3.1.2] [eq. 2]. They
are matched to lidar points with known calibration, projected into the grid and
smoothed over time [arxiv:2412.07826 §3.1.3]. A buffer keeps samples of
(feature, speed, roughness) under the tire, with roughness from bandpower of
the accelerations and suspension travel [arxiv:2412.07826 §3.2]
[arxiv:2412.07826 §3.3].

A Gaussian process with an RBF kernel reads mean and variance of roughness and
speed for each cell. A CVaR-style value turns them into a risk-aware cost and
speed limit [arxiv:2412.07826 §3.4] [arxiv:2412.07826 eq. 3-5]. One user click
on a lethal object links that feature to high roughness, so similar features
get high cost without retraining [arxiv:2412.07826 §3.5]. Cells far from every
cluster centre are marked lethal [arxiv:2412.07826 §3.5.1]. The roughness fit
needs a 0-to-1 score from a passenger during a short drive [arxiv:2412.07826 §3.2].

The paper's implementation details differ in places from the official code,
see Contradictions. Card: [SALON: Self-supervised Adaptive Learning for Off-road Navigation](/papers/2412.07826).

## Results

Numbers are author-reported. The HDMapNet and BEVFormer numbers are on nuScenes.
The V2X-ViT numbers are on the authors' simulated V2XSet. The TopoLogic, TopoFG
and TopoPoint numbers are on OpenLane-V2, so they cannot be set against the nuScenes numbers. The
SALON numbers come from one ATV course with one safety driver, and cannot be set
against any of them. Each table is for one task only.

**HDMapNet, map elements** [arxiv:2107.06307 tab. I] [arxiv:2107.06307 tab. II]

| Input | All IoU | Divider IoU | Crossing IoU | Boundary IoU | All mAP |
|---|---|---|---|---|---|
| Cameras, HDMapNet(Surr) | 32.9 | 40.6 | 18.7 | 39.5 | 22.7 |
| LiDAR, HDMapNet(LiDAR) | 29.5 | 26.7 | 17.3 | 44.6 | 11.6 |
| Cameras + LiDAR, HDMapNet(Fusion) | 44.5 | 46.1 | 31.4 | 56.0 | 30.6 |

**BEVFormer, 3D detection, nuScenes val** [arxiv:2203.17270 tab. 2]

| Method | Backbone | NDS ↑ | mAP ↑ | mAVE ↓ |
|---|---|---|---|---|
| FCOS3D | R101 | 0.415 | 0.343 | 1.292 |
| DETR3D | R101 | 0.425 | 0.346 | 0.842 |
| BEVFormer-S (single frame) | R101 | 0.448 | 0.375 | 0.802 |
| BEVFormer | R101 | 0.517 | 0.416 | 0.394 |

- On nuScenes test, the ResNet-101 BEVFormer reaches 53.5 NDS. The 56.9 NDS
  row uses a VoVNet-99 backbone that was pre-trained on depth estimation with
  extra data, so it is not like-for-like [arxiv:2203.17270 tab. 1]
  [arxiv:2203.17270 tab. 1 caption].
- Velocity error falls with the temporal layer: test mAVE goes from 0.925
  (single frame) to 0.435 [arxiv:2203.17270 tab. 1].
- **Segmentation, nuScenes val, BEV IoU.** The segmentation-only BEVFormer
  scores 80.1 on road and 25.7 on lane. Lift-Splat with the same heads scores
  77.7 and 20.0 [arxiv:2203.17270 tab. 4]. These are BEV mask IoUs, not the
  vector metrics above.
- **Waymo, vehicles.** The front-camera-only build scores above the six-camera
  build on APH at 3D IoU 0.5, level 1 (0.308 against 0.280). This is one metric
  on one split, and the card does not explain it [arxiv:2203.17270 tab. 3].
- **Occlusion.** At 0 to 40% visibility, recall beats BEVFormer-S and DETR3D by
  more than 6 points [arxiv:2203.17270 §4.5] [arxiv:2203.17270 fig. 3].

**V2X-ViT, 3D vehicle detection, V2XSet, AP** [arxiv:2203.10638 tab. 1]

| Model | Perfect AP@0.5 | Perfect AP@0.7 | Noisy AP@0.5 | Noisy AP@0.7 |
|---|---|---|---|---|
| No Fusion | 0.606 | 0.402 | 0.606 | 0.402 |
| Early Fusion | 0.819 | 0.710 | 0.720 | 0.384 |
| DiscoNet | 0.844 | 0.695 | 0.798 | 0.541 |
| V2X-ViT | 0.882 | 0.712 | 0.836 | 0.614 |

- Under the noisy setting, V2X-ViT is 23.0 AP points above no fusion at AP@0.5
  and 21.2 points at AP@0.7. Under the perfect setting it is 3.8 and 1.7 points
  above the best intermediate baseline [arxiv:2203.10638 §4.3].
- Delay-aware encoding raises noisy AP@0.7 from 0.601 to 0.614 in the full model
  [arxiv:2203.10638 tab. 2].
- Inference on a Tesla V100 takes 57 ms for the full model and 28 ms for the
  one-layer variant. Their noisy AP@0.7 is 0.614 and 0.591 [arxiv:2203.10638 tab. 4].
- The dataset is simulated (CARLA and OpenCDA). The authors note bias and
  generalization issues for real-world use [arxiv:2203.10638 §5].

**Lane topology, OpenLane-V2 subset_A, centreline, no SDMap, v2.1.0**
[arxiv:2405.14747 tab. 1]

| Method | DET_l | TOP_ll | TOP_lt | OLS |
|---|---|---|---|---|
| TopoNet | 28.6 | 10.9 | 23.8 | 39.8 |
| TopoLogic | 29.9 | 23.9 | 25.4 | 44.1 |
| TopoFG | 33.8 | 30.8 | 30.9 | 48.0 |
| TopoPoint | 31.4 | 28.7 | 30.0 | 48.8 |

- TopoFG's table lists the TopoNet and TopoLogic rows above with the same
  numbers [arxiv:2511.12590 tab. 1]. Its subset_A row gains 3.9 OLS, 6.9 TOP_ll
  and 4.0 TOP_lt over TopoLogic. It also reports subset_B at 45.4 OLS, against
  42.3 for TopoLogic [arxiv:2511.12590 tab. 1]. These are the authors' numbers.
- TopoPoint's row is from its own subset_A table, scored with the OpenLane-V2
  v2.1.0 devkit [arxiv:2505.17771 tab. 1] [arxiv:2505.17771 §4.1].
  Its table has no TopoFG row, so its 48.8 and TopoFG's 48.0 are two author
  claims, not one comparison. The card does not state a map-prior setting.
  On subset_B it reports OLS 49.2 and DET_p 45.1, against 42.3 and 39.2 for
  TopoLogic [arxiv:2505.17771 tab. 1].

**Multi-task BEV map, BEV IoU, test sets** [arxiv:2606.02979 tab. 2]

| Dataset | Chen et al., single-task BEV | 15L+MGN (multi-task) | 15L (no adaptive weights) | 1L (one LiDAR channel) |
|---|---|---|---|---|
| CARLA set A | 0.654 | 0.594 | 0.575 | 0.613 |
| CARLA set B | 0.567 | 0.630 | 0.603 | 0.589 |
| CARLA set C | 0.637 | 0.630 | 0.602 | 0.606 |
| nuScenes-lidarseg | 0.788 | 0.833 | 0.824 | 0.804 |

- The multi-task model beats the single-task BEV baseline on set B and on
  nuScenes-lidarseg, and trails it on sets A and C. The paper's headline is a
  total metric over four tasks, not BEV alone: 15L+MGN has a total metric of
  1.393, 1.086 and 1.069 on sets A, B and nuScenes-lidarseg, against 1.430,
  1.160 and 1.124 for the combined single-task baselines. On set C it is 0.999
  against 0.976 [arxiv:2606.02979 tab. 2] [arxiv:2606.02979 §5.4].
- Adaptive weights lower the total metric against 15L on all four test sets.
  The 15L model has a depth error no higher, and a semantic-segmentation IoU
  higher, than 15L+MGN on every set. The authors call this a trade-off between
  the two groups of tasks [arxiv:2606.02979 tab. 2] [arxiv:2606.02979 §5.2].
- Model size, simulated sets: 2,521,504 parameters against 136,728,752 for the
  combined baselines, under 2% [arxiv:2606.02979 tab. 3] [arxiv:2606.02979 §5.4].
- The ± values in this paper are variances of per-batch scores, not standard
  errors [arxiv:2606.02979 tab. 2] [code: predict.py#L222-L233]. The BEV IoU
  here is on the paper's own CARLA and nuScenes-lidarseg splits, not on
  nuScenes map elements, so it cannot be set against the rows above.

**SALON, ATV off-road navigation, figure-8 with five laps** [arxiv:2412.07826 tab. 4]

- The table gives per-lap counts. At R_max 0.2, SALON has 3 interventions and
  2 undesirable behaviours across the five laps. The geometric cost has 23 and
  7, and Velociraptor has 4 and 8. The totals are sums the card computed from
  the per-lap values, not the authors' numbers [arxiv:2412.07826 tab. 4].
- Velociraptor is faster, at 4.82 m/s average against 3.60 for SALON at R_max
  0.2. The authors accept that trade-off because they weight interventions more
  heavily [arxiv:2412.07826 §4.2.3]. At R_max 0.4, SALON's interventions tie
  Velociraptor at 4 [arxiv:2412.07826 tab. 4].
- Removing the most common class and speed from the buffer, instead of FIFO,
  raises average pairwise buffer distance from 3.47 to 4.79 in scenario 1
  [arxiv:2412.07826 tab. 1].

## When to use which

The choice depends on the sensors on the vehicle, the output the downstream
code needs, and the compute budget. Each row cites only the numbers the paper
gives for that setup.

| Situation | Use | Why |
|---|---|---|
| Map elements, cameras only | Per-camera lift (Surr) | 32.9 IoU and 22.7 mAP, the best camera-only map result in the paper [arxiv:2107.06307 tab. I] [arxiv:2107.06307 tab. II] |
| Map elements, cameras and LiDAR both present | Fusion | 44.5 IoU and 30.6 mAP, best on every class [arxiv:2107.06307 tab. I] [arxiv:2107.06307 tab. II] |
| Map elements, LiDAR only, boundaries matter most | LiDAR pillars | Boundary IoU 44.6, against 39.5 camera-only. Dividers and crossings are worse [arxiv:2107.06307 tab. I] |
| Map elements, dividers or crossings matter most, LiDAR present | Fusion | Divider IoU 46.1 for fusion, 40.6 for cameras, 26.7 for LiDAR [arxiv:2107.06307 tab. I] |
| 3D boxes and velocity from a camera-only rig | Query-based attention | 0.517 NDS and 0.394 mAVE on val, against 0.425 NDS for DETR3D with the same backbone [arxiv:2203.17270 tab. 2] |
| Camera-only, several past frames available | Query-based, with temporal layer | 51.7 NDS with four frames, against 44.8 for one frame [arxiv:2203.17270 tab. 2] [arxiv:2203.17270 tab. 7] |
| Tight latency budget | Fewer layers or a 100 × 100 grid | Default BEV encoder takes 130 ms per frame on a V100. One layer at 200 × 200 takes 25 ms for 0.501 NDS. One layer at 100 × 100 takes 7 ms for 0.478 NDS [arxiv:2203.17270 tab. 6] |
| Memory-limited training | Local deformable attention | The global-attention row uses about 36 GB against about 20 GB. The global row also uses single-scale features and a 100 × 100 grid, so the gap mixes both settings [arxiv:2203.17270 tab. 5] |
| Noisy camera calibration | Plan for about 14% NDS loss | At noise level 4, BEVFormer NDS drops by 14.3%. A model trained with noisy extrinsics drops by 8.9% [arxiv:2203.17270 app. B] [arxiv:2203.17270 fig. 6] |
| Road or lane masks, not vectors | Query-based segmentation head | Segmentation-only BEV IoU of 80.1 on road and 25.7 on lane [arxiv:2203.17270 tab. 4]. Not comparable to HDMapNet's vector metrics |
| Several connected vehicles or roadside units with LiDAR, and 3D boxes needed | Cross-agent fusion (V2X-ViT) | Noisy AP@0.7 of 0.614, against 0.402 with no fusion, on the simulated V2XSet [arxiv:2203.10638 tab. 1] |
| Localization noise and transmission delay expected | Cross-agent fusion with delay encoding | Delay encoding adds 0.011 to 0.020 AP@0.7 across 100 to 400 ms delays in the noisy setting [arxiv:2203.10638 tab. 3] |
| Tight latency budget, cross-agent | One-layer variant (V2X-ViT-S) | 28 ms against 57 ms on a V100, with noisy AP@0.7 of 0.591 against 0.614 [arxiv:2203.10638 tab. 4] |
| Lane connections at intersections needed, from cameras, on OpenLane-V2-style lanes | Lane topology with geometric and similarity fusion (TopoLogic) | TOP_ll 23.9 against 10.9 for TopoNet on subset_A. Geometry alone gives 20.1, against 10.8 for an MLP on the queries [arxiv:2405.14747 tab. 1] [arxiv:2405.14747 tab. 4] |
| Lane ends decide the connections, from cameras, on OpenLane-V2-style lanes, and a fine-grained lane representation is wanted | Fine-grained lane queries (TopoFG) | OLS 48.0 against 44.1 for TopoLogic, and TOP_ll 30.8 against 23.9 [arxiv:2511.12590 tab. 1]. It adds a mask head and denoising groups in training, and the paper gives no runtime or memory [arxiv:2511.12590 §3.2] [arxiv:2511.12590 §3.4] |
| Endpoints of connected lanes miss each other, from cameras, on OpenLane-V2-style lanes, and an endpoint score is wanted | Endpoint detection (TopoPoint) | OLS 48.8 and DET_p 52.6 on subset_A, against 44.1 and 45.2 for TopoLogic [arxiv:2505.17771 tab. 1]. The point-lane matching is not active in the released code, see Contradictions [code: projects/topopoint/core/lane/coders/lane_coder.py#L90-L201] |
| Trained lane model already in place, no retraining wanted | Geometric post-processing step | TopoNet TOP_ll goes from 10.9 to 22.3 without retraining. The card reports this for three models, and the repository's code covers only TopoLogic's own head [arxiv:2405.14747 tab. 5] |
| Depth, segmentation and a BEV map from one network, with a small model and one forward pass, and LiDAR present | Multi-task BEV map (compact network) | 2.5M parameters, against 136.7M for the combined single-task baselines, with a total metric below them on three of four sets. BEV IoU is below the single-task baseline on set A (0.594 against 0.654) [arxiv:2606.02979 tab. 2] [arxiv:2606.02979 tab. 3] |
| Off-road ground, lidar present, roughness learned online from the vehicle's own driving, with a one-click fix for lethal objects | Costmap from foundation-model features (SALON) | Over five laps at R_max 0.2, 3 interventions against 4 for Velociraptor, with a lower average speed (3.60 against 4.82) [arxiv:2412.07826 tab. 4]. Needs a roughness signal from the vehicle and passenger scores for fitting [arxiv:2412.07826 §3.2]. One course, one driver, no variance reported |
| Several tasks trained together, loss weights hard to set by hand | Adaptive loss weighting (modified GradNorm, once per epoch) | Total metric lower than fixed weights on all four test sets, for example 1.393 against 1.448 on set A. The ± values are per-batch variances, not seed spread [arxiv:2606.02979 tab. 2] [arxiv:2606.02979 §3.3] |

Neither paper gives memory or runtime for the other's variants, so the cost of
the lift-based and query-based designs cannot be compared from the cards
[arxiv:2107.06307 §4.1]. Query-based inference is stateful: frames have to
arrive in time order, and the history resets when the scene changes
[code: projects/mmdet3d_plugin/bevformer/detectors/bevformer.py#L243-L251].

## Implementation sketch

The variants share the same output: a top-down grid that a decoder reads. They
differ in how the grid is filled. The sketches follow the cards' pseudocode,
written in our own words.

Lift-based (HDMapNet), common core and fusion step:

```pseudocode
# Encode each camera's image  [arxiv:2107.06307 §3.1]
# Input: N cameras, each (3, H, W). Output: N perspective maps (K, h, w).
feats = [image_encoder(img) for img in cameras]

# Lift each camera to the top-down grid with a per-camera MLP  [eq. 1]
# Output: N maps (K, Hb, Wb), one per camera.
bev_cam = [view_mlp[i](feats[i]) for i in range(N)]

# Move each camera's grid into the ego frame via extrinsics  [§3.1]
# Aggregation across cameras: average (paper) or max (official code)
bev_img = aggregate([warp(b, extrinsics[i]) for i, b in enumerate(bev_cam)])

# Encode LiDAR points as pillars, then as a BEV map  [§3.1] [§4.1]
bev_lidar = pillar_encoder(points)    # PointNet width 64

# Fusion: concatenate channels after upsampling, or use one branch alone
bev = concat([bev_img, bev_lidar])    # Fusion variant

# Fully convolutional decoder with three heads  [§3.1]
seg = seg_head(bev)                   # per-cell class
emb = emb_head(bev)                   # instance embedding
dirn = dir_head(bev)                  # lane direction, discretized
```

Query-based (BEVFormer), one timestamp:

```pseudocode
# BEVFormer encoder, one timestamp  [arxiv:2203.17270 §3.1]
# Input: images (6, H, W, 3); prev_bev (cells, C) or None
# Output: bev (cells, C)

def bevformer_encode(images, prev_bev, ego_shift, ego_yaw_delta):
    features = fpn(backbone(images))
    queries = bev_queries + bev_positional_embedding  # [§3.2]
    # Move the history into the current ego frame  [§3.4]
    if prev_bev is not None:
        prev_bev = shift_and_rotate(prev_bev, ego_shift, ego_yaw_delta)
    for layer in range(6):
        # Temporal self-attention over current and aligned history  [§3.4]
        queries = temporal_self_attention(queries, prev_bev)
        # Spatial cross-attention into the camera features  [§3.3]
        queries = spatial_cross_attention(queries, features)
        queries = feed_forward(queries)                # [§3.1]
    return queries
```

TopoLogic's view transformer is BEVFormer-style [arxiv:2405.14747 §3.2], so the
query-based encoder above applies to it. Its topology step runs on the lane
queries after each decoder layer [arxiv:2405.14747 §3.3]. The card has the
pseudocode for it.

Vectorization at inference is specific to HDMapNet. The class map is clustered
with DBSCAN on the instance embeddings, redundant cells are suppressed, and the
rest are connected into ordered lines using the predicted directions
[arxiv:2107.06307 §3.1] [code: postprocess/cluster.py#L159]
[code: postprocess/connect.py#L87-L94]. BEVFormer's detection head instead
outputs boxes directly [arxiv:2203.17270 app. A.3].

Lane topology with fine-grained queries (TopoFG), pairwise head and inference
adjacency:

```pseudocode
# Pairwise connectivity from fine-grained queries  [arxiv:2511.12590 §3.4]
# Input: queries (N, k, d), k = 11 points per lane
# Output: topology logits (N, N), one score per ordered pair (i, j)

def connectivity_logits(queries):
    # Keep only the first and last query of each lane  [§3.4]
    start = queries[:, 0]                          # (N, d)
    end = queries[:, -1]                           # (N, d)
    # Pair the end of lane i with the start of lane j  [eq. 6]
    pairs = concat(end[:, None], start[None])      # (N, N, 2d), broadcast
    # One shared MLP scores every ordered pair  [eq. 7]
    return mlp(pairs)                              # (N, N)

# Inference adjacency: learnt score plus a geometric term  [§3.4] [code]
# The code measures the gap with an L1 distance; the paper says Euclidean
def adjacency(logits, lanes):
    # Gap from the end of lane i to the start of lane j
    gap = abs(lanes[:, -1][:, None] - lanes[:, 0][None]).sum(-1)   # (N, N)
    # TopoLogic-style mapping; 11.5275 is a code value, not in the paper
    geometric = exp(-gap ** 2 / 11.5275) * (1 - eye(N))
    return sigmoid(logits) + geometric
```

Cross-agent fusion (V2X-ViT), ego side:

```pseudocode
# Warp each agent's BEV map into the ego frame; mark padded pixels
# [arxiv:2203.10638 §3.1] [app. B.1]
# Input: maps[i] (H, W, C) per agent i, with poses and delays per agent
feats, masks = warp_to_ego(maps, poses)

# Add a delay encoding to each agent's features  [§3.2.3] [eq. 10]
feats = [add_delay_encoding(f, d) for f, d in zip(feats, delays)]

# Three encoder layers: attention across agents, then within windows  [§4.2]
for _ in range(3):
    # Attention across agents at each cell, by node type  [§3.2.1]
    feats = hmsa(feats, agent_types, masks)
    # Window attention with windows of 4, 8 and 16  [§3.2.2]
    feats = mswin(feats)
    feats = mlp(feats)

boxes, scores = detection_head(feats[ego])    # [§3.1]
```

## Contradictions

Between papers: TopoPoint re-ran the TopoLogic code for its baseline and got
slightly lower scores than TopoLogic reports on subset_A. TopoLogic gives OLS
44.1, DET_l 29.9 and TOP_ll 23.9 [arxiv:2405.14747 tab. 1]. TopoPoint's baseline
gives OLS 43.4, DET_l 29.2 and TOP_ll 23.4 [arxiv:2505.17771 tab. 2]. TopoFG's
table repeats the TopoNet and TopoLogic rows with TopoLogic's numbers
[arxiv:2405.14747 tab. 1] [arxiv:2511.12590 tab. 1], so that is not a conflict.
The other disagreements below are inside one paper, between its text, tables
and official code.

- **HDMapNet camera aggregation.** The paper says the average over cameras;
  the code takes the max [arxiv:2107.06307 §3.1] [code: model/homography.py#L292].
- **V2X-ViT compression.** The paper describes 1 × 1 convolutions with a default
  rate of 32. The code uses a 3 × 3 convolution with batch norm and ReLU, and the
  released config turns it off [arxiv:2203.10638 §3.1] [arxiv:2203.10638 §4.2]
  [code: v2xvit/models/sub_modules/naive_compress.py#L5-L24]. Compression changes
  the transmitted size, so a reproduction has to pick one.
- **V2X-ViT learning-rate schedule.** The paper says the rate drops by ×0.1 every
  10 epochs [arxiv:2203.10638 §4.2]. The code uses multistep decay at epochs 15
  and 50, over 60 epochs [code: v2xvit/tools/train_utils.py#L199-L202].
- **HDMapNet semantic loss.** The paper says cross-entropy; the code uses a
  per-class sigmoid loss with positive weight 2.13 [arxiv:2107.06307 §4.1]
  [code: loss.py#L26-L33].
- **HDMapNet distance margin.** Eq. 4 uses margin 2δ_d, the code uses δ_d with
  an extra factor of one half [arxiv:2107.06307 eq. 4] [code: loss.py#L89].
- **HDMapNet AP thresholds.** The paper's table uses 0.2, 0.5 and 1.0 with no
  units. The code uses 2, 4 and 6 metres [arxiv:2107.06307 tab. II]
  [code: evaluate_json.py#L10]. Check units before comparing numbers.
- **HDMapNet headline gain.** The abstract says the fusion model beats existing
  methods "by more than 50% in all metrics". On all-class IoU the gain over
  IPM(CB) is about 37% and over Surr about 35% [arxiv:2107.06307 tab. I]. The
  text's +12.1 IoU and +13.1 mAP [arxiv:2107.06307 §1] are against different
  baselines for each metric [arxiv:2107.06307 tab. II].
- **BEVFormer temporal combination.** Eq. 5 sums the current and the history
  streams. The code averages the two [arxiv:2203.17270 eq. 5]
  [code: projects/mmdet3d_plugin/bevformer/modules/temporal_self_attention.py#L257-L262].
- **BEVFormer anchor heights.** The paper says heights are spread uniformly
  from −5 m to 3 m. The code uses four cell centres at −4.5, −2.17, 0.17 and
  2.5 m [arxiv:2203.17270 §4.2]
  [code: projects/mmdet3d_plugin/bevformer/modules/encoder.py#L62-L63].
  Use the code's values to reproduce the results.
- **TopoLogic distance mapping.** The paper's mapping is exp(−x^α / (λσ)), with
  α and λ starting at 0.2 and 2 [arxiv:2405.14747 eq. 4] [arxiv:2405.14747 §4.2].
  The code uses exp(−d^P / w), with P starting at 2 and w at 10, and does not
  use σ [code: projects/topologic/models/modules/sgnn_decoder.py#L25-L28]
  [code: projects/topologic/models/modules/sgnn_decoder.py#L90-L98]. The
  released post-processing path hard-codes P, w and the fusion weights rather
  than reading the learned values [code: projects/topologic/models/dense_heads/topologic_head.py#L444-L452].
  Check which values a reproduction uses before comparing numbers.
- **TopoPoint point-lane matching.** The paper averages each point with the lane
  ends within 1.5 m and gives every matched lane the average. The released code
  has this block commented out. Its active refinement matches each lane end to
  other lanes' starts, using a similarity threshold of 0.7 and no 1.5 m rule
  [arxiv:2505.17771 §3.7] [arxiv:2505.17771 alg. 1]
  [code: projects/topopoint/core/lane/coders/lane_coder.py#L90-L201]
  [code: projects/topopoint/models/dense_heads/topopoint_head.py#L602-L631].
- **TopoPoint attention mask.** The paper adds the point-lane and lane-lane
  matrices to the attention logits. The code passes no mask to the decoder
  [arxiv:2505.17771 eq. 7]
  [code: projects/topopoint/models/modules/sgnn_decoder.py#L281-L301].
- **TopoFG distance metric.** The paper says the distance between boundary
  points is Euclidean [arxiv:2511.12590 §3.4]. The code sums absolute coordinate
  differences (L1) [code: projects/topofg/models/dense_heads/topofg_head.py#L771].
  A reproduction has to pick one.
- **TopoFG geometric term.** The paper does not say whether the geometric term is
  used in training. The code adds it at inference only, with both weights set to
  1, and trains the learnt score alone [code:
  projects/topofg/models/dense_heads/topofg_head.py#L462-L465] [code:
  projects/topofg/models/dense_heads/topofg_head.py#L785-L792].
- **TopoFG ablation text against its tables.** The text says the global spatial
  prior improves every metric [arxiv:2511.12590 §4.4]. In Table 4, adding it to
  the sequential prior lowers TOP_ll (26.1 against 26.7) and TOP_lt (26.6 against
  27.3) [arxiv:2511.12590 tab. 4]. The text also says sampled reference points
  give the best result on every metric [arxiv:2511.12590 §4.4]. In Table 5 the
  query initialisation alone has a higher TOP_ll (28.6 against 26.8)
  [arxiv:2511.12590 tab. 5].
- **Compact multi-task network, depth loss scale.** Eq. 2 gives 0.5 times the
  squared error inside δ = 0.5. The code's smooth-L1 call returns twice that at
  every error size, so the depth term is twice the paper's [arxiv:2606.02979 eq. 2]
  [code: mylib/function.py#L42-L49] [code: train.py#L92-L95]. An implementation
  that follows eq. 2 literally shifts the relative task weights at the start.
- **Compact multi-task network, second stage.** The paper says bottleneck 2
  stores "extracted information" from the other tasks. The code instead passes
  the depth, segmentation and LiDAR-segmentation outputs through a second encoder,
  which feeds bottleneck 2 and the BEV decoder's skip inputs
  [arxiv:2606.02979 §3.1] [code: mylib/arch.py#L263-L300]
  [code: mylib/arch.py#L321-L324].
- **Compact multi-task network, loss-weight timing.** Algorithm 1 updates the
  weights before the backward pass at the last step. The code renormalises them
  after the epoch's validation, so the new weights act from the next epoch. Its
  L_i(0) is reset at the first mini-batch of every epoch [arxiv:2606.02979 alg. 1]
  [code: train.py#L467-L468] [code: train.py#L125-L131].
- **Compact multi-task network, LiDAR and label preprocessing.** The paper's
  LiDAR and DVS projections (eq. 18 to 22) are not in the repository. The loader
  reads precomputed files, so the top-view stack cannot be rebuilt from the code
  [arxiv:2606.02979 eq. 18] [code: mylib/util.py#L56-L60]. The paper prints the
  rounding brackets for eq. 20 to 22 without saying whether it floors or rounds.
- **Compact multi-task network, class counts.** The paper uses 32 nuScenes
  classes for segmentation, LiDAR segmentation and BEV [arxiv:2606.02979 §4.2].
  The code's nuScenes variant has 23 outputs for segmentation and LiDAR
  segmentation and 9 for BEV, and the shipped configs select the other variant
  [code: mylib/arch.py#L394-L398] [code: train.py#L26].
- **SALON one-shot label and buffer eviction.** The paper links one clicked
  feature to high roughness and removes a sample from the most common class and
  speed [arxiv:2412.07826 §3.3] [arxiv:2412.07826 §3.5]. The official code seeds
  eight samples with label 1.0, never evicts them, and evicts at random from one
  of ten velocity bins. These are the SALON card's code findings, see its Paper
  vs code section ([card](/papers/2412.07826)).
- **SALON GP fitting.** The paper describes GP regression with no online fitting
  [arxiv:2412.07826 §3.4]. The code refits the GP every 10 cycles and fine-tunes
  the speed GP with SGD, learning rate 0.01 [card](/papers/2412.07826).

## Papers

- [HDMapNet: An Online HD Map Construction and Evaluation Framework](/papers/2107.06307):
  predicts vectorized local map elements in BEV from cameras, LiDAR or both,
  with a per-camera lift, a pillar LiDAR encoder and a fusion variant.
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](/papers/2203.17270):
  fills the BEV grid with learnable queries that use spatial and temporal
  deformable attention, and reads 3D boxes and map segmentation from it.
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](/papers/2203.10638):
  fuses LiDAR feature maps from connected vehicles and roadside units with
  heterogeneous multi-agent self-attention and multi-scale window attention.
- [TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenes](/papers/2405.14747):
  predicts which lane connects to which at intersections, by fusing an
  endpoint-distance score with a lane-query similarity score on a BEV encoder.
- [Fine-Grained Representation for Lane Topology Reasoning](/papers/2511.12590):
  gives each lane one query per ordered point, seeded from a predicted BEV lane
  mask, and connects lanes through their first and last queries.
- [TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Driving](/papers/2505.17771):
  adds a query set for lane endpoints that shares attention and graph layers
  with the lane queries, and refines lane ends with the detected points at
  inference.
- [A Concise Survey on Lane Topology Reasoning for HD Mapping](/papers/2504.01989):
  sorts lane-topology methods into procedural, aerial-imagery and onboard
  families, defines the metrics, and tabulates reported scores. It proposes no
  method.
- [Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion](/papers/2606.02979):
  one small network predicts depth, segmentation, LiDAR segmentation and a BEV
  map from RGB, DVS and a LiDAR height stack, and adapts its loss weights with
  a modified GradNorm.
- [SALON: Self-supervised Adaptive Learning for Off-road Navigation](/papers/2412.07826):
  fills a BEV grid with foundation-model features matched to lidar points, learns
  roughness online with a Gaussian process, and outputs a cost map and a speed
  map for off-road driving.