Object queries (learned query slots in transformer decoders)

Overview

Dense vision models predict one thing per pixel, anchor or patch. Many tasks instead want a small list of outputs: one entry per object, one point per head, or one mask per class. Object queries give a transformer decoder that list directly. You allocate a set of query slots, let them read the image features through attention, and read one output off each slot with a shared head 2005.12872 §3.2 2105.05633 §3.2. Later variants keep this pattern but change where a slot's position comes from, how a slot is assigned to a training target 2108.06152 §3.3 2211.12860 §3.3 2202.13065 §3.3, or what a slot predicts 2204.00987 §3.2.2 2205.14871 §3.2 2110.00061 §4. TrackFormer keeps the same decoder but feeds it the previous frame's outputs as well, so the slots persist across frames 2101.02702 §3.2. DETR3D keeps learned slots and gives each one a 3D reference point that is projected into every camera 2110.06922 §3.3. VisTR applies the same kind of decoder to a whole video clip, and groups its slots into one sequence per instance, matched as a unit 2011.14503 §3.2. TadTR uses learned action queries over the snippets of a video, each predicting one action's class, start and end 2106.10271 §III-A. A 2025 survey of lane topology describes the onboard pipeline's centerline decoder the same way: learned query vectors, each mapping one centerline instance 2504.01989 §2.3. Mask3D takes the non-learned route: its slots sit at farthest-point-sampled positions of a 3D scene, and each slot predicts one instance mask 2210.03105 §3, §3.1. Two newer uses change what a slot is for. D3G keeps DETR's object queries and treats them as the nodes of a dependency graph between objects 2409.02035 §4.2. UniV2X sends a roadside unit's query slots to a vehicle, which pairs them with its own slots before planning 2404.00717 §3.3.

What the variants share:

• A fixed number of query slots. Most variants use learned embeddings; Mask3D's positions come from sampling the input points instead 2210.03105 §3.1. N=100 in DETR 2005.12872 app. A.4, N=500 in CLTR 2202.13065 §4.1, 15 queries for table detection and 125 for table structure and functional analysis in PubTables-1M 2110.00061 app. Training Details, N=300 in Conditional DETR 2108.06152 §3.5, 300 or 900 in Co-DETR depending on the base detector 2211.12860 §4.1, N=300 in OrientedFormer 2409.19648 tab. 9, 64 bin queries plus one scene query in BinsFormer 2204.00987 §4.2.4, K (the number of semantic classes) in Segmenter 2105.05633 §3.2, 10 global queries in IAT 2205.14871 §3.2, and n·T = 360 in VisTR, which is 10 instances in each of 36 frames 2011.14503 §4.1. Mask3D uses 100 in all of its paper experiments 2210.03105 app. §1. D3G does not state N; its released code uses 100 (see the D3G card, Hyperparameters) 2409.02035 §4.1. UniV2X's roadside queries are the output of a tracker, one per tracked agent, so their count varies with the scene and is not stated as a number in the paper 2404.00717 §3.2. PolarFormer's learned queries are described without a count in the paper 2206.15398 §3.4. DeepInteraction uses 200 slots, but they are started from heatmap peaks of each scene, not from a learned table alone 2208.11112 §4.2. TrackFormer uses N_object=500 learned object queries on MOT17, plus one track query per active track 2101.02702 §4.2 2101.02702 §3.2, so its total slot count changes from frame to frame. DETR3D uses 900 2110.06922 tab. 6. TadTR uses 40 action queries on THUMOS14, 30 on HACS and 10 on ActivityNet 2106.10271 §IV-C. TopoMLP uses 300 lane queries, each predicting four Bezier control points 2310.06753 §4.2, tab. 3. TopoFG uses 200 lanes with 11 queries each, 2200 slots in total (our arithmetic) 2511.12590 §4.2. TopoPoint uses 200 point queries and 300 lane queries 2505.17771 §4.2, tab. 5.
• Slots interact. Self-attention runs over the slots as well as between slots and image features, so each output is computed with the others in view 2005.12872 §3.2 2105.05633 §3.2 2409.19648 §3.4 2206.15398 §3.4. DETR-style decoders for tables and crowds keep this 2110.00061 §4 2202.13065 §3.2. TrackFormer runs self-attention over all of its track and object queries together 2101.02702 §3.1 2101.02702 app. A.3, and DETR3D applies multi-head self-attention across its queries after each sampling step 2110.06922 §3.3. TopoPoint merges its point and lane queries into one self-attention 2505.17771 §3.4, eq. 3. TadTR's decoder runs self-attention across its action queries before each cross-attention step 2106.10271 §III-A. Mask3D's queries also run self-attention between queries 2210.03105 §3. Two variants are exceptions. Co-DETR's auxiliary query groups are isolated from one another 2211.12860 §1. IAT's queries have no query-to-query step at all 2205.14871 §3.2. In D3G, the graph layers run attention over pairs of slots after the decoder, so each pair's relation is computed with the rest of the graph in view 2409.02035 §4.2.
• Parallel decoding. All slots of a group are produced in one pass, not one after another 2005.12872 §3.2 2204.00987 §3.2.2. TrackFormer decodes every query of a frame in one pass 2101.02702 §3.1; DETR3D decodes all its queries in each of its six layers 2110.06922 §3.3; VisTR decodes the queries of a whole clip in one pass 2011.14503 §3.1.
• Shared heads applied to each slot. One head per output type, applied to every slot independently 2005.12872 §3.2 2105.05633 eq. 4 2204.00987 §3.2.2 2202.13065 §3 2210.03105 §3 2409.02035 §4.1 2106.10271 §III-A.

The variants differ in two places: how a slot's input enters the decoder (a positional encoding, a spatial query, a token, a box-derived Gaussian, a reference point refined in every layer, a 1-D action centre refined in every layer 2106.10271 §III-A, separate learned reference points for endpoint and lane queries, a heatmap peak, a learned 3-D anchor per control point, a BEV-mask prior plus a point-index embedding per lane point, a global query, the previous frame's output for the same object, or a slot's output in each frame of a clip; Mask3D starts each slot from a sampled scene position with zero features; in UniV2X, a roadside tracker's query moved into the vehicle's frame; in D3G, a node whose pair scores come from learned edge maps), and how a slot is tied to a training target (learned assignment by matching, identity matching for carried-over slots, a fixed one-to-one map to a class, a pre-assigned positive from a one-to-many head, a matched sequence of per-frame slots as in VisTR, a Hungarian match on mask costs as in Mask3D, or no per-slot tie at all because the loss acts on a dense output map, as in IAT and BinsFormer). Two papers add a third case. UniV2X pairs vehicle and roadside slots with Hungarian matching at fusion time, not as a training target 2404.00717 §3.3. D3G matches slots to objects and then applies a BCE loss to the relation matrix of the matched slots 2409.02035 §4.3.

Variants

Detection slots with bipartite matching (DETR)

DETR feeds N=100 learned positional embeddings, the object queries, into a 6-layer transformer decoder of width 256 2005.12872 §3.2, §4.1, app. A.4. Card: End-to-End Object Detection with Transformers.

• How a slot enters. The decoder input starts at zero. The object queries are added to the queries and keys (never the values) of every attention layer, and are shared across layers 2005.12872 §3.2, eq. 7, app. A.1, app. A.3. Each decoder layer runs self-attention over the N slots and cross-attention into the encoder output 2005.12872 §3.2.
• What a slot outputs. A 3-layer MLP gives a normalized box (center, height, width) and a linear layer with softmax gives a class among K classes plus "no object" \varnothing 2005.12872 §3.2.
• How a slot gets its target. No slot is tied to a class or region in advance. Per image, a Hungarian matching pairs each ground-truth object with one slot; unmatched slots are trained to predict \varnothing 2005.12872 §3.1, eq. 1, eq. 2. Details of the loss are on the bipartite matching page.
• Deep supervision. The shared heads and the matching loss are applied after every decoder layer 2005.12872 §3.2.

Reported behaviour:

• AP rises after every decoder layer, by 8.2 from the first to the last. NMS helps the first layer's output but slightly lowers AP at the last layers, where it removes true positives 2005.12872 §4.2, fig. 4.
• Where positional information is injected matters. With learned queries and sine spatial encodings both added at every attention layer, the R50 model gets 40.6 AP; adding both only at the decoder input gives 39.2; with no spatial encodings and queries only at the input, 32.8 2005.12872 tab. 3.
• R50 DETR reaches 42.0 AP on COCO val, equal to a tuned Faster R-CNN-FPN, with 61.1 vs 53.4 AP on large and 20.5 vs 26.6 on small objects 2005.12872 tab. 1.
• N is a hard cap. In a synthetic image with one class tiled on a 10x10 grid, DETR finds all instances up to about 50 and about 30 on average when all 100 are present 2005.12872 app. A.5, fig. 12.
• The slot outputs also drive masks: for panoptic segmentation, each decoder output embedding attends over the encoder output to give per-object heatmaps that an FPN-like head upsamples to masks 2005.12872 §4.4, fig. 8.

Conditional spatial queries (Conditional DETR)

Conditional DETR keeps DETR's learned object queries but gives each slot an explicit "where to look" query. The decoder's cross-attention then compares content with content and position with position, instead of mixing them. Card: Conditional DETR for Fast Training Convergence.

• How a slot enters. Query and key are the concatenation of a content part and a spatial part, so the dot product splits into a content term plus a spatial term 2108.06152 §3.3, eq. 3. The spatial query is the sinusoidal embedding of a reference point, scaled element-wise by a vector predicted from the previous decoder layer's output 2108.06152 §3.3, eq. 5, eq. 6.
• Where the reference point comes from. It is predicted from the object query with a small FFN, or set as a learnable per-query parameter 2108.06152 §3.5, §4.3. The object query still feeds the self-attention and the first decoder layer 2108.06152 §3.5.
• How a slot gets its target. Unchanged: Hungarian matching and a DETR-style loss, here with focal classification and 300 queries 2108.06152 §3.5.

Reported behaviour (R50 unless noted, COCO val, author-reported):

• At 50 epochs it reaches 40.9 AP against 34.9 for DETR; DETR reaches 42.0 only at 500 epochs 2108.06152 tab. 1. The 6.67x (R50, R101) and 10x (DC5) speed-ups are the epochs DETR needs to match its own 500-epoch result, divided by the epochs Conditional DETR needs 2108.06152 §4.2.
• Part of this gain is not the attention change. Focal loss and center-offset regression alone give 36.9 AP at 50 epochs; the conditional spatial query adds the remaining 4.0 2108.06152 tab. 4.
• The form of the spatial query matters. Predicting the scale from the self-attention output instead of the layer's decoder embedding gives 40.2 AP, against 40.9 2108.06152 tab. 3.
• The reference point matters too. Fixing it at (0,0), as DETR does, gives 36.8 AP at 50 epochs; learned per-query points give 40.7; predicting it from the object query gives 40.9 2108.06152 §4.3.
• Encoder cost is unchanged. Global single-scale encoder attention remains, and the DC5 variants cost 195 to 262 GFLOPs 2108.06152 tab. 1, §4.2.

Class embeddings as mask queries (Segmenter mask transformer)

Segmenter's mask transformer uses K learnable class embeddings, one per semantic class, randomly initialised 2105.05633 §3.2, §4.2. The authors cite DETR's object queries as an inspiration 2105.05633 §3.2. Card: Segmenter: Transformer for Semantic Segmentation.

• How a slot enters. The class embeddings are tokens: they are concatenated with the N patch encodings from the ViT encoder, and a 2-layer transformer encoder (self-attention only) processes the joint sequence 2105.05633 §3.2, §4.3.
• What a slot outputs. A mask: the scalar product between each L2-normalised output patch embedding and each output class embedding gives an N \times K score map, reshaped to the patch grid and bilinearly upsampled to pixels 2105.05633 eq. 4, §3.2.
• How a slot gets its target. Fixed: slot k is class k. Training is plain per-pixel cross-entropy, with no matching 2105.05633 §3, §4.2.
• The authors suggest replacing class embeddings with object embeddings to handle panoptic segmentation, but do not evaluate it 2105.05633 §3.2.

Reported behaviour, ADE20K val single-scale mIoU, linear decoder vs mask transformer 2105.05633 tab. 4:

The gain is not uniform across object sizes: Seg-B-Mask/16 is below the linear Seg-B/16 on large objects (52.66 vs 53.28) 2105.05633 tab. 4. Throughput barely changes (DeiT-B/16: 77 vs 76 images/s; ViT-L/16: 33 vs 31) 2105.05633 tab. 6. On Cityscapes the largest mask model uses a 1-layer mask transformer because of memory 2105.05633 §4.4.

Training-only query groups from one-to-many heads (Co-DETR)

Co-DETR keeps a DETR-family detector unchanged at inference, but adds auxiliary detection heads and query groups during training. The heads use classic one-to-many label assignment, which gives the encoder dense supervision; their positives become extra query groups for the decoder. Card: DETRs with Collaborative Hybrid Assignments Training.

• How a slot enters. A group's queries are built per image from the positive boxes of one auxiliary head: a linear map of a box positional encoding plus a linear map of the pyramid feature at each positive 2211.12860 §3.3, eq. 4. These queries are not fixed learned embeddings; they change with the image.
• Groups. Each auxiliary head forms its own group. Groups pass through the same decoder layers with shared weights, and each group is isolated from the others 2211.12860 §1, §3.3.
• How a slot gets its target. Every query in an auxiliary group is a positive with a pre-assigned target, so no matching is run for it 2211.12860 §3.3. The main branch still uses one-to-one Hungarian matching 2211.12860 §3.3, eq. 6.
• Encoder supervision. Each auxiliary head is also trained on the encoder pyramid, with positives and negatives from its own assigner. This is the dense signal the encoder receives 2211.12860 §3.2, eq. 1, eq. 2.
• Inference. Only the original DETR branch runs. There are no extra parameters or compute at test time, and the base method uses no NMS 2211.12860 §1.

Reported behaviour (Deformable-DETR R50 unless noted, COCO val, author-reported):

• At 12 epochs, AP is 37.1 with no auxiliary head, 42.3 with one (ATSS) and 42.9 with two (ATSS and Faster R-CNN). At 36 epochs the same comparison is 43.3, 46.8 and 46.5, so the second head does not help there 2211.12860 tab. 2.
• Both parts matter. At 12 / 36 epochs, the auxiliary head alone gives 41.6 / 46.2, the customized positive queries alone give 40.5 / 45.3, and both together give 42.3 / 46.8, against 37.1 / 43.3 for the baseline 2211.12860 tab. 9.
• Training cost rises with the number of heads. For Deformable-DETR++ R50 at 12 epochs, GPU hours go 70 (no head), 86 (one head), 120 (two heads), and memory goes from 12808 MB to 14387 MB at two heads 2211.12860 tab. 7.
• Extra training with a different budget: a 36-epoch Co-Deformable-DETR run (K=1, 288 GPU hours, 46.8 AP) is above the plain baseline at 150 epochs (1000 GPU hours, 45.9 AP) 2211.12860 tab. 10.
• More heads is not better. Six different heads score 48.9 AP, below ATSS plus Faster R-CNN (49.5); the authors attribute this to conflicting assignments 2211.12860 tab. 7, §4.4.
• The strongest DINO-based rows also use QFL and NMS, so their gains over DINO mix the training scheme with post-processing 2211.12860 §4.3, tab. 4.

Table objects from an unchanged DETR (PubTables-1M)

PubTables-1M applies DETR to three table tasks without architectural changes: table detection, table structure recognition and functional analysis. The contribution is the dataset; the detector is a reference point. Card: PubTables-1M: Towards comprehensive table extraction from unstructured documents.

• How a slot enters. Learned object queries, as in DETR 2110.00061 §4. The appendix sets 15 queries for table detection and 125 for structure and functional analysis 2110.00061 app. Training Details.
• What a slot outputs. For structure, one detector predicts six object types: the whole table, its columns, its rows, its column-header cells, its projected row headers and its spanning cells. The grid cell is the intersection of a column and a row and is not predicted as its own type 2110.00061 §4.
• How a slot gets its target. DETR's one-to-one matching. The paper does not state the matching weights 2110.00061 app. Training Details.
• Training targets are dilated. Each shared boundary between neighbouring rows, and between neighbouring columns, moves to the midpoint of the gap, so the boxes fill the gaps. Scoring then tightens the boxes to the text they contain 2110.00061 §4, app. Scoring. This is a change to the targets, not to the slots.

Reported behaviour (ResNet-18 backbone, PubTables-1M test split, author-reported):

• Table detection: AP 0.966 for DETR against 0.825 for Faster R-CNN 2110.00061 tab. 3. For table structure and functional analysis the same comparison is 0.912 against 0.722 2110.00061 tab. 3.
• Structure recognition on canonical test data, all tables: content accuracy 0.8138 and GriTS_Top 0.9845 for DETR, against 0.1039 and 0.8616 for Faster R-CNN 2110.00061 tab. 4.
• The DETR and Faster R-CNN runs share the paper's own data, which the paper canonicalises. The gap therefore does not isolate the detector from the data changes 2110.00061 §3, §5.

Head points with a kNN-context matching cost (CLTR)

CLTR predicts each head as a point, not a box, so a slot outputs a point and a confidence. Training pairs slots with heads in a way that also looks at each head's neighbours. Card: An End-to-End Transformer Model for Crowd Localization.

• How a slot enters. The 500 learned queries are the decoder's object queries 2202.13065 §4.1. The cross-attention uses the conditional form, with the learned embedding in the query, following Conditional DETR 2202.13065 §3.2.
• What a slot outputs. A point, from a regression head, and a confidence, from a classification head 2202.13065 §3.
• How a slot gets its target. One-to-one Hungarian matching. The cost adds the L1 distance between points and the L1 distance between kNN-context values, then subtracts the predicted confidence. The kNN value of each prediction is an output of the network 2202.13065 §3.3, eq. 2, eq. 3. The loss is focal over all slots plus L1 on the matched points, with weight λ=2.5 2202.13065 §3.4, eq. 5.
• No boxes, density maps or NMS. Inference keeps slots with confidence at least 0.35 and counts them 2202.13065 §1, §4.1.

Reported behaviour (ResNet-50 at stride 32, point labels only, author-reported):

• UCF-QNRF localisation F1, averaged over σ from 1 to 100 px: 80.97 for CLTR against 80.34 for TopoCount 2202.13065 tab. 2.
• ShanghaiTech Part A at σ=4: F1 43.2 for CLTR against 41.1 for TopoCount 2202.13065 tab. 3.
• NWPU-Crowd test: F1 68.5 for CLTR against 69.1 for TopoCount, which uses box labels in training 2202.13065 tab. 1.
• Slot count on UCF-QNRF: F1 79.89 with 300 queries, 80.97 with 500, 80.34 with 700 2202.13065 tab. 7. Each setting is one run 2202.13065 tab. 7.
• Matching cost on UCF-QNRF: F1 80.02 with plain L1 plus confidence, 80.97 with the kNN term. One run 2202.13065 tab. 8.

Carried-over track slots (TrackFormer)

TrackFormer tracks objects with a DETR-style decoder that runs once per frame. Each frame it gets two kinds of slot: one track query per active track, which is the object's output from the previous frame, and a fixed set of learned object queries that can start new tracks 2101.02702 §3.2. Card: TrackFormer: Multi-Object Tracking with Transformers.

• How a slot enters. A track query is the previous frame's output embedding for that object, taken from the last decoder layer before the prediction heads 2101.02702 §3.2 2101.02702 app. A.3. The object queries are learned embeddings, 500 for the MOT17 model 2101.02702 §3.2 2101.02702 §4.2. For the deformable-attention variant, a track query's reference point is the centre of its previous box 2101.02702 §4.2.
• What a slot outputs. A box and a class from small MLP heads, as in DETR 2101.02702 §3.1. A track starts when an object query scores above 0.4 and ends when its score falls below 0.4 2101.02702 §3.2 2101.02702 app. A.4. Re-identification from an inactive pool is described in the paper 2101.02702 app. A.4; see the card's Paper vs code for its default setting in the released code.
• How a slot gets its target. Track-query outputs are matched to the ground-truth object with the same identity. Tracks that have disappeared are trained towards background. New objects are matched to object queries with the cost in eq. 2 and eq. 3 2101.02702 §3.3 2101.02702 eq. 2 2101.02702 eq. 3. The loss sums over all predictions, object and track 2101.02702 §3.3 2101.02702 eq. 4.
• Frame pairs and augmentations. Training uses a previous frame sampled from a window around frame t. The paper drops each track query with probability 0.4 (false negatives) and adds false-positive track queries with probability 0.1 2101.02702 §3.3 2101.02702 app. A.1. The window size is not stated. The released code's augmentation and gradient choices differ from the text; see the TrackFormer card, Paper vs code.

Reported behaviour (author-reported, MOT17 unless noted):

• MOT17 test, private detections, CrowdHuman-only training data: 74.1 MOTA and 68.0 IDF1, against 69.1 and 63.9 for TraDeS 2101.02702 tab. 1.
• MOT17 test, public detections with the IoU filter: 62.3 MOTA and 57.6 IDF1, against 62.1 and 62.8 for TMOH, which is higher on IDF1 2101.02702 tab. 1. The paper treats the public-detection filter as a choice that changes the numbers 2101.02702 app. B.1.
• MOTS20 test: 54.9 sMOTSA, against 40.6 for Track R-CNN 2101.02702 tab. 2.
• Ablation on the MOT17 training data, 50/50 frame split. Each row removes one component on top of the rows above it. The full model reaches 71.3 MOTA and 73.4 IDF1; the last row, without track queries and with greedy centre matching, reaches 61.0 MOTA and 45.1 IDF1 2101.02702 tab. 3. Read the row differences as cumulative, not as the effect of one component alone.

3D reference points projected into camera images (DETR3D)

DETR3D predicts 3D boxes from six cameras with no depth network and no NMS. It keeps learned object queries, but each query first gives a 3D reference point. That point is projected into every camera image, and the image features found there update the query 2110.06922 §3.3. Card: DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries.

• How a slot enters. 900 learned queries of width 256 2110.06922 tab. 6 2110.06922 §4.1. Each decoder layer decodes a 3D reference point from the query 2110.06922 eq. 1. The point is appended with 1 and projected with each camera's 3×4 matrix 2110.06922 eq. 2. Bilinear samples are taken from each of the four FPN levels and each camera 2110.06922 eq. 3. The samples that fall inside the image are averaged and added back to the query as a residual 2110.06922 eq. 4 §3.3.
• What a slot outputs. A box and a class, predicted from the query after every layer. At inference only the last layer is used and no NMS is applied 2110.06922 eq. 5 2110.06922 §3.3 2110.06922 §4.1.
• How a slot gets its target. Hungarian matching on a cost that combines class probability and an L1 box term. A focal loss covers all predictions, and an L1 loss covers only the matched boxes. The loss is applied to every decoder layer 2110.06922 §3.4 2110.06922 §3.3. The paper gives no weights for these terms.
• Calibration is required. The method assumes the camera projection matrices are known 2110.06922 §3.1.

Reported behaviour (nuScenes, author-reported):

• Validation, all cameras: 0.374 NDS and 0.303 mAP for the plain model, against 0.373 NDS and 0.299 mAP for FCOS3D, which uses NMS 2110.06922 tab. 1. The CBGS variant reaches 0.434 NDS and 0.349 mAP 2110.06922 tab. 1.
• Translation error is worse than FCOS3D: mATE 0.860 against 0.785. The paper suspects FCOS3D's direct depth prediction supervises translation more strongly 2110.06922 §4.2 2110.06922 tab. 1.
• Test set: 0.479 NDS and 0.412 mAP, a leaderboard result as of 13 October 2021 2110.06922 tab. 2 2110.06922 §4.2.
• Query count: 30 queries give 0.201 mAP, 100 give 0.313 and 900 give 0.346. The mAP peak is at 600 (0.347) 2110.06922 tab. 6.
• Layers: NDS rises from 0.380 after the first decoder layer to 0.425 after the sixth 2110.06922 tab. 5.
• Backbone: ResNet-50 gives 0.373 NDS and ResNet-101 gives 0.425 2110.06922 tab. 7.
• Against the authors' pseudo-LiDAR baseline, a supervised depth network followed by CenterPoint, DETR3D reaches 0.374 NDS against 0.160 2110.06922 tab. 4 2110.06922 §4.4.

Bezier-curve lane queries with pairwise topology heads (TopoMLP)

TopoMLP detects lanes and traffic elements from several cameras, then predicts which lanes connect to each other and which lanes connect to which traffic element. Each lane query is a curve, not a box: it outputs a few Bezier control points, and the curve is evaluated from them. Card: TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoning.

• How a slot enters. Each lane query starts from a learnable 3-D anchor point per control point. The anchors go through a linear layer and a transformer decoder 2310.06753 §3.1, eq. 1. The decoded features then give offsets to the anchors 2310.06753 §3.1.
• What a slot outputs. Control points, one set per lane, with M = 4 2310.06753 §4.2, tab. 3. A second MLP predicts the class score of the lane 2310.06753 §3.1.
• Topology from the slots. The lane features are paired with each other (N_L x N_L) and with the traffic queries (N_L x N_T) by repeating and concatenating them. An MLP scores each pair 2310.06753 §3.3, eq. 3 2310.06753 §3.4, eq. 4. The camera view matrix is added to the lane features before the lane-traffic pairing 2310.06753 §3.4.
• How a slot gets its target. Bipartite matching assigns slots to lanes and traffic elements for the detection losses. The topology losses reuse that same assignment, so an unmatched query gets no topology supervision 2310.06753 §3.5.
• Design claim. The authors argue that the topology step is simple once detection is good. In their motivating test, giving the topology heads ground-truth detections raised TOP_ll only from 9.5 to 10.0 and TOP_lt from 28.3 to 30.9 on Swin-B 2310.06753 §1, fig. 1.

Reported behaviour (OpenLane-V2, author-reported, 24 epochs unless stated):

• Subset A, ResNet-50: OLS 38.2 against 35.6 for TopoNet; TOP_ll 7.2 against 4.1 2310.06753 tab. 1. With extra YOLOv8 proposals, OLS is 41.2 2310.06753 tab. 1.
• Subset B, ResNet-50: OLS 38.7 against 33.2 for TopoNet 2310.06753 tab. 2.
• Lane queries: 200, 300 and 500 give OLS 36.9, 38.2 and 38.0 2310.06753 tab. 3.
• Metric: raising confidences of the same TopoMLP predictions moves OLS from 38.2 to 42.2. The adjusted metric TOP† gives 36.3 for the original and 34.5 for the raised-confidence version 2310.06753 tab. 4. The paper's own ablations use one training run each, with no variance reported 2310.06753 tab. 3.
• The paper describes 300 lanes and no extra training branch 2310.06753 §3.1, §4.2. The official code adds a one-to-many lane branch in training, and its topology heads use only the one-to-one lanes. See the card's Paper vs code before reproducing numbers.

Fine-grained per-point queries for lane topology (TopoFG)

TopoFG gives each lane 11 ordered queries, one per point, where TopoMLP and TopoLogic use one query per lane. Only the first and last query of each lane feed a pairwise connectivity head. Card: Fine-Grained Representation for Lane Topology Reasoning.

• How a slot enters. A query is the sum of a lane-level spatial prior and a point-level sequential prior 2511.12590 §3.3, eq. 4. The spatial prior is a learned lane query refined from a predicted BEV lane mask, which is turned into weights and encoded over the BEV grid 2511.12590 §3.2, eq. 1, eq. 2. The sequential prior is a learned point query plus an encoding of the point's index, from 1 to 11 2511.12590 §3.2, eq. 3.
• Where the sampling points come from. Deformable cross-attention reads the BEV features at reference points taken from the predicted lane mask, not from a learned table alone 2511.12590 §3.3, eq. 5.
• Slots interact along and across lanes. Each of the six decoder layers attends across lanes at the same point index, then along the points of each lane, then reads the BEV features 2511.12590 §3.3.
• What a slot outputs. One key point of the lane per query, following TopoLogic 2511.12590 §3.5. Lane connections come from the first and last query of each lane, through a shared MLP over ordered pairs 2511.12590 §3.4, eq. 6, eq. 7. At inference the learned similarity is summed with a geometric term built from the distance between boundary points 2511.12590 §3.4.
• How a slot gets its target. One-to-one Hungarian matching for the lanes 2511.12590 §3.4. For the topology, the paper adds G = 5 denoising groups, each a noisy copy of every ground-truth lane. Their adjacency is the fixed ground-truth matrix, so the labels do not shift with the matching step. The groups are used in training only 2511.12590 §3.4, §4.2.

Reported behaviour (OpenLane-V2 subset_A, ResNet-50, author-reported; the paper gives no seeds or variance):

• OLS 48.0 against 44.1 for TopoLogic, and TOP_ll 30.8 against 23.9 2511.12590 tab. 1. On subset_B, OLS is 45.4 against 42.3 2511.12590 tab. 1.
• Ablation, cumulative: baseline 44.1; plus the priors 45.4; plus the decoder 45.8; plus the boundary-point topology 48.0 2511.12590 tab. 3.
• Denoising groups alone add 1.5 OLS over the same model without them (47.3 against 45.8). Boundary-point topology alone adds 0.8 (46.6 against 45.8) 2511.12590 tab. 6.
• The text says the global spatial prior improves all metrics. In Table 4 the full prior gives lower TOP_ll (26.1 against 26.7) and TOP_lt (26.6 against 27.3) than the sequential prior alone 2511.12590 §4.4, tab. 4.
• The released code differs from the paper in several places, including the distance metric and the inference geometry. See the card's Paper vs code before reproducing numbers.

Endpoint queries merged with lane queries (TopoPoint)

TopoPoint adds one query per lane endpoint, next to the lane queries. The two sets share one self-attention and a point-lane graph, so detected endpoints can pull connected lane ends toward one shared point. Card: TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Driving.

• How a slot enters. The decoder has 200 point queries and 300 lane queries, each set with its own learnable reference point 2505.17771 §4.2, §3.4. Both sets go into one self-attention. The paper adds geometric terms, built from the previous layer's predictions, to the attention logits 2505.17771 §3.4, eq. 3, eq. 4-8.
• What a slot outputs. A point query gives one 3D endpoint and a score. A lane query gives 11 3D points and a score 2505.17771 §3.4, eq. 17. Pairwise MLP embeddings give the point-lane, lane-lane and lane-traffic topology matrices 2505.17771 §3.5, eq. 18-20.
• Graph updates. A point-lane graph convolution passes messages both ways between points and lanes. It runs before and after the lane-lane and lane-traffic branches 2505.17771 §3.4, eq. 11-16.
• How a slot gets its target. The point and lane losses are focal classification plus L1 regression, and the three topology losses are focal 2505.17771 §3.6, eq. 21. The paper does not state the matching costs 2505.17771 §3.6.
• Inference step. Points and lanes scoring above 0.3 are kept. Each kept point is averaged with the lane endpoints within 1.5 m, and each matched lane takes that average as its endpoint 2505.17771 §4.2, alg. 1. The released code does not run this as written; see the TopoPoint card, Paper vs code.

Reported behaviour (OpenLane-V2 subset_A and subset_B, ResNet-50, 24 epochs, author-reported):

• subset_A: OLS 48.8 and DET_p 52.6 2505.17771 tab. 1. The TopoLogic row gives OLS 44.1, copied from an earlier paper, and DET_p 45.2, computed by the authors 2505.17771 tab. 1, §4.1.
• subset_B: OLS 49.2 against 42.3 for TopoLogic, and DET_p 45.1 against 39.2 2505.17771 tab. 1.
• Ablation, from the authors' re-run of the TopoLogic code at OLS 43.4 and DET_p 44.5: native-scale front view adds 2.6 OLS, merged self-attention 1.6, the point-lane graph 0.7 and the inference step 0.5 2505.17771 tab. 2. The first step is an input-resolution change, not a query change.
• Query counts: 200 point and 300 lane queries give the best OLS, 48.8. 300 point queries with 300 lanes give 48.3, and 100 point queries with 200 lanes give 46.5 2505.17771 tab. 5.

Relevance of a slot to image regions (post-hoc explanation)

This variant does not change the slots. It explains a trained DETR-style decoder: for each detected object, which image regions its query relied on. Card: Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers.

• How it works. Each attention map is weighted by the gradient of the target score, its positive part is kept, and the heads are averaged. Relevance is then propagated layer by layer. In the DETR decoder, the query self-attention rule runs first and the query-to-image rule second; the row of the detected object gives one score per image region 2103.15679 §3, §3.2, eq. 5, eq. 10.
• Target. The paper uses the model's output for the class of the detected object, but does not say whether that output is a logit or a probability 2103.15679 §3.
• No training. DETR is used as trained, and the method adds no parameters 2103.15679 §5.
• Segmentation from relevance. Queries whose class probability is above 50% are kept. For each, the relevance row is thresholded with Otsu's method and upsampled to a mask. The procedure runs once per kept query 2103.15679 app. C, alg. 1.

Reported behaviour (author-reported, COCO val, weakly supervised DETR segmentation):

• AP 13.1 against 7.2 for the best baseline, Transformer attribution. Rollout and raw attention reach 0.1 and 5.6 2103.15679 tab. 1.
• Both the accumulation over layers and the normalisation step matter for this result. Without the accumulation, AP falls to 0.1; without normalisation, to 11.7 2103.15679 tab. 3.
• The paper's LRP variant is close to the raw-attention version on LXMERT, so the paper does not need LRP 2103.15679 app. E, tab. 5.

Oriented-box positional queries (OrientedFormer)

OrientedFormer is a DETR-style decoder for rotated boxes in remote-sensing images. It uses no encoder and 300 queries. Each query carries a content part and a positional part, which is the oriented box (x, y, w, h, θ) and is updated from the content in every layer 2409.19648 §3.1, §3.2 2409.19648 alg. 1. Card: OrientedFormer.

• How a slot enters. The box becomes a 2-D Gaussian, and its sinusoidal encoding is replaced by the expectation of sine and cosine under that Gaussian 2409.19648 §3.3, eq. 2, eq. 6. Cross-attention samples features at points offset from the box centre and rotated by θ [arxiv:2409.19648 §3.5, eq. 13, eq. 14].
• Self-attention bias. The squared 2-Wasserstein distance between two slots' Gaussians is turned into an additive bias on the attention logits. The bias has no parameters 2409.19648 §3.4, eq. 8, eq. 9.
• How a slot gets its target. One-to-one Hungarian matching with a focal, an L1 and a rotated-IoU loss 2409.19648 §3.6, eq. 19. The paper claims no hand-designed post-processing 2409.19648 §1. The official code selects its initial queries with an NMS step, which the card's Paper vs code section describes.

Reported behaviour (ResNet-50, DIOR-R AP50, author-reported):

• Positional encoding: none 66.85, learnable 64.27, deformable 65.97, DAB 66.03, Gaussian 67.28. The Gaussian encoding is 0.4 AP50 above no encoding 2409.19648 tab. 10.
• Query count: 100 gives 65.16, 300 gives 67.28, 500 gives 66.93 2409.19648 tab. 9.
• DOTA-v1.0 single-scale AP50 is 75.37 against 74.16 for ARS-DETR. Oriented R-CNN is higher at 75.87 2409.19648 tab. 4.
• Cost: 529 GFLOPs at 11.4 FPS, against 186 GFLOPs for ARS-DETR and Deformable DETR-O 2409.19648 tab. 14. The authors report 12 training epochs for their model against 36 for the two baselines 2409.19648 §4.8.

Reference-point queries on a polar bird's-eye-view map (PolarFormer)

PolarFormer detects 3D objects from six cameras on a polar grid of radius and azimuth. Its decoder uses learned queries that carry a reference point, refined in every layer. Card: PolarFormer: Multi-camera 3D Object Detection with Polar Transformer.

• How a slot enters. One learned embedding is split into a content part and a position part. The position part gives a 3-D reference point (azimuth, radius, height), through a sigmoid of a linear map 2206.15398 §3.4. Deformable cross-attention samples the polar map around the reference point 2206.15398 §3.3, §3.4.
• Refinement. Each layer adds offsets to the reference point, so the slot moves across the map 2206.15398 §3.4.
• What a slot outputs. A centre, a log size, and sine and cosine of the heading and velocity, both measured relative to the azimuth of the object 2206.15398 §3.4.
• How a slot gets its target. The paper names focal and L1 losses but not a matching rule 2206.15398 §3.4. The released code uses Hungarian matching; see the card's Paper vs code.
• Not object slots. The cross-plane encoder also has "ray queries", one per polar ray and image column. These are not object slots and are not matched to objects 2206.15398 §3.1.

Reported behaviour (author-reported, nuScenes, six cameras, single frame):

• Val, ResNet-101 with an FCOS3D initialisation: 39.6 mAP and 45.8 NDS, against 37.5 and 44.8 for BEVFormer-S 2206.15398 tab. 3.
• Test, same setting: 41.5 mAP and 47.0 NDS, against 40.9 and 46.2 for BEVFormer-S 2206.15398 tab. 1.
• The temporal variant's numbers are not reproducible from the released code, and the paper's improved setting uses extra pretraining data 2206.15398 tab. 1, §4.

Heatmap-initialised slots with modality interaction (DeepInteraction)

DeepInteraction keeps one representation per sensor and lets LiDAR and camera features exchange information. Its decoder refines 200 slots that start at the strongest peaks of a heatmap built from the LiDAR features. Card: DeepInteraction: 3D Object Detection via Modality Interaction.

• How a slot enters. Each slot starts at a peak of the BEV heatmap, as in TransFusion (no card; named only) 2208.11112 §4.2. The start is scene-dependent, not a learned embedding alone.
• How a slot is refined. The slot's parameters are generated from the slot itself and applied to a 7 x 7 RoI pooled from the other modality. Odd layers pool from the image, even layers from the LiDAR BEV map 2208.11112 §3.2.
• What a slot outputs. Centre, height, size, rotation and velocity, through heads applied to the refined slots 2208.11112 §3.2.
• How a slot gets its target. Hungarian matching as in TransFusion, with the same loss applied to every layer's predictions 2208.11112 §3.2. The paper does not mention non-maximum suppression, and the released code applies none at test time (see the card's Paper vs code).

Reported behaviour (nuScenes, author-reported):

• Test: 70.8 mAP and 73.4 NDS for the base model with a ResNet-50 image backbone 2208.11112 tab. 1.
• Decoder layer type, val, with both branches: 68.6 mAP and 71.6 NDS for DETR-style layers, 69.9 and 72.6 for the interaction layers 2208.11112 tab. 3.
• Alternating LiDAR and image refinement against LiDAR only: 69.9 against 69.2 mAP on val 2208.11112 tab. 3.
• Speed on a V100: 2.6 FPS against 3.8 for TransFusion 2208.11112 tab. 2.
• The best query setting in the paper's ablation uses 200 slots in training and 300 at test: 70.1 mAP and 72.7 NDS on val 2208.11112 tab. 3. None of the ablations in tab. 3 varies the start of the slots, so the gain from heatmap starts is not separated from the rest of the decoder.

Per-image depth bins from queries (BinsFormer)

BinsFormer uses the queries to generate depth bins for each image rather than object outputs. A Transformer decoder reads multi-scale features with 64 bin queries and one scene query 2204.00987 §3.2.2, §4.2.4. Each pixel's depth is a probability-weighted sum of bin centres 2204.00987 §3.2.3, eq. 2. Card: BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation.

• How a slot enters. Learned query embeddings, one per bin and one for the scene. The paper says the queries start at zero and carry learned positional embeddings 2204.00987 §3.2.5.
• What a slot outputs. A bin-length logit, softmaxed over the bins to give the bin widths, and a bin embedding from a 3-layer MLP. The scene query feeds a classifier over 25 indoor scene classes on NYU 2204.00987 §3.2.2, §3.2.4, §4.2.1.
• How a slot gets its target. No slot is matched to an object. The loss is a scale-invariant depth loss on pixels plus a scene cross-entropy term, both on the output maps 2204.00987 §3.2.3, §3.2.4, eq. 3.
• Scales. The decoder runs over three feature scales with three layers each, nine in total 2204.00987 §4.2.4.

Reported behaviour (author-reported, NYU Depth v2 unless noted; the ablation backbone is not stated):

• Adding a bin query to a regression baseline raises δ1 from 0.852 to 0.870. The full model with scene loss and multi-scale refinement reaches 0.890 2204.00987 tab. 5.
• Number of bins: 1 bin does not converge, 8 bins give δ1 0.889, the default 64 give 0.890, and 512 give 0.883 2204.00987 tab. 6.
• Feature scales: the finest scale alone gives 0.882 with three layers; the default three scales give 0.890 with nine layers 2204.00987 tab. 7.
• With a Swin-Large backbone pre-trained on ImageNet-22K, δ1 is 0.925 against 0.903 for AdaBins 2204.00987 tab. 3.

Global queries for image-level parameters (IAT)

IAT is a light enhancement network for low-light and mis-exposed photos. Its global branch does not predict per-object outputs. It predicts one colour matrix and one gamma for the whole image, using 10 learned queries: nine for the matrix and one for gamma 2205.14871 §3.2. Card: IAT, a 90K-parameter transformer for light and exposure correction.

• How a slot enters. The queries cross-attend to keys and values from a low-resolution encoding of the image. There is no query-to-query self-attention and no residual around the attention. A feed-forward network follows 2205.14871 §3.2. The paper starts the queries at zero; the card's Paper vs code section notes that the released code starts them at one.
• What a slot outputs. Gamma is one plus a linear map of its query. The colour matrix is the identity plus a linear map of the other nine queries, so the model starts as the identity 2205.14871 §3.2.
• How a slot gets its target. None per slot. The whole image is supervised by an L1 loss, or by a smooth L1 plus VGG loss on LOL 2205.14871 §4.1.

Reported behaviour (LOL-V2-real PSNR, ablation, author-reported):

• With the proposed normalisation and local branch only, PSNR is 20.01. Adding the global matrix gives 21.95, adding the global gamma gives 22.76, and both give 23.50 2205.14871 tab. 7.
• The full model has 0.09M parameters and reports PSNR 23.50 and SSIM 0.824 on LOL-V2-real 2205.14871 tab. 1.

Instance-sequence slots over a video clip (VisTR)

VisTR runs the object-query decoder over a whole clip of T = 36 frames. It has N = n·T learned queries, with n = 10 instances per frame, so each slot is one instance in one frame, and the slots of one instance form a sequence over time 2011.14503 §3.1, §4.1. Card: End-to-End Video Instance Segmentation with Transformers.

• How a slot enters. Each query is a separate learned embedding with the width of the pixel features. The encoder's space-time tokens carry a fixed 3D sine/cosine positional encoding, with d/3 channels per axis 2011.14503 §3.1, eq. 1.
• What a slot outputs. For each frame, a box and a class from small heads, as in DETR 2011.14503 §3.2. Masks come from a separate stage: each slot's per-frame mask feature is stacked over the clip, and three 3D convolutions turn that stack into a mask sequence 2011.14503 §3.3.
• How a slot gets its target. At the level of sequences. The matching cost of a sequence is computed over all of its frames, and a one-to-one Hungarian assignment pairs sequences with ground-truth instances 2011.14503 §3.2, eq. 2, eq. 6. The box loss and the mask loss are both averaged over the T frames 2011.14503 eq. 8, eq. 9.
• Tracking by layout. Because slot k is the same instance in every frame, no separate linking step is needed. The paper states this as a design goal 2011.14503 §1, §3.2.

Reported behaviour (author-reported, YouTube-VIS validation unless noted):

• Main table: 40.1 AP and 64.0 AP50 with ResNet-101, and 36.2 AP with ResNet-50 2011.14503 tab. 2. On the same ResNet-50 backbone, MaskTrack R-CNN has 30.3 AP in the same table, and its numbers come from re-implementations in the MaskTrack R-CNN paper 2011.14503 tab. 2.
• Speed: 27.7 FPS with ResNet-101 including data loading, and 57.7 without 2011.14503 tab. 2. Images are loaded serially in the measurement, which the authors say costs much time.
• Clip length, ResNet-50, 10 epochs: AP 29.7, 30.5, 31.7 and 33.3 for T = 18, 24, 30 and 36 2011.14503 tab. 1.
• Query layout, same setting: one query per instance and frame (360) gives 33.3 AP. Ten instance embeddings shared across frames give 32.0, and 36 frame embeddings give 13.7 2011.14503 tab. 1.
• Positional encoding, same setting: 28.4 AP without it and 33.3 with it 2011.14503 tab. 1.
• 3D convolutions in the mask head: 34.4 AP with them, 33.3 without 2011.14503 tab. 1.
• The authors call VisTR the most accurate single-model method. MaskProp with ResNet-101 reaches 42.5 AP in the same table, and the authors exclude it because it combines several networks and post-processing 2011.14503 tab. 2, §4.3. This is their argument, not a result of the table.

Non-parametric instance queries on 3D scans (Mask3D)

Mask3D segments object instances in a 3D point cloud. It has no learned query table. It places 100 slots at farthest-point-sampled positions of the scene, and the decoder turns each slot into one instance: a class and a binary voxel mask. Card: Mask3D: Mask Transformer for 3D Semantic Instance Segmentation.

• How a slot enters. Features start at zero. Only the position is used, Fourier-encoded from the sampled point, and it stays the same across layers 2210.03105 §3.1, app. §1. The paper says the slot count can change at inference without retraining 2210.03105 §3.1.
• Masked cross-attention. A slot attends only to voxels inside its mask from the previous layer; the other voxels are excluded by adding minus infinity to their logits 2210.03105 §3, eq. 3. Attention runs over the four coarsest of five scales. The decoder repeats three times with shared weights, giving 12 refinement steps 2210.03105 §3, app. §1.
• What a slot outputs. A mask from a dot product between the slot's instance feature and the per-voxel features, then a sigmoid and a 0.5 threshold 2210.03105 eq. 1, §3. A class from a linear layer with softmax over C+1 classes, the extra one being "no object" 2210.03105 §3. The mask module runs after every layer, so each layer gives an auxiliary mask.
• How a slot gets its target. Hungarian matching on a cost of Dice, BCE and class terms, with weights 2, 5 and 2 2210.03105 eq. 4, §3.1. The loss on matched pairs is 5 times BCE plus 2 times Dice 2210.03105 eq. 5. Unmatched slots get a "no object" class weight of 0.1, and the per-layer losses are summed 2210.03105 §3.1, eq. 6.

Reported behaviour (ScanNet, author-reported):

• ScanNet test mAP 56.6 against 50.4 for SoftGroup, the best listed baseline 2210.03105 tab. 1. The paper's headline gains on S3DIS and ScanNet200 mix pretrained and non-pretrained baselines, so compare table rows, not abstract figures 2210.03105 tab. 2, tab. 3.
• Slot design, ScanNet val at 5 cm: zero features with sampled positions reach 40.6 ± 0.3 mAP, against 39.7 ± 0.7 for learned parametric queries and 38.4 ± 0.3 when the sampled points' features initialise the slots 2210.03105 tab. 4.
• Mask loss, same setting: BCE alone 27.0, Dice alone 38.0, both 40.6 2210.03105 tab. 4.
• Query count: fewer queries lower accuracy, and more add less than 1 mAP. The paper reports this in a figure it does not tabulate 2210.03105 §4.2.

Learned query tokens that are not object slots (DualVLN)

DualVLN's planner also uses learned tokens, but they do not output one object each. Its latent tokens condition a downstream policy, and nothing is matched to them. Card: Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation.

• How a token enters. Four learnable <TRAJ> tokens are appended to the planner's text after the pixel-goal text. They start random and are trained by prompt tuning 2512.08186 §3.2, app. A. Their final-layer hidden states form the latent goal 2512.08186 app. A, §3.3.
• Visual compression. A Q-Former reads the fused features of two frames and outputs 32 visual-memory tokens 2512.08186 §3.2. These are inputs to the policy, not per-object outputs.
• What the tokens output. None directly. A diffusion transformer generates 32 waypoints, conditioned on the latent goal and the visual tokens 2512.08186 §3.2.
• How they get their target. None per token. The policy is trained with a flow-matching loss on the trajectory 2512.08186 eq. 3. The matching and slot-count rules above do not apply.

Reported behaviour: the paper says that removing the latent goal makes results worse, but gives the numbers only in a figure 2512.08186 fig. 7.

Object queries as dependency-graph nodes (D3G)

D3G detects objects with a DETR-style decoder, then predicts for every pair of slots whether one object has to be removed before the other, for picking from a cluttered bin. The slots are not used as boxes alone: a dense graph transformer turns the N node embeddings into an N x N relation matrix. Card: A Modern Take on Visual Relationship Reasoning for Grasp Planning.

• How a slot enters. Learned object queries, as in DETR, give N node embeddings 2409.02035 §4.1. Three learned maps turn the nodes into an initial node state and two edge inputs. Pairwise concatenation of the two edge inputs gives an N x N x 2F tensor, which a learned map halves to F 2409.02035 §4.2.
• Graph layer. Each layer has S heads. Every pair of nodes gets a score per head, which is multiplied by an edge projection and summed over channels. A softmax over the first node index gives the attention, and the node and edge features are updated from it 2409.02035 §4.2, eq. 1-4. The authors argue this dense form is cheaper than message passing over sparse graphs 2409.02035 §4.2.
• What a slot outputs. Class and box, as in DETR. The relation output is the N x N logit matrix 2409.02035 §4.2.
• How a slot gets its target. One-to-one Hungarian matching of queries to objects, with a class loss that includes the empty class and a box loss on the matched objects 2409.02035 §4.3, eq. 5-7. The relation loss is a binary cross-entropy over the P x P matched sub-matrix, against the ground-truth dependency matrix 2409.02035 §4.3, eq. 8-9.

Reported behaviour (author-reported, D3GD benchmark, AP_rel unless noted):

• Synth Easy: 76.27 for D3G (DETR) against 73.29 for VMRN, which uses Mask-RCNN 2409.02035 tab. 3. Synth Hard: 43.99 against 42.49 tab. 3. Synth-to-real: 24.44 against 13.36 tab. 3.
• Detection mAP is not the best in these rows. On Synth Easy, D3G (DETR) reaches 83.22 against 86.96 for VMRN 2409.02035 tab. 3. The paper does not analyse this gap.
• The synth-to-real OR and OP values vary a lot, because no validation set is available to tune the score threshold 2409.02035 §6.3.
• On the standard VMRD benchmark with PascalVOC mAP, D3G reaches 96.50 and an IA of 78.33, against 96.40 and 75.33 for GGNN 2409.02035 tab. 6.

Roadside query fusion across vehicles (UniV2X)

UniV2X is an end-to-end planner that also uses a roadside unit's camera. The roadside does not send a dense feature map for the agents and lanes: it sends its tracked agent queries, its lane queries and one occupancy map with its flow. The vehicle aligns the roadside queries with its own, fuses the matched pairs and then plans. Card: End-to-End Autonomous Driving through V2X Cooperation.

• How a slot enters. A roadside agent query is a 256-dimensional feature with a tracking id and a reference point 2404.00717 §3.2. The vehicle advances it to its own timestamp with a predicted flow, which the paper gives as eq. 2, and moves its reference point into the vehicle frame with the relative pose 2404.00717 §3.3, eq. 2-3.
• How slots are matched. Vehicle and roadside queries are paired by Hungarian matching on the Euclidean distance of their reference points. Each matched pair is fused into the vehicle query, and each unmatched roadside query is appended to the vehicle's set 2404.00717 §3.3. This pairing happens at fusion time, on the vehicle, and is not a training target.
• Other slots. Roadside lane queries are concatenated with the vehicle's lane queries 2404.00717 §3.4. The roadside occupancy map is expanded over future steps with the flow, moved into the vehicle frame and combined with the vehicle's map by max 2404.00717 §3.5, eq. 1.
• Training. The fusion modules are trained in a stage of their own, with the tracking and mapping losses 2404.00717 eq. 5.

Reported behaviour (author-reported, DAIR-V2X, averaged over 2.5, 3.5 and 4.5 s for planning):

• Collision rate 0.34% for UniV2X, against 0.89% with no fusion and 0.83% for BEV feature fusion 2404.00717 tab. 2.
• The L2 error is not the best. UniV2X gives 3.48 m, against 3.44 m with no fusion and 3.30 m for BEV feature fusion 2404.00717 tab. 2.
• Transmission: 8.09e5 BPS for UniV2X, against 8.19e7 for BEV feature fusion 2404.00717 tab. 2.
• Car detection mAP is 0.295, against 0.268 for V2X-ViT and 0.243 for early fusion. AMOTA is 0.239, below 0.287 for V2X-ViT and 0.263 for late fusion 2404.00717 tab. 3.
• Latency of 500 ms: mAP 25.9 without flow prediction and 28.5 with it, against 29.5 at 0 ms 2404.00717 tab. 9. The released code has no flow module (see the card's Paper vs code), so this row cannot be reproduced from it.

Per-cell density queries beside waypoint queries (InterFuser)

InterFuser is an end-to-end driving model with one transformer over tokens from several cameras and a LiDAR bird's-eye-view image. Its decoder has three query groups: waypoint queries, one query per cell of a 20 x 20 density map, and one traffic-rule query 2207.14024 §3.2. Each cell covers 1 m, and the map is in the ego-vehicle frame, so it covers a 20 x 20 m area and not the whole scene 2207.14024 §3.1.

• How a slot enters. Every query gets a learned positional embedding. There is no reference point and no matching step: the cell's position in the grid fixes which target that query is trained on 2207.14024 §3.2.
• How a cell is read. Each density query goes through its own three-layer MLP to seven channels: object existence, a 2-D offset from the cell centre, box size, heading and velocity 2207.14024 §3.1, §3.2.
• Training. The loss has three terms: L1 on the waypoints, a density-map term and a binary cross-entropy term for the traffic outputs 2207.14024 eq. 2, app. A. The density term averages its error over positive and negative labels separately, so rare objects weigh as much as common ones 2207.14024 app. A.
• Who reads the map. The map is not a detection list. A cell counts as an object if its existence probability exceeds 0.9, or exceeds 0.5 and is a local maximum among its neighbours; the second rule avoids counting one object several times 2207.14024 §3.3, app. G. A separate controller uses these objects, the traffic outputs and the waypoints to choose a speed 2207.14024 §3.3, app. B.

Reported behaviour (author-reported, CARLA only):

• Public CARLA leaderboard, accessed June 2022: driving score 76.18, route completion 88.23 and infraction score 0.84, ranked first 2207.14024 tab. 1.
• Town05 Long: driving score 51.6 for the full model. Removing the safety controller gives 43.9, and replacing transformer fusion with concatenated inputs gives 21.5 2207.14024 tab. 3.

Card: Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer. The released density head, label encoding and controller differ from the paper; see the card's Paper vs code before reusing any of them.

Action segment queries with temporal deformable attention (TadTR)

TadTR finds action instances in an untrimmed video. Its learned action queries are shared by all videos. Each one outputs a class and a segment. The encoder has 2 layers and the decoder has 4, both of width 256 2106.10271 §III-A, §IV-A. Card: End-to-end Temporal Action Detection with Transformer.

• How a slot enters. A linear layer and a sigmoid predict each query's first reference centre from its embedding. Each decoder layer runs self-attention across the queries, then samples the encoder memory around the reference with temporal deformable cross-attention, using 4 sampling points for each of 8 heads 2106.10271 §III-A, §IV-A.
• What a slot outputs. After each layer, an FFN predicts a class probability and a segment, given as a normalised centre and length. Layer l refines the segment of layer l-1 with an offset, through a sigmoid and its inverse. The next layer's reference is the refined centre 2106.10271 §III-A, eq. 4, eq. 5.
• Actionness. A separate head scores each segment. The segment is widened by a factor of 1.5, pooled to 16 bins by temporal RoIAlign, and regressed to the largest IoU with the video's ground-truth actions 2106.10271 §III-A, §IV-A, tab. 11. The paper's detection score is the square root of the class probability times this value 2106.10271 §III-B.
• How a slot gets its target. Hungarian matching pairs each action query with at most one ground-truth action, and unmatched queries predict no action. No NMS is applied 2106.10271 §III-B. The paper's default classification cost is cross-entropy; focal loss is used in its improved setting 2106.10271 §III-B, §IV-A.

Reported behaviour (author-reported):

• HACS validation, average mAP: 30.83 for TadTR against 27.48 for G-TAD. Per video, 19 ms against 941 ms, and 0.1 GFLOPs against 45.7 2106.10271 tab. 2.
• HACS ablations: removing the encoder gives 26.94, replacing temporal deformable attention with dense attention gives 13.58, removing the actionness head gives 28.51 and removing segment refinement gives 27.65, against 30.83 for the full model 2106.10271 tab. 4.
• Encoder choice on HACS, without NMS: the Transformer encoder gives 30.83 average mAP, a two-layer 1-D CNN gives 27.95 2106.10271 tab. 5.

When to use which

Implementation sketch

Input: image features F (S tokens x d) from a CNN+encoder or a ViT
Params: Q (num_slots x d), learned

DETR-style (queries as positional encodings, cross-attention):   [arxiv:2005.12872 §3.2, app. A.3]
  h = zeros(num_slots, d)
  for each decoder layer:
      h = SelfAttn(q = h + Q, k = h + Q, v = h)          # slots see each other
      h = CrossAttn(q = h + Q, k = F + pos_F, v = F)     # slots read the image
      h = FFN(h)                                          # residual + dropout + post-LayerNorm
      out_l = heads(LayerNorm(h))                         # shared heads, every layer
  targets: Hungarian matching per image (see bipartite-matching)

Segmenter-style (queries as tokens, joint self-attention):   [arxiv:2105.05633 §3.2, eq. 4]
  x = concat(F_patches, Q)                    # Q: one embedding per class, num_slots = K
  for each of M layers (M = 2):
      x = TransformerBlock(x)                 # pre-norm self-attention + MLP
  p, c = split(x)                             # patches (N x d), classes (K x d)
  masks = normalize(p) @ c^T                  # N x K
  reshape to grid, bilinear upsample, per-pixel cross-entropy with target class k for slot k

Conditional DETR cross-attention, per decoder layer:   [arxiv:2108.06152 §3.3, eq. 3-6]
  s   = sigmoid(FFN_ref(O))            # reference point, from object query O
  p_s = sine_embed(s)                  # same layout as the key pos. embedding
  lam = FFN_scale(f_prev)              # per-channel scale from previous layer
  p_q = lam * p_s                      # conditional spatial query
  q   = concat(c_q, p_q) per head      # content and spatial halves
  k   = concat(c_k, p_k) per head
  h   = CrossAttn(q, k, v = c_v)       # logit = c_q.c_k + p_q.p_k

Co-DETR training step (Deformable-DETR-style base):   [arxiv:2211.12860 §3.2, §3.3]
  memory  = encoder(backbone(images))
  pyramid = memory + [conv_s2(memory[-1])]        # one extra coarse level
  main    = decoder(Q_learned, memory)            # Hungarian loss per layer
  for head in aux_heads:                          # e.g. ATSS, Faster R-CNN
      preds = head(pyramid)
      pos, boxes, targets = head.assign(preds, gt)   # one-to-many, eq. 1
      q   = Linear(PE(boxes)) + Linear(gather(pyramid, pos))   # eq. 4
      out = decoder(q, memory)                    # own group, no matching
      loss += lambda_1 * detr_loss(out, targets)  # eq. 5
      loss += lambda_2 * head.loss(preds, pos)    # eq. 2, 3
  inference: run decoder(Q_learned, memory) only

CLTR matching, one image   [arxiv:2202.13065 §3.3, eq. 2-3]
  cost = L1(gt_point, pred_point) + L1(gt_knn, pred_knn) - pred_conf
  match = hungarian(cost)                     # one-to-one, unmatched = background
  loss  = focal(pred_conf, labels) + 2.5 * L1(pred_point[match], gt_point)

TrackFormer, one frame   [arxiv:2101.02702 §3.2, app. A.3, app. A.4]
  Q_track = [t.embedding for t in tracks]     # last decoder output per track
  h = Decoder(concat(Q_track, Q_object), memory = Encoder(frame))  # self-attn over all
  for each object-query output with score > 0.4:  start a new track
  for each track with score < 0.4:                 end the track

DETR3D, one decoder layer   [arxiv:2110.06922 §3.3, eq. 1-4]
  c   = Phi_ref(q)                          # 3D reference point per query
  uv  = project(T_m, [c, 1]) for each camera m, mapped to [-1, 1]
  f   = mean of bilinear(F_level,m, uv) over visible samples
  q   = q + f                               # residual, eq. 4
  q   = MultiHeadSelfAttn(q)                # queries exchange information
  box, class = heads(q)                     # eq. 5, loss on every layer

VisTR, one clip of T frames   [arxiv:2011.14503 §3.1, §3.2]
  feats = CNN(frames)                       # T frame maps, in time order
  mem   = Encoder(flatten(conv1x1(feats)) + PE3d(T, H, W))
  out   = Decoder(Q, mem)                   # Q: n*T learned queries, one pass
  seq   = out.reshape(T, n, d)              # slot t*n + k is instance k in frame t
  for k in range(n):
      cost[k] = sum over frames of class and box terms for sequence k
  pairs = hungarian(cost)                   # one-to-one, over sequences
  loss  = per pair: class NLL, box loss and mask loss, each averaged over T

TopoPoint, one point-lane layer   [arxiv:2505.17771 §3.4, eq. 3-16]
  # Merged self-attention with geometric bias from last layer's predictions
  bias = [[0, M_pl], [M_pl.T, M_ll]]          # eq. 4-7, paper only
  h    = LayerNorm(SelfAttn(concat(P, L), bias))   # eq. 3, eq. 8
  # Each set reads the BEV map at its own learned reference points
  P = FFN(DeformAttn(h_P, ref_P, bev))        # eq. 9-10
  L = FFN(DeformAttn(h_L, ref_L, bev))
  # Point-lane graph: messages in both directions
  P, L = PLGCN(P, L, A_pl)                    # eq. 11-13
  # Lane-lane and lane-traffic graphs, then a linear map to lane width
  L = Linear(concat(GCN_ll(L) + L, GCN_lt(T) + L))   # eq. 14
  P, L = PLGCN(P, L, A_pl)                    # eq. 15-16

Mask3D, one refinement step   [arxiv:2210.03105 §3, eq. 1-3]
  inst  = MLP(h)                              # instance feature per query
  masks = inst @ F0^T                         # per-voxel logits at full resolution
  allow = resize(sigmoid(masks_prev), scale) >= 0.5   # voxels inside the last mask
  h     = CrossAttn(q = h, k = F_s, v = F_s, bias = where(allow, 0, -inf))
  h     = SelfAttn(h)                         # queries see each other
  h     = FFN(h)
  # run for 4 scales, coarse to fine, 3 times over: 12 steps   [§3]

D3G relation step, one image   [arxiv:2409.02035 §4.2, eq. 1-4]
  # N object queries from the DETR decoder, shape (N, F)
  nodes = decoder(Q_learned, memory)
  # pairwise concatenation of two learned edge maps, shape (N, N, F)
  e = pair_concat(M_e1(nodes), M_e2(nodes))
  for layer in range(L):              # L = 2 in the paper's chosen setting
      nodes, e = DenseGraphLayer(nodes, e)   # S heads, attention over pairs
  rel = Linear(e)                     # (N, N) relation logits
  # train: BCE on the P x P sub-matrix of matched queries vs ground truth

UniV2X agent fusion, one frame   [arxiv:2404.00717 §3.3, eq. 2-3]
  # latency compensation by predicted flow (paper only, not in released code)
  inf = advance_by_flow(inf, dt)
  inf.ref = to_vehicle_frame(inf.ref, pose)   # reference points, eq. 3
  cost = cdist(veh.ref, inf.ref)              # Euclidean distance
  veh_i, inf_i = hungarian(cost)              # one-to-one pairs
  veh[veh_i] = fuse(veh[veh_i], inf[inf_i])   # matched pairs update vehicle
  veh = concat(veh, inf[unmatched(inf_i)])    # roadside queries not matched

Notes:

• DETR's decoder layers are post-norm (residual, dropout, LayerNorm) 2005.12872 eq. 6; the first self-attention of the first layer can be skipped, since all slots start at zero 2005.12872 app. A.3.
• DETR's inference keeps every slot: a slot predicting \varnothing is scored with its second-best class, which adds 2 AP over dropping such slots 2005.12872 §4.
• Segmenter's released code differs from eq. 4 2105.05633 eq. 4: it projects decoder input, applies a LayerNorm after the blocks, uses separate learned projections for patches and class embeddings, L2-normalises both (so masks are cosine similarities) and applies a LayerNorm over the class dimension before upsampling, with softmax only inside the loss. The paper's own order (softmax, then layer norm) is internally inconsistent; the card's "Paper vs code" section has the code references.
• Conditional DETR: concatenate content and spatial parts within each head, not across the full 256-d vector, or the two halves land in different heads 2108.06152 §3.3. The attention scale then follows the concatenated per-head dimension. The first decoder layer is handled differently, and the released code predicts the reference point once and reuses it in every layer; see the Conditional DETR card, Paper vs code.
• Co-DETR: the paper sends only positive boxes into the auxiliary groups 2211.12860 §3.5. The released code pads each group with negative anchors; see the Co-DETR card, Paper vs code.
• Co-DETR: eq. 6 puts \lambda_2 \mathcal{L}^{enc} inside the sum over the L decoder layers 2211.12860 eq. 6. Implementing the auxiliary encoder loss once, at weight 2, gives the encoder only 1/L of the weight the formula describes.
• CLTR's kNN value: the paper uses k=4 neighbours 2202.13065 §4.1, eq. 3. The released code averages three, because it drops the point itself from its four-point query. The paper's sum does not say whether the point itself counts; see the CLTR card, Paper vs code.
• CLTR's training crops: the paper observes that its patches hold fewer than 500 people 2202.13065 §5.4.1. The released code also drops training crops whose point count reaches the query count, a rule the paper does not state.
• OrientedFormer: the Gaussian encoding uses 2^(z−r) and 2^(z+r) as printed in eq. 13, which does not match the paper's own z and r definitions 2409.19648 eq. 11, eq. 13. Follow the definitions, and rotate about the box centre, not the image origin as eq. 14 reads literally 2409.19648 §3.5.
• OrientedFormer's self-attention bias. The paper adds the bias to the logits as is 2409.19648 eq. 10. The card's Paper vs code section reports a learnable per-head scale in the released code, which the paper does not describe. Implementing the paper alone gives different attention behaviour.
• PolarFormer's reference points are 3-D in the paper 2206.15398 §3.4. The released deformable sampling uses only azimuth and radius, and the height is refined by regression alone; see the card's Paper vs code.
• DeepInteraction's decoder depth: the paper says five cascaded decoder layers 2208.11112 §4.2. The released code has one transformer decoder layer and four interaction layers. Follow the code if you want to reproduce the reported numbers.
• BinsFormer's loss placement. The paper applies the depth and scene losses to every decoder layer, weighted by scale 2204.00987 §3.2.5, eq. 4. The card's Paper vs code section reports that the released Swin config supervises three layers and uses a scene-loss weight of 1e-2, not the paper's 1e-3.
• IAT's query initialisation. The paper starts the global queries at zero 2205.14871 §3.2. The card's Paper vs code section reports that the released code starts them at one.
• TrackFormer's track count changes from frame to frame, but the decoder stacks track queries per image. The paper pads the batch with false-positive queries 2101.02702 §4.2. The released code gives every image the same count per batch instead, so a per-image random count needs padding to batch. See the TrackFormer card, Paper vs code.
• TrackFormer's identity matching must be forced. Track-query outputs go to the ground truth of their own track, and false-positive track queries never match 2101.02702 §3.3. A plain Hungarian match over all slots breaks identity.
• TrackFormer's previous-frame pass: the paper does not say whether gradients flow through it. The released code runs it without gradients by default 2101.02702 §3.3; see the card's Paper vs code.
• DETR3D's aggregation: the paper's eq. 4 averages visible samples, with a denominator that has an unspecified ε 2110.06922 eq. 4 2110.06922 §3.3. The released code uses learned sigmoid weights per camera, level and point, and does not normalise 2110.06922 §3.3; see the DETR3D card, Paper vs code. Following the equation can give a different feature scale.
• DETR3D's reference points: the released code detaches them between layers, so no gradient flows from later layers into earlier reference points. The paper does not mention this 2110.06922 eq. 1; see the card.
• VisTR's slot count is tied to the clip length: N = n·T, with n = 10 and T = 36 2011.14503 §4.1. The released matcher hard-codes 10 slots per frame in frame-major order matcher.py, so changing the query count needs a change there too. Its inference also takes only the first 36 frames of a video. See the VisTR card, Paper vs code.
• TopoPoint: the paper adds the geometric matrices to the self-attention logits 2505.17771 §3.4, eq. 7. The released code builds no such mask and uses the matrices only as graph adjacencies. Its point-to-lane refinement is commented out. Following the equations alone gives a different model from the released one; see the TopoPoint card, Paper vs code.
• Mask3D's query count: the paper uses 100 in every experiment 2210.03105 app. §1. The released ScanNet and ScanNet200 scripts use 150 and the STPLS3D script 160. Check the release scripts before reproducing a table row; see the Mask3D card, Paper vs code.
• Mask3D's top-k: the paper computes confidences for queries by their dominant class and describes no top-k step 2210.03105 §3.1. The released code keeps the top query-class pairs, so one query can yield several instances. See the Mask3D card, Paper vs code.
• Mask3D's empty attention masks: the paper does not say what happens when a query's mask covers no voxel. The released code unmasks all voxels for that query. See the Mask3D card, Paper vs code.
• D3G's relation loss scale: eq. 9 sums the BCE over all P x P pairs 2409.02035 eq. 9. The released code divides that sum by the number of matched objects, per image [see the D3G card, Paper vs code]. A literal implementation of eq. 9 trains with a different loss scale.
• D3G's edge softmax: check the axis against the code before implementing eq. 2. The card's Paper vs code section gives the lines; the paper's text does not settle it.
• D3G's loss is not only on the last layer in the code. The released code also computes the relation loss on each auxiliary decoder layer [see the D3G card, Paper vs code]; whether those terms are summed depends on the trainer, which the repository does not include.
• UniV2X's latency step: eq. 2 is the only description of query flow, and the released code has no flow module 2404.00717 eq. 2. Plan without latency compensation unless you add the module yourself.
• UniV2X's fusion gate: the paper describes a Hungarian match on Euclidean distance 2404.00717 §3.3. The released code adds a per-axis distance gate and a score gate for vehicle queries [see the UniV2X card, Paper vs code]. A paper-only implementation can match pairs that the code would reject.

Contradictions

Two papers diagnose slow DETR convergence differently. Neither tests the other's explanation, and this page does not resolve them.

• Conditional DETR attributes it to decoder cross-attention depending heavily on content embeddings; object queries give only a generic spatial prior 2108.06152 §1.
• Co-DETR attributes it to sparse one-to-one supervision: with few positive queries, the encoder features separate foreground from background poorly, and decoder cross-attention learns inefficiently 2211.12860 §1, §3.4.

Their baselines also differ, so their numbers are not directly comparable. Conditional DETR reports 50, 75 and 108 epochs 2108.06152 tab. 1. Co-DETR's table reports Conditional DETR-C5 at 36 epochs, 39.4 AP without an auxiliary head 2211.12860 tab. 2. The Segmenter, BinsFormer, IAT, PubTables-1M, CLTR, PolarFormer, DeepInteraction and Mask3D results measure different tasks or use different datasets, so they are not comparable to the detection tables or to each other. Mask3D's 3D instance tables compare against bottom-up methods such as SoftGroup and HAIS 2210.03105 tab. 1. TrackFormer's MOT and MOTS numbers measure tracking, and DETR3D's nuScenes rows compare against camera-only baselines (CenterNet, FCOS3D, Mono3D, DHNet, PGD, DD3D); neither set of tables includes PolarFormer or DeepInteraction 2110.06922 tab. 1 2110.06922 tab. 2 2101.02702 tab. 1. VisTR's video table compares against tracking and mask-propagation methods, not against the detectors above 2011.14503 tab. 2. Learned slots (DETR, CLTR, PolarFormer) and heatmap-started slots (DeepInteraction) are not compared against each other in any of these papers, so this page cannot rank them.

The lane-topology papers report TopoMLP differently. TopoMLP reports OLS 38.2 and TOP_ll 7.2 on subset A with ResNet-50 2310.06753 tab. 1. The survey repeats TOP_ll 7.2 with DET_l 28.3 2504.01989 tab. 3. TopoFG lists TopoMLP's official model at OLS 44.1 and TOP_ll 21.7, evaluated with the released model 2511.12590 tab. 1. TopoPoint lists OLS 44.1 and TOP_ll 21.6 2505.17771 tab. 1, but copies these rows from the TopoLogic and TopoFormer papers, except DET_p, which it computed. The survey gives TopoLogic DET_l 34.4 and TOP_ll 23.4 on subset A 2504.01989 tab. 3, against 29.9 and 23.9 in TopoPoint 2505.17771 tab. 1. The survey does not say whether its numbers were re-run or copied. The settings differ, and this page does not resolve these gaps.

D3G's own tables give the S=2, L=2 Easy AP_rel as 76.77 in tab. 5, and as 76.27 in tab. 3 and tab. 4 2409.02035 tab. 3 2409.02035 tab. 4 2409.02035 tab. 5. The other columns agree. The numbers in this page use tab. 3, and the paper does not say which one is right.

TadTR's ActivityNet headline of 36.75 mAP uses TSP features and external classifiers. Its self-contained rows use two-stream features and reach 28.21 (TadTR) and 29.90 (TadTR*) 2106.10271 tab. 3. On THUMOS14, the 56.7 average is the starred setting with IBIF and focal loss, and the default TadTR row is 49.3 2106.10271 tab. 1. This page does not merge the two settings. TadTR's temporal-action tables measure a different task from the detection, segmentation and lane tables above, so they are not comparable to them.

InterFuser's CARLA results are closed-loop driving scores from a simulator, and its ablations use the Town05 Long route. They are not comparable to the detection, segmentation, lane or depth tables above 2207.14024 tab. 1, tab. 3.

Papers

• End-to-End Object Detection with Transformers: introduces learned object queries decoded in parallel and assigned to objects by Hungarian matching.
• Conditional DETR for Fast Training Convergence: adds a per-slot spatial query, predicted from a reference point, so that cross-attention separates position from content.
• DETRs with Collaborative Hybrid Assignments Training: trains extra one-to-many heads and query groups built from their positives; inference is the unchanged DETR.
• Segmenter: Transformer for Semantic Segmentation: uses one learned query per class, processed jointly with ViT patch tokens, to produce class masks by scalar product.
• PubTables-1M: Towards comprehensive table extraction from unstructured documents: applies unchanged DETR, with 15 and 125 learned queries, to table detection, structure and functional analysis on a dataset with canonicalised structure labels.
• An End-to-End Transformer Model for Crowd Localization: regresses one point and one confidence per learned query, and matches queries to heads with a kNN-context cost.
• TrackFormer: Multi-Object Tracking with Transformers: carries each track's previous output embedding into the next frame as a track query, next to learned object queries for new objects.
• Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers: propagates relevance through DETR's decoder to show which image regions each detected query used.
• OrientedFormer: gives each query an oriented box as its positional part, encoded as a Gaussian, with a Wasserstein self-attention bias and rotated cross-attention sampling.
• PolarFormer: Multi-camera 3D Object Detection with Polar Transformer: decodes learned queries with refined 3-D reference points on a polar BEV map built from six cameras.
• DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries: decodes 900 learned queries to 3D reference points, projects them into six cameras and samples image features there, with no NMS.
• TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoning: decodes 300 lane queries into Bezier control points and pairs them with traffic queries in small MLP heads that predict topology.
• DeepInteraction: 3D Object Detection via Modality Interaction: starts 200 slots at BEV heatmap peaks and refines them by alternating image and LiDAR RoI pooling.
• IAT, a 90K-parameter transformer for light and exposure correction: uses ten global queries, without self-attention, to predict one colour matrix and one gamma per image.
• BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation: uses bin queries and a scene query in a Transformer decoder to generate per-image depth bins.
• End-to-End Video Instance Segmentation with Transformers: decodes n·T learned queries over a 36-frame clip, and matches each per-instance sequence of slots to a ground-truth instance as a unit.
• Fine-Grained Representation for Lane Topology Reasoning: describes each lane with 11 ordered queries, seeded by a BEV lane mask and point-order priors, and pairs only the first and last query of each lane.
• A Concise Survey on Lane Topology Reasoning for HD Mapping: surveys lane-topology methods and the scores reported for them on Topo-Boundary and OpenLane-V2. It has no method of its own.
• TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Driving: adds endpoint queries, merged with lane queries in one self-attention, and at inference pulls connected lane ends to a shared detected point.
• Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation: uses four learned latent tokens and a 32-token Q-Former for a navigation policy. These are not object slots.
• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation: predicts 100 non-parametric instance queries, placed at sampled scene points, and refines them with masked cross-attention to per-instance masks trained by Hungarian matching.
• A Modern Take on Visual Relationship Reasoning for Grasp Planning: uses DETR-style object queries as nodes of a dense graph transformer that predicts a dependency matrix between objects.
• End-to-End Autonomous Driving through V2X Cooperation: fuses the roadside unit's tracked agent queries into the vehicle's queries by Hungarian matching, before an end-to-end planner.
• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer: decodes waypoint queries, 400 density-map cell queries and one traffic query from one transformer over camera and LiDAR tokens, and feeds the density map to a safety controller.
• End-to-end Temporal Action Detection with Transformer: learned action queries each predict one segment and class, refined layer by layer with temporal deformable attention and scored by an actionness head, with no anchors or NMS.