Efficient vision backbones

Overview

An efficient vision backbone is a feature extractor for classification, detection or segmentation that aims to be fast on a target device at a given accuracy. The common approach has been to minimize FLOPs (multiply-adds), mostly with depthwise or group convolutions plus data shuffling, concatenation or pooling; mobile vision transformers add modified attention on top 2303.03667 §1.

The FasterNet paper argues that FLOPs are the wrong target on their own. Latency is FLOPs divided by FLOPS, the operations per second the hardware actually achieves 2303.03667 eq. 1. Depthwise and group convolutions do little arithmetic per byte of memory traffic, so they reach low FLOPS and their FLOPs savings only partly turn into speed 2303.03667 §3.1, tab. 1. One example from the paper: CycleMLP-B1 has about half the FLOPs of ResNet50 but takes 116.1 ms against 73.0 ms on CPU 2303.03667 §1, tab. 3.

The idea every variant shares: choose operators and layouts by measured latency or throughput on the deployment hardware, not by operation counts.

This page draws on twenty-nine cards. Run, Don't Walk (FasterNet) is the main source for operator and block design. ResT is a multi-scale transformer whose attention is made cheap by a strided depth-wise conv on keys and values. YOLO-LITE is a detector found by trial and error for computers without a GPU, and is a measured example rather than a general backbone. Rethinking Vision Transformers for MobileNet Size and Speed (EfficientFormerV2) is a hybrid transformer sized by a search on phone latency, and MambaVision is a convolution-then-hybrid backbone with Mamba-style and attention blocks. EfficientViT is a transformer with one attention layer per block, tuned for measured throughput. TinyViT is a small hybrid transformer family whose accuracy comes mostly from distillation during pretraining rather than from its block design. Once-for-All trains one network whose sub-networks are specialized per device without retraining, and is a way to get many backbones from one training run. Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone uses an off-the-shelf MobileNet backbone on a phone. YOLO5Face is a face detector whose smallest models swap in ShuffleNetV2 backbones 2105.12931 §3.7, and HarDNet-MSEG is a polyp segmentation network whose encoder is the HarDNet68 backbone 2101.07172 §3.1. Both are task-specific: their numbers are for face detection and polyp segmentation, not ImageNet classification. YOLOv10 is a detector rather than a backbone study, included because it targets real-time latency without a non-maximum suppression step. The edge deep learning survey gives a device-tier taxonomy and reviews compression routes, but runs no experiments of its own. EfficientFormer is a pure transformer whose early stages pool instead of attend, with attention only in the last stage, sized by a latency-driven search. RepViT is a pure CNN that borrows its layout choices from small vision transformers. A survey of modern object detection models collects author-reported detector and lightweight-network numbers from other papers, without rerunning them. Coordinate Attention is an attention block for mobile CNNs that keeps row and column position, and PVT v2 is a pyramid transformer whose attention cost grows linearly with the token count. XCiT replaces token-to-token attention with cross-covariance attention over feature channels, so its cost grows linearly with the number of patches 2106.09681 §3.1, §3.2. RTMDet is a real-time one-stage detector family built from 5×5 depth-wise blocks and a soft label assignment, with latency reported on one GPU with TensorRT 2212.07784 §3.1, tab. 2. HaloNet builds a ResNet-like backbone from blocked local self-attention. RTMPose is a real-time top-down pose estimator whose CSPNeXt backbone comes from RTMDet and whose head predicts coordinates as bin distributions. LocalMamba changes the scan order of a Vision Mamba backbone so that small windows are read together. EdgeNeXt is a mobile hybrid whose SDTA block runs channel-wise attention, so its attention cost grows linearly with the number of pixels 2206.10589 §3. Lite-HRNet is a small high-resolution network for pose and segmentation whose conditional channel weights replace the costly 1×1 convolutions of a shuffle block 2104.06403 §3.2. The YOLO review covers the YOLO detector family from v1 to v12 without running experiments, so it adds caveats rather than numbers. EdgeViT is a light transformer whose blocks mix neighbours with convolutions and run attention over one sampled token per window 2205.03436 §3.2. HiLo, used in LITv2, splits each attention layer's heads between windowed attention and attention to pooled keys 2205.13213 §4.1. Lightweight OpenPose is a bottom-up pose network built for CPUs, a task-specific entry like YOLO-LITE 1811.12004 §4.1. The other architectures named below (MobileNetV2, ShuffleNetV2, GhostNet, MobileViT, ConvNeXt, Swin and others) have no cards here. They appear only in the source papers' tables, and this page uses only those measurements.

Variants

Operator choice for spatial mixing

The FasterNet paper benchmarks four 3×3 spatial operators, each as 10 stacked layers on four feature-map shapes 2303.03667 §4.1, tab. 1. Averages over the four shapes:

• Regular convolution reaches the highest FLOPS but costs the most FLOPs 2303.03667 tab. 1.
• Depthwise convolution (DWConv) has the fewest FLOPs, h \times w \times k^2 \times c. Networks widen it to recover accuracy (×6 in MobileNetV2 inverted residuals), and then its memory access, \approx h \times w \times 2c', exceeds that of a regular conv 2303.03667 §3.1, eq. 2, eq. 3. It reaches the lowest FLOPS on all three devices 2303.03667 tab. 1.
• Group convolution (GConv) lies between the two 2303.03667 tab. 1.
• Partial convolution (PConv) runs a regular k \times k conv on a contiguous block of c_p = r \cdot c channels (the first or the last ones) and passes the other channels through unchanged. With r = 1/4 it has 1/16 the FLOPs and about 1/4 the memory access of a regular conv 2303.03667 §3.2, eq. 4, eq. 5. Its FLOPS are 10.5×, 6.2× and 22.8× those of DWConv on GPU, CPU and ARM 2303.03667 §4.1.

PConv is always followed by a 1×1 pointwise conv. The pair covers a T-shaped receptive field (full k \times k window on c_p channels, centre position on all channels) at lower cost than a direct T-shaped conv 2303.03667 §3.3, eq. 6, eq. 7. Trained to imitate the first 3×3 conv of each ResNet50 stage, PConv+PWConv reaches a lower average test MSE (0.0123) than GConv+PWConv (0.0133) and DWConv+PWConv (0.0148) 2303.03667 §4.2, tab. 2. In stage 1 GConv+PWConv is slightly better (0.0065 vs 0.0069) 2303.03667 tab. 2.

FasterNet: a plain CNN built on PConv

Four stages at widths c, 2c, 4c, 8c. The stem is a 4×4 stride-4 conv with BN, and the stages are joined by 2×2 stride-2 merging convs with BN 2303.03667 §3.4, fig. 4, tab. 8. Each block is PConv (3×3, r = 1/4) → 1×1 conv expanding to 2d → BN → activation → 1×1 conv back to d, plus a residual shortcut. That is the block's only normalization and activation 2303.03667 §3.4, tab. 8. Most blocks sit in stage 3, because later stages reach higher FLOPS 2303.03667 §3.4, tab. 1. BN is used rather than LayerNorm because it can be folded into the adjacent conv at inference (see batchnorm-folding) 2303.03667 §3.4. Six sizes, T0 to L, span 0.34 to 15.5 GFLOPs and 71.9% to 83.5% ImageNet-1k top-1 2303.03667 tab. 3, tab. 8. Card: Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks.

Hybrid transformer sized by phone latency: EfficientFormerV2

EfficientFormerV2 starts from a fast transformer and changes its blocks. A depthwise 3×3 conv moves into the feed-forward blocks, and global self-attention appears only in stages 3 and 4 of a four-stage network 2212.08059 §3.1, §4.1. Its sizes come from a search that scores each candidate on parameter count and iPhone 12 latency together, through one combined score 2212.08059 §4.2.1. The search loop starts from the largest sub-network and removes the action that costs the least accuracy per unit of score gained 2212.08059 §4.2.3. Author-reported ImageNet-1K top-1 with their recipe: S0 reaches 75.7% at 3.5M parameters and 0.9 ms on the iPhone 12 neural engine, and S1 reaches 79.0% at 6.1M parameters and 1.1 ms 2212.08059 tab. 2. Card: Rethinking Vision Transformers for MobileNet Size and Speed. Reproducing the search needs a latency table for your own device, because the iPhone 12 table is not released 2212.08059 §4.2.1.

Convolutional early stages, Mamba-attention hybrid late stages: MambaVision

MambaVision is a four-stage hierarchy. Stages 1 and 2 are residual convolution blocks. Stages 3 and 4 use a Mamba-style mixer in the first half of their layers and self-attention in the second half 2407.08083 §3.1, §3.2.2. The mixer splits its input in two: a selective scan runs on one half, a convolution with a SiLU activation on the other, and the two are concatenated 2407.08083 eq. 7. In the authors' ablation, attention in the last half of each late stage gives the best result: 82.3% top-1, against 81.3% for random placement and 81.9% for attention in only the last quarter 2407.08083 tab. 5. Reported top-1 is 82.3% for Tiny (31.8M parameters, 6298 img/s) and 84.2% for Base 2407.08083 tab. 1. The classification throughput is measured on one A100 at batch 128 2407.08083 tab. 1. Card: MambaVision. The official code is non-commercial, so implement from the paper and the card rather than copying code.

Small hybrid transformers pretrained by distillation: TinyViT

TinyViT is a four-stage hierarchy. Stage 1 uses MBConv blocks, and stages 2 to 4 use window attention, with a 3×3 depthwise conv between attention and the MLP 2207.10666 §3.2, app. A. Its three sizes, 5M, 11M and 21M parameters, come from progressive model contraction: candidates are made by changing widths, depths, window sizes and expansion ratios, and the most accurate candidate that meets the size and throughput limits is kept 2207.10666 §3.2. The paper does not state the search budget 2207.10666 §3.2.

The main contribution is the pretraining recipe. A large teacher, CLIP-ViT-L/14 in the main runs, is run once over the augmented ImageNet-21k views. For each view the pipeline stores the top-K probabilities with their class indices (K = 100 on ImageNet-21k) and one seed that replays the augmentation. The student trains on these stored targets, with no teacher in memory and no ground-truth labels 2207.10666 §3.1, eq. 1, eq. 2, §5.1, §5.2. For small models this adds 2.1 to 2.2 points over plain ImageNet-21k pretraining (DeiT-Ti, DeiT-S, Swin-T) 2207.10666 tab. 2. Author-reported ImageNet-1k top-1 with distillation: 80.7% for 5M, 83.2% for 11M and 84.8% for 21M at 224px 2207.10666 tab. 4. Stored labels for ImageNet-21k take 481 GB at K = 100 over 90 epochs 2207.10666 §5.2. Card: TinyViT: Fast Pretraining Distillation for Small Vision Transformers.

Off-the-shelf MobileNet inside a detector, run on a phone

The road-damage paper does not design a backbone. It trains two standard single-shot detectors at 300×300 input, SSD Inception V2 and SSD MobileNet, because the goal is to run on a smartphone 1801.09454 §4, §5. Its phone timing is the only one on this page taken on a phone rather than a benchmark CPU or ARM board. SSD MobileNet takes 30.6 ms per image on a GPU (GRID K520) and 1500 ms on a Nexus 5X, against 63.1 ms for SSD Inception V2 on the same GPU 1801.09454 tab. 3. The paper calls 1.5 s per frame real-time for one photo per second while driving 1801.09454 §5. The card does not state which MobileNet width is used, whether its backbone is pretrained, or the batch size 1801.09454 §4.1.2. Its per-class results are recall and precision, not mAP, and they compare two detectors on one dataset, so they are not a general backbone ranking 1801.09454 tab. 2. Card: Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone.

Hand-searched 7-layer detector for computers without a GPU: YOLO-LITE

YOLO-LITE is a detector rather than a backbone study. The authors start from Tiny-YOLOv2 and vary layers, filters and input size, train each variant on PASCAL VOC, and keep the best one as the base for the next round 1811.05588 §3.1. The chosen Trial 3 without batch norm has seven convolutions (16 to 256 filters, then a 125-filter 1×1 output) and five 2×2 max-pools 1811.05588 tab. VII. Author-reported on VOC: 33.57% mAP at 21 FPS on a Dell XPS 13 without GPU, against 40.48% and 2.4 FPS for Tiny-YOLOv2 on the same laptop 1811.05588 tab. III, §2.2. Removing batch norm cost 1.0 mAP point in that trial and raised speed from 9.5 to 21 FPS 1811.05588 tab. III. Each configuration was run once, and no variance is reported 1811.05588 tab. III. The retrained COCO 2014 model reaches 12.26% mAP at 21 FPS 1811.05588 tab. V. The paper says its accuracy limits real applications, and it is the authors' own claim 1811.05588 §7. Speed comes from one laptop and one live-demo script 1811.05588 §3.1. Card: YOLO-LITE: A Real-Time Object Detection Algorithm Optimized for Non-GPU Computers.

Pyramid transformer with reduced-key attention: ResT

ResT follows the ResNet layout: a stem of three 3×3 convs (strides 2, 1 and 2), then four stages with widths C, 2C, 4C and 8C, giving a feature pyramid for detection and segmentation 2105.13677 §2, §2.3 2105.13677 tab. 1. Its attention keeps queries at full resolution, but computes keys and values from a token grid shrunk by a strided depth-wise conv, with reduction 8, 4, 2 and 1 across the stages 2105.13677 §2.2, tab. 1. The heads are mixed by a 1×1 conv and instance norm, which the paper says restores head diversity 2105.13677 §2.2, eq. 4. On the simple ResT-Lite recipe, removing both drops top-1 from 72.88% to 71.72% 2105.13677 tab. 7. Position comes from a per-channel gate, x times a sigmoid of a 3×3 depth-wise conv 2105.13677 §2.4, eq. 8. In the same ablation, no positional encoding gives 71.54% and the gate gives 72.88% 2105.13677 tab. 8. The paper says the gate lets the network accept any input size, but its ablation only runs at 224 px 2105.13677 §2.4, tab. 8. Author-reported on ImageNet-1k at 224 px, measured on one V100: ResT-Lite reaches 77.2% at 1246 img/s with 1.4 GFLOPs, and ResT-Small 79.6% at 1043 img/s with 1.9 GFLOPs 2105.13677 tab. 2. Card: ResT: An Efficient Transformer for Visual Recognition. Implement from the code as well as the paper, because it differs in several places (see Contradictions).

Pyramid transformer with pooled keys and values: PVT v2

PVT v2 keeps the four-stage pyramid of PVT v1 and changes three things 2106.13797 §3.1. Keys and values are average-pooled to a fixed 7×7 grid, so attention cost grows linearly with the number of tokens rather than quadratically 2106.13797 §3.2, eq. 1, eq. 2. Queries still come from every token, so the query and output projections still scale with the token count 2106.13797 §3.2. Neighbouring patches overlap in the downsampling step 2106.13797 §3.3. A 3×3 depthwise conv inside the feed-forward block takes the place of a fixed position table 2106.13797 §3.4, fig. 2. The authors say this supplies position, but no experiment in the paper varies the input size 2106.13797 §3.4, §4.4.1.

Author-reported ImageNet-1K top-1: B2 reaches 82.0% with 25.4M parameters and 4.0 GFLOPs, against 81.3% for Swin-T at 4.5 GFLOPs. B5 reaches 83.8% with 82.0M parameters and 11.8 GFLOPs 2106.13797 tab. 2. The linear variant B2-Li reaches 82.1% at 3.9 GFLOPs 2106.13797 tab. 2. On COCO with RetinaNet 1x, B2 reaches 44.6 AP and B4 reaches 46.1 AP 2106.13797 tab. 3. B2-Li gives 43.6 AP at 227.4 RetinaNet GFLOPs, against 44.6 AP at 290.7 for B2 2106.13797 tab. 6. In the ablation on PVT v1-Small, the overlapping embedding raises top-1 from 79.8 to 81.1, and the convolutional feed-forward block raises it to 82.0 2106.13797 tab. 6. The paper's tables list parameters, GFLOPs and accuracy, but no measured latency or throughput 2106.13797 tab. 2. Card: PVT v2: Improved Baselines with Pyramid Vision Transformer.

Light transformer with convolutional local mixing and sampled global attention: EdgeViT

EdgeViT replaces multi-head self-attention with a local-global-local block 2205.03436 §3.2. Depth-wise and pointwise convolutions first mix each token with its neighbours. Self-attention then runs over one sampled token per r×r window, so the global cost falls with the fourth power of the sample rate 2205.03436 §3.2, app. A.1. A transposed convolution spreads each sampled token's result back over its window 2205.03436 §3.2. The four stages use sample rates 4, 2, 2 and 1 2205.03436 §3.3. Author-reported ImageNet-1K top-1 with CPU latency on a Galaxy S21 (FP32, batch 1): XXS 74.4% at 32.8 ms and 0.6 GFLOPs, XS 77.5% at 54.1 ms, and S 81.0% at 85.3 ms 2205.03436 tab. 2. The same table has MobileNet-v3 0.75 at 73.3% and 23.0 ms, and EfficientNet-B0 at 77.1% and 52.1 ms 2205.03436 tab. 2. In the ablation on a model rescaled to about 0.5 GFLOPs, removing local aggregation costs 1.7 points, from 74.4% to 72.7% 2205.03436 tab. 4. The authors also report COCO and ADE20K results 2205.03436 tab. 5, tab. 6. Their efficiency metric divides top-1 by energy in mJ, which favours less accurate models, so the authors compare Pareto-optimal points 2205.03436 §1, tab. 3. Card: EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers. The released code covers ImageNet classification only, so the dense-task numbers have no code here (see the card).

One attention layer per block, tuned for throughput: EfficientViT

EfficientViT argues that attention is memory-bound. In Swin-T and DeiT-T, reshaping, addition, copying and normalization make up a large share of runtime 2305.07027 §2.1. Its block is a sandwich: one self-attention layer between feed-forward layers, with a depth-wise 3×3 conv before each feed-forward layer 2305.07027 §3.1, eq. 1. The attention is cascaded group attention. Each head gets its own channel split, and the previous head's output is added to that split before its attention is computed 2305.07027 §3.1, eq. 2, eq. 3. Six sizes, M0 to M5, change widths, depths and head counts 2305.07027 tab. 1. Author-reported ImageNet-1K top-1 on a V100: M0 reaches 63.2% at 27,644 img/s, and M5 reaches 77.1% at 10,621 img/s, against 75.2% at 7,560 img/s for MobileNetV3-Large 2305.07027 tab. 2. M5 has 12.4M parameters and 522M FLOPs 2305.07027 tab. 2. Its CPU result goes the other way for ONNX: on a Xeon CPU M5 runs at 56.8 img/s against 39.1 for MobileNetV3-Large, but in ONNX on the same CPU it runs at 62.5 img/s against 70.5 2305.07027 tab. 2. In the ablation (M4, 100 epochs), a Swin-style block in place of the sandwich costs 3.0 points of top-1 at similar GPU speed, and cascaded group attention gains 1.1 points over standard multi-head attention 2305.07027 tab. 6. Card: EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention.

Channel-wise attention with linear cost: XCiT

XCiT replaces the token-to-token attention matrix with a d \times d cross-covariance matrix between feature channels. Queries and keys are scaled to unit length over the tokens, and the matrix is built from them 2106.09681 §3.1, §3.2, eq. 1. Token self-attention costs O(N²d) time for N tokens, while the paper's cross-covariance form costs O(Nd²/h) time and O(d²/h + Nd) memory with h heads 2106.09681 §3.2. So the cost is linear in the number of patches, which is the property the variant is chosen for. Each block adds a local patch interaction (LPI), two depth-wise 3×3 convolutions with batch norm and GELU between them, because the attention alone does not mix neighbouring patches 2106.09681 §3.3. The paper reports that dropping the XCA layer costs 6.1 points of top-1 on XCiT-S12/16, and dropping LPI costs 1.2 points 2106.09681 tab. 4. Author-reported ImageNet-1k top-1 with distillation at 224 px: S12/16 reaches 83.3% at 4.8 GFLOPs and M24/16 reaches 84.3% at 16.2 GFLOPs 2106.09681 tab. 2. The paper's throughput table is the only speed evidence. On one V100 at batch 64 and 1024 px, XCiT-S12/16 runs 37 img/s at 7.3 GB, against 29 img/s at 26.9 GB for Swin-T and out of memory for DeiT-S 2106.09681 tab. D.5. At 224 px, however, ResNet-50 is faster than XCiT-S12/16 on that V100 (1171 against 781 img/s) 2106.09681 tab. D.5. Card: XCiT: Cross-Covariance Image Transformers.

Depth-wise multi-scale convs with channel-wise attention: EdgeNeXt

EdgeNeXt is a four-stage mobile hybrid. Most blocks are convolution encoders: a depth-wise k×k conv, layer norm, two pointwise layers and a residual connection. The kernel grows from 3 to 9 across the stages 2206.10589 §3, tab. 1. The last block of stages 2 to 4 is an SDTA encoder. It splits channels into subsets and runs a chain of depth-wise 3×3 convs over them, which gives several receptive field sizes in one block. It then applies self-attention across channels, not pixels, through a C×C score matrix, so the attention cost grows linearly with the number of pixels. The C×C matrix still grows with the square of the channel count 2206.10589 §3 SDTA Encoder, eq. 2–eq. 4.

Author-reported ImageNet-1K top-1 at 256 px: S (5.6M parameters, 1.3G MAdds) reaches 79.4%, against 78.4% for MobileViT-S at 2.01G MAdds with the same parameter count 2206.10589 tab. 2. The authors' latency figures favour S as well: 48.8 ms against 53.0 ms on a Jetson Nano with TensorRT FP16 at batch 1, and 332 µs against 559 µs on an A100 in PyTorch at batch 256 2206.10589 tab. 3. The card notes that the A100 unit is not defined, so compare the ratios, not the absolute values. Removing the SDTA encoders drops S to 78.3% top-1 2206.10589 tab. 7. Card: EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications. The official code is MIT-licensed and differs from the paper in several details (see Contradictions).

Train once, specialize per device: Once-for-All

Once-for-All trains one super-network whose sub-networks differ in depth, width, kernel size and input resolution, and all of them share one set of weights. The space has roughly 2×10^19 sub-networks, stored in 7.7M parameters 1908.09791 §3.2. Progressive shrinking trains the largest sub-network first, then adds smaller kernel, depth and width choices in that order, with the largest network as a distillation teacher 1908.09791 §3.3, app. B. For a device, nothing is trained. The authors sample 16K sub-networks and fit an accuracy predictor, build a latency lookup table for the target, and run an evolutionary search under a latency or MAC limit 1908.09791 §3.4, app. A. The predictor data costs 40 GPU hours, and the paper says this cost stays constant as deployments grow 1908.09791 §3.4, tab. 1. Specialized sub-networks reach 76.0% top-1 at 230M MACs and 58 ms on a Pixel 1 with no fine-tuning, against 75.2% for MobileNetV3-Large at 219M MACs and 58 ms 1908.09791 tab. 1. The 80.0% headline, at 595M MACs under the mobile setting, needs 75 epochs of fine-tuning 1908.09791 §4.2, tab. 1. Card: Once-for-All: Train One Network and Specialize it for Efficient Deployment.

Detector without non-maximum suppression: YOLOv10

YOLOv10 trains two heads with the same structure. The one-to-many head gets several positive predictions per object, as usual for YOLO. The one-to-one head gets one, and only it runs at inference, so no NMS step is needed 2405.14458 §3.1. Both heads score a prediction with one matching metric, m = s · p^α · IoU^β, with α = 0.5 and β = 6.0 in the reported setting 2405.14458 eq. 1, §4.3. Efficiency changes include a lighter classification head, a decoupled downsampling step, and compact inverted blocks in stages picked by a rank analysis. Accuracy changes include a 7×7 depth-wise kernel in small models and one partial self-attention block after stage 4 2405.14458 §3.2. On YOLOv10-S, NMS-free one-to-one training gives 44.3 AP at 2.44 ms, against 44.9 AP at 7.07 ms for the one-to-many model with NMS 2405.14458 tab. 2. The full model reaches 46.3 AP at 2.49 ms 2405.14458 tab. 2. Author-reported COCO results on a T4 with TensorRT FP16 at batch 1 are 38.5 AP at 1.84 ms for N, 46.3 AP at 2.49 ms for S, 51.1 AP at 4.74 ms for M and 54.4 AP at 10.70 ms for X 2405.14458 tab. 1, §4.1. These are 500-epoch numbers. The 300-epoch runs score lower, for example 53.6 AP for X 2405.14458 tab. 19. Card: YOLOv10: Real-Time End-to-End Object Detection.

Real-time detector with large-kernel depth-wise blocks: RTMDet

RTMDet is a one-stage detector, not a backbone study. Its blocks replace the 3×3 convolution of a CSPDarkNet block with a 5×5 depth-wise convolution, which the authors say enlarges the receptive field at low cost 2212.07784 §3.2. In the ablation, a 5×5 kernel gives 50.9 AP at 2.11 ms against 50.0 AP at 2.10 ms for 3×3, and 7×7 gives 51.1 AP at 2.73 ms 2212.07784 tab. 5. Fewer blocks with wider channels (3-6-6-3 rather than 3-9-9-3) are used to keep parallel latency down, at 50.9 AP and 2.11 ms against 51.4 AP and 2.60 ms 2212.07784 tab. 5. Label assignment is dynamic, takes the IoU as a soft classification target and uses a log-IoU regression cost 2212.07784 §3.3, eq. 2, eq. 3. On RTMDet-s, the paper's assignment reaches 44.5 AP against 43.2 AP for SimOTA 2212.07784 §4.4. Author-reported COCO val2017 results at 640 px, FP16 TensorRT, batch 1 on one NVIDIA 3090, with NMS excluded: RTMDet-s reaches 44.6 AP at 1.22 ms, against 40.5 AP at 1.20 ms for YOLOX-s and 43.5 AP at 0.92 ms for YOLOv6-s 2212.07784 tab. 2. RTMDet-x reaches 52.8 AP at 3.10 ms 2212.07784 tab. 2. The paper's own speed comparison favours YOLOv6 at the small end, so the advantage is in accuracy per latency at the middle and large sizes. The benchmark pipeline is not in the repository, so the latency numbers cannot be reproduced from the code 2212.07784 §4.1. Card: RTMDet: An Empirical Study of Designing Real-Time Object Detectors.

Pooling early stages, attention only in the last stage: EfficientFormer

EfficientFormer is a pure transformer built to be fast on a phone. Its first two stages use 3×3 average pooling as the token mixer, with 1×1 convs for the MLP and batch norm in place of layer norm. Global self-attention appears only in the last stage 2206.01191 §4.1, tab. 6. Stage depths and widths are chosen by a supernet and a per-block latency table measured on the device. The search removes blocks or narrows stages until a latency target is met 2206.01191 §4.2, alg. 1. Author-reported ImageNet-1K top-1 on an iPhone 12 Neural Engine, at 300 epochs: L1 79.2% at 1.6 ms, L3 82.4% at 3.0 ms and L7 83.3% at 7.0 ms 2206.01191 tab. 1. The released repository has no search code, so the final models can be rebuilt from it but not re-searched; see the card. Card: EfficientFormer: Vision Transformers at MobileNet Speed.

Pure CNN with transformer layout choices: RepViT

RepViT starts from MobileNetV3-L and changes it step by step, adopting layout choices from small vision transformers 2307.09283 §3. Each block splits the spatial mixer from the channel mixer. The spatial mixer is a depthwise 3×3 conv and a depthwise 1×1 conv, plus the identity, then batch norm. At inference the three paths fold into one depthwise conv 2307.09283 §3.2. The channel mixer expands to twice the width, applies GELU and projects back 2307.09283 §3.2. In the step-by-step ablation, the aligned MobileNetV3-L reaches 71.5% top-1 at 1.01 ms, and the final block layout reaches 77.4% at 0.87 ms 2307.09283 §3.1–§3.4, fig. 2. Author-reported ImageNet-1K top-1 on iPhone 12 Core ML, with distillation, at 300 epochs: M0.9 78.7% at 0.9 ms, M1.0 80.0% at 1.0 ms, M1.5 82.3% at 1.5 ms and M2.3 83.3% at 2.3 ms 2307.09283 tab. 1. Without distillation, M2.3 reaches 82.5% at 2.3 ms 2307.09283 tab. 2. The distilled models use RegNetY-16GF as teacher 2307.09283 §4.1. Full block configs are in the paper's supplementary material, which is not in our source; see the card. Card: RepViT: Revisiting Mobile CNN From ViT Perspective.

Attention block that keeps position: Coordinate Attention

Coordinate attention is a block for mobile CNNs, not a backbone of its own. It is inserted into a backbone such as MobileNetV2 or MobileNeXt 2103.02907 §3.3, fig. 3. Squeeze-and-excitation averages each channel over the whole map, which drops position. Coordinate attention averages along the width to get one profile per row, and along the height to get one profile per column 2103.02907 §3.2.1, eq. 4, eq. 5. A shared 1×1 conv reduces the joined profiles by ratio r. Two further 1×1 convs and sigmoids then produce a row gate and a column gate, and the input is multiplied by both 2103.02907 §3.2.2, eq. 6–eq. 9.

With r = 32 in MobileNetV2, the authors report 74.3% ImageNet top-1, against 73.5% with SE and 72.3% for the baseline. The block adds about 0.45M parameters and 10M M-Adds over the baseline, and it takes 17–19 ms on a Pixel 4 against 14–16 ms 2103.02907 tab. 1. The gain also holds at width multipliers 0.75 and 0.5 and on MobileNeXt 2103.02907 tab. 2, tab. 3. On detection, SSDLite320 on COCO reaches 24.5 AP, against 22.3 for the baseline and 23.7 with SE 2103.02907 tab. 6. On segmentation the paper reports larger gains: DeepLabV3 on Pascal VOC reaches 73.32 mIoU at output stride 16, against 70.84 for the baseline 2103.02907 §4.4.2, tab. 8. The text does not say where inside the MobileNetV2 unit the block goes (see Contradictions). Card: Coordinate Attention for Efficient Mobile Network Design.

Local window self-attention with haloing: HaloNet

HaloNet keeps the ResNet-50 layout and builds its spatial operation from local 2D self-attention 2103.12731 §3.3. The image is cut into b × b query blocks. Each block gets one shared memory of (b + 2h) × (b + 2h) pixels, the block plus a band of h pixels on every side, and attention runs on all blocks in parallel with no mask 2103.12731 §3.2, fig. 1. Downsampling is a strided attention layer rather than attention followed by pooling, which the authors say cuts downsampling FLOPs by 4× without changing accuracy 2103.12731 §3.2. The windows are not centred on each pixel, so the layer is equivariant only to shifts that are multiples of the block size 2103.12731 §3.2.

Author-reported: H7 reaches 84.9% top-1 on ImageNet from scratch, and the family matches or slightly beats EfficientNet at equal parameters 2103.12731 §4.1, fig. 4. In a 150-epoch, 256 px comparison, HaloNet-50 reaches 78.6% against 77.6% for ResNet-50 2103.12731 tab. 3. Keeping convolution in stage 1 costs 0.3 points and cuts training time: H7 gets 84.6% at 1.4 normalised training time, against 84.9% and 1.9 for pure attention 2103.12731 tab. 4. As a detector backbone, two halo layers in place of the last three convolutions of a ResNet-50 give 45.2 box AP against 44.5 for ResNet-50 with SE, at 540 ms against 446 ms backbone time on two TPUv3 cores 2103.12731 tab. 5. The speed results are all TPUv3 numbers, and pure attention HaloNets train more slowly than matched EfficientNets 2103.12731 §5, fig. 8. Card: HaloNet.

Head split between windowed and pooled-key attention: HiLo, as used in LITv2

HiLo gives a fraction alpha of the heads a global branch and the rest a local branch 2205.13213 §4.1, eq. 6. The local (high-frequency) heads run self-attention inside non-overlapping s×s windows, with s = 2 on ImageNet 2205.13213 §4.1, §5.1. The global (low-frequency) heads take queries from every token, but keys and values from tokens average-pooled over the same windows 2205.13213 §4.1. With alpha = 0.5 the cost is about (7/4)ND² + s²ND for the local branch and (3/4 + 1/s²)ND² + (1/s²)N²D for the global one, so the quadratic term is divided by s² but not removed 2205.13213 §4.1. LITv2 is a four-stage backbone built on this layer. Stages 1 and 2 have only a feed-forward block with a 3×3 depth-wise conv. Stage 3 uses alpha 0.9 and s = 2, and stage 4 uses window size 1, so its attention is standard 2205.13213 §4.3, app. A. Position comes from the zero padding of that conv, and the relative position bias is dropped 2205.13213 §4.2. Author-reported ImageNet-1K top-1 with throughput on one RTX 3090 at batch 64: LITv2-S 82.0% at 1,471 img/s, LITv2-M 83.3% at 812 img/s and LITv2-B 83.6% at 602 img/s 2205.13213 tab. 1, §5.1. In the LITv2-S attention ablation, plain multi-head attention reaches 82.3% at 1,293 img/s and HiLo 82.0% at 1,471 img/s 2205.13213 tab. 4. Card: Fast Vision Transformers with HiLo Attention.

Real-time pose network with SimCC heads: RTMPose

RTMPose is a top-down pose estimator. A separate detector finds each person, and the pose network runs on the crop 2303.07399 §1, §3.1. Its backbone, CSPNeXt-m, is the one from RTMDet 2303.07399 §3.1. The efficiency choice is in the head. Each keypoint is predicted as one distribution over x bins and one over y bins, instead of a heat map 2303.07399 §3.1. The head is a 7×7 convolution, a 256-dimensional expansion and one gated attention unit 2303.07399 §3.3, §3.4, tab. 2. Training starts from a heat-map pretraining of the backbone, then uses EMA, a flat-cosine schedule and a two-stage augmentation 2303.07399 §3.2, tab. 7.

Author-reported COCO val2017 AP without flip, with a Faster R-CNN detector and no extra data: RTMPose-t, -s, -m and -l reach 65.8, 69.6, 73.6 and 74.8 at 0.36, 0.68, 1.93 and 4.16 pose GFLOPs 2303.07399 tab. 4. For the pose model alone at batch 1, RTMPose-s takes 13.89 ms on a Snapdragon 865 with four threads, and RTMPose-m takes 2.288 ms with TensorRT FP16 on a GTX 1660 Ti 2303.07399 §4.3, tab. 10–11. Card: RTMPose.

Bottom-up pose network for CPUs: Lightweight OpenPose

Lightweight OpenPose is a task network, so it sits beside YOLO-LITE as a measured example. It keeps OpenPose's bottom-up design of keypoint heat maps and part affinity fields, grouped into people. It replaces the VGG-19 trunk with a dilated MobileNet v1 whose output stride stays at 8 1811.12004 §4.1.1. The network keeps one initial stage and one refinement stage. Each refinement block replaces a 7×7 convolution with a 1×1, a 3×3 and a dilated 3×3 convolution, plus a residual connection 1811.12004 §4.1.2. Author-reported: 9 GFLOPs and 42.8 AP on COCO validation, against 61.7 GFLOPs and 43.4 AP in the matching row of the paper's table 2 1811.12004 tab. 2, tab. 5. End-to-end it reaches 28 fps on an Intel NUC and 26 fps on an i7-6850K, against 1.17 and 0.95 fps for the baseline on a video with more than 20 people 1811.12004 tab. 6. Before the rewrite, resizing feature maps to input size was the slowest step, at 10.5 fps on its own 1811.12004 tab. 3. The paper parallelises keypoint extraction with OpenCV in C++, and it upsamples by a factor of 8 instead of to full input size, which it reports keeps accuracy 1811.12004 §4.2. Card: Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose.

High-resolution branches with conditional channel weights: Lite-HRNet

Lite-HRNet keeps the parallel multi-resolution branches of HRNet, built from shuffle blocks 2104.06403 §3.1. Its unit splits each branch's channels in two and rescales the working half in two ways. A cross-resolution weight map comes from pooled features of all branches, and a per-branch spatial weight vector comes from global pooling. The weights carry information across channels and resolutions, and the authors say they play the role a 1×1 convolution plays for channel mixing 2104.06403 §3.2, eq. 2–eq. 6. Without pretraining, Lite-HRNet-18 reaches 64.8 COCO val AP at 256×192 with 0.20 GFLOPs, and Lite-HRNet-30 reaches 67.2 with 0.31 GFLOPs 2104.06403 tab. 3. Its naive variant with 1×1 convolutions costs 194.8 MFLOPs for 62.5 AP, against 551.7 MFLOPs for 55.2 AP for Small HRNet-W16 2104.06403 tab. 6. The same blocks are used for Cityscapes segmentation, trained from scratch, with 73.8 val mIoU for Lite-HRNet-18 and 76.0 for Lite-HRNet-30 2104.06403 tab. 8. The paper reports parameters and GFLOPs only, with no latency 2104.06403 tab. 3. Card: Lite-HRNet: A Lightweight High-Resolution Network. The repository has pose code only, so the segmentation results can be checked only against the paper.

Local window scans for vision Mamba: LocalMamba

Vision Mamba models read image patches as one sequence, so a row-by-row order puts vertical neighbours far apart 2403.09338 §1, §4.1, fig. 1. LocalMamba adds a local scan: the token grid is cut into w × w windows, and each window is read in full before the next 2403.09338 §4.1. Each block runs several scans, with one selective SSM per branch, and sums the results 2403.09338 §4.1, fig. 3a. A differentiable search over eight candidate directions per layer picks the four with the largest softmax weight after the supernet is trained 2403.09338 §4.2, eq. 5.

Author-reported ImageNet-1K top-1: LocalVim-T reaches 76.2% at 1.5 GFLOPs against 73.1% for Vim-Ti, and LocalVMamba-T reaches 82.7% at 5.7 GFLOPs against 82.2% for VMamba-T at 5.6 GFLOPs 2403.09338 tab. 2. Without the search, LocalVim-T reaches 75.8%, and the local scan alone added to Vim-T gives 74.1% 2403.09338 tab. 5. On COCO with Mask R-CNN at 1×, LocalVMamba-T reaches 46.7 box AP at 291 GFLOPs against 46.5 for VMamba-T at 286 GFLOPs 2403.09338 tab. 3. The speed evidence is FLOPs only: scanning adds none, and no latency or throughput is reported 2403.09338 §4.3, tab. 1. Card: LocalMamba.

Baselines measured against FasterNet

FasterNet's ImageNet table covers DWConv/GConv-based mobile CNNs (ShuffleNetV2, MobileNetV2, GhostNet), mobile ViTs (MobileViT, EdgeNeXt), ViT/MLP-style models (PVT, CycleMLP, PoolFormer, Swin) and CNNs (ResNet50, ConvNeXt), all timed with the same protocol 2303.03667 tab. 3. Selected rows at matched accuracy (GPU throughput at batch 32 on an RTX 2080Ti; CPU and ARM latency at batch 1, single thread, on an i9-9900X and a Cortex-A72) 2303.03667 §4, tab. 3:

MobileViT and EdgeNeXt are evaluated at 256×256, the rest at 224×224 2303.03667 tab. 3. ShuffleNetV2, MobileNetV2 and GhostNet were trained without FasterNet's training recipe, and the authors call that comparison "not totally fair" 2303.03667 app. A.

When to use which

Implementation sketch

The FasterNet block, with the parts the design depends on 2303.03667 §3.2, §3.4, tab. 8:

pconv(x, conv_kxk, c_p):                  # stride 1, input and output have c channels
    head = x[:, :c_p]                     # contiguous first (or last) c_p channels
    tail = x[:, c_p:]                     # passed through unchanged, not dropped
    return concat(conv_kxk(head), tail)   # in-place write avoids copying tail

fasternet_block(x, d):                    # d = stage width
    y = pconv(x, conv3x3(d/4 -> d/4), c_p = d/4)   # no norm, no activation
    y = conv1x1(d -> 2d)(y)
    y = act(bn(y))                        # the block's only norm + activation
    y = conv1x1(2d -> d)(y)
    return x + y

backbone(img, c, depths, act):            # act = GELU for T0/T1, ReLU for T2+
    x = bn(conv4x4_stride4(3 -> c)(img))
    for stage s in 1..4:
        if s > 1: x = bn(conv2x2_stride2(width -> 2*width)(x))   # downsampling lives here
        repeat depths[s]: x = fasternet_block(x, width)
    return x                              # classification adds pool -> 1x1 conv 1280 -> act -> FC

deploy: fold every BN into the conv in front of it before timing [arxiv:2303.03667 §4]

Where variants differ: DWConv- and GConv-based designs replace pconv with a depthwise or grouped 3×3 conv over all channels, often with a wider expansion 2303.03667 §3.1. Mobile ViTs add attention and extra normalization and activation layers 2303.03667 §1. EfficientFormerV2 puts a depthwise conv inside each feed-forward block and uses attention only in stages 3 and 4 2212.08059 §3.1, §4.1. MambaVision uses a Mamba-style mixer in the first half of each late stage and self-attention in the second half 2407.08083 §3.2.2. EfficientViT's block is one self-attention layer between two feed-forward layers, and its attention is cascaded across head splits instead of standard multi-head attention 2305.07027 §3.1, eq. 1, eq. 2. Once-for-All keeps a MobileNetV3-style unit layout and changes training, so every sub-network shares one set of weights 1908.09791 §3.2, §4.1. YOLOv10 adds a second head during training only. Inference uses the one-to-one head, and the detector output needs no NMS 2405.14458 §3.1. TinyViT keeps a standard block layout and changes training instead: it stores top-K teacher probabilities and a seed per augmented view, then trains the student on the sparse targets 2207.10666 §3.1, eq. 2. ResT starts with a convolutional stem and builds a pyramid, and it replaces standard attention: keys and values come from a strided depth-wise reduction, and position comes from a pixel-attention gate 2105.13677 §2.2, §2.4. YOLO-LITE is a plain stack of 3×3 convs with leaky ReLU and 2×2 max-pools, without batch norm 1811.05588 tab. VII. RepViT keeps a CNN layout but splits each block into a re-parameterised depthwise token mixer and a channel mixer with expansion 2 2307.09283 §3.2. EfficientFormer replaces the token mixer with 3×3 average pooling in its first two stages, and only its last stage uses multi-head attention 2206.01191 §4.1. Coordinate attention is a block added inside the MobileNetV2 unit. It pools along rows and columns separately instead of over the whole map 2103.02907 §3.2. PVT v2 keeps the PVT v1 pyramid, pools keys and values to a 7×7 grid, and adds a depthwise conv to its feed-forward block 2106.13797 §3.2, §3.4. XCiT replaces the token attention matrix with a channel-by-channel cross-covariance over unit-normalised queries and keys, and adds a convolutional local patch interaction to each block 2106.09681 §3.2, §3.3. RTMDet replaces the 3×3 convolution in its blocks with a 5×5 depth-wise convolution and uses the IoU as a soft target in label assignment 2212.07784 §3.2, §3.3. HaloNet keeps the ResNet-50 bottleneck and swaps its spatial operation for blocked local attention 2103.12731 §3.3. LocalMamba adds windowed scans to a Vision Mamba block and sums one selective-SSM output per scan direction 2403.09338 §4.1. RTMPose keeps the detector separate and predicts each keypoint as two bin distributions, which avoids the upsampled heat maps that heat-map methods need 2303.07399 §2, §3.1. Exact widths, depths and training settings per FasterNet size are on the card. EdgeNeXt's SDTA block splits channels into subsets, runs a chain of depth-wise 3×3 convs across them, and then applies attention across channels with a C×C score matrix 2206.10589 §3 SDTA Encoder, eq. 2–eq. 4. Lite-HRNet's unit splits each branch in two, weights the working half with a cross-resolution map and a spatial vector, and shuffles the channels afterwards 2104.06403 §3.1, §3.2, eq. 2, eq. 5. EdgeViT adds local convolutions before and after one sampled-token attention step, with a transposed convolution for propagation 2205.03436 §3.2. HiLo splits the heads of one attention layer into a window group and a pooled-key group 2205.13213 §4.1. Lightweight OpenPose changes the trunk and the refinement blocks of a bottom-up pose network, and rewrites its grouping post-processing 1811.12004 §4.1, §4.2.

Contradictions

The cards do not measure the same thing. FasterNet times classification networks on an i9-9900X and a Cortex-A72 2303.03667 §4. EfficientFormerV2 times classification networks on an iPhone 12 neural engine and a Pixel 6 CPU 2212.08059 §5.1, and MambaVision reports A100 throughput at batch 128 2407.08083 tab. 1. The road-damage paper times SSD detectors on a GRID K520 and a Nexus 5X 1801.09454 §5. TinyViT's throughput is from one V100 at 224px 2207.10666 tab. 4. EfficientViT uses the largest power-of-two batch that fits for GPU runs and one thread at batch 16 on CPU 2305.07027 §4.1. Once-for-All times specialized sub-networks on a Pixel 1 with TF-Lite at batch 1 1908.09791 §4.2, and YOLOv10 uses TensorRT FP16 at batch 1 on a T4 2405.14458 §4.1. Their latencies are not comparable and no contradiction follows. Across devices the orderings differ as well: FasterNet-T0 is faster than MobileNetV2 on GPU, CPU and ARM 2303.03667 tab. 3, while on an iPhone 12 neural engine EfficientFormerV2-S0 only ties MobileNetV2x1.0 at 0.9 ms 2212.08059 tab. 2. A ranking from one device does not carry over to another. EfficientViT-M5 shows the same pattern against MobileNetV3-Large: faster on the V100 and on the Xeon CPU column, slower in ONNX on that Xeon 2305.07027 tab. 2. The paper's text also disagrees with its own table on speed ratios. The abstract and §1 put M2 at 5.8× and 3.7× the speed of MobileViT-XXS on GPU and CPU, and 7.4× in ONNX, while tab. 2 gives about 4.1×, 4.1× and 3.8× 2305.07027 §1, tab. 2. The paper does not say how the text figures were computed, so the values here are from the table. The edge survey gives two top-1 values for MobileNetV2 in different sections, 72.0% in §4.4.1 and 70.6% as the baseline in §4.4.4. FasterNet's table gives 72.0% 2605.06714 §4.4.1, §4.4.4 2303.03667 tab. 3. The survey's table also gives frame counts and timings that do not agree with each other 2605.06714 tab. 3. One caveat inside FasterNet's own data: the per-shape operator table shows that DWConv still has lower absolute CPU latency and higher GPU throughput than PConv on all four tested shapes, while PConv is faster only on ARM 2303.03667 tab. 1. The paper's argument is about FLOPS, that is speed relative to the work done, and it pairs this with PConv's better approximation of a regular conv 2303.03667 §4.1, §4.2. "PConv is faster than DWConv" holds for whole networks at matched accuracy 2303.03667 tab. 3, not for single operators at equal width.

YOLO-LITE gives three VOC mAP values for the same chosen model: 33.81% in the abstract, 33.77% in tab. V and 33.57% in tab. III, which the repository README repeats 1811.05588 tab. III, tab. V. The spread is 0.24 points, so cite which figure you use. The card lists further internal discrepancies, such as a layer count in the prose that differs from tab. VII 1811.05588 tab. IV, tab. VII.

ResT's released code departs from its paper in several places, and the paper does not say so. With one head in stage 1, the code skips the 1×1 conv and instance norm that the paper's eq. 4 applies to every stage 2105.13677 eq. 4, tab. 1. The code also adds a LayerNorm after the spatial reduction, a batch norm after each stage's patch embedding, and a final LayerNorm before pooling, and the paper mentions none of them 2105.13677 §2.2, §2.3. Its GL positional variant is defined in the code in a different form and no model uses it, and the LE variant from the ablation is not in the repository 2105.13677 app. A.4, tab. 8. Implement what the code does where the two disagree, and say which you followed.

EfficientViT's paper and released code also differ. The paper runs attention over all tokens, but the code partitions stage 1 into 7×7 windows, and the code adds a learned attention bias and a ReLU before the output projection, which eq. 2 does not show 2305.07027 §3.1, eq. 2. Its learning rate is 1e-3 in the paper, but the code multiplies it by batch size over 512, which gives 4e-3 at a total batch of 2048. The paper does not say whether 1e-3 is before or after this scaling 2305.07027 §4.1. Once-for-All's paper gives absolute learning rates for the shrinking stages, which match the released code only with 32 GPUs 1908.09791 app. B. The paper does not say the code scales these by GPU count. For YOLOv10, the released code detaches the one-to-one head's inputs, so its loss does not reach the shared backbone and neck, and the paper does not mention this 2405.14458 §3.1. Choose which behaviour to implement, and say which one you followed.

EfficientFormer's released code differs from its paper in several places. For L3, the paper's table gives three MB4D and three MB3D blocks in stage 4, while the code builds two and four 2206.01191 tab. 6. The paper's attention scale is 1/√C_j, but the code divides by √32 2206.01191 eq. 6. Both block types also carry a learned per-channel residual scale that equations 4 and 5 omit 2206.01191 eq. 4, eq. 5. Implement the code where they disagree, and say which you followed. RepViT's paper gives stage depths of 2:2:14:2 2307.09283 §3.3, while the code's M0.9 config has 3, 4, 16 and 3 blocks per stage repvit.py. The paper does not say how it counts depth, so this stays unresolved.

PVT v2's released code has layers its paper does not describe. Its linear SRA applies a 1×1 conv, LayerNorm and GELU to the pooled 7×7 grid before the key and value projections, and the linear variants add a ReLU after the first feed-forward layer 2106.13797 §3.2, §3.4. The released checkpoints load into that code, so a faithful implementation needs those layers. Table 1 gives three blocks in stage 2 of B2, B2-Li and B3, but the code builds four 2106.13797 tab. 1. The sources do not settle which depth is right, so state which one you built.

Coordinate attention's code also departs from its text. The paper gives the bottleneck width as C/r, while the released code uses max(8, C // r), which changes the small layers 2103.02907 §3.2.2 coordatt.py. The paper calls the activation between the two convs "non-linear" without naming it, and the code uses BatchNorm followed by h-swish 2103.02907 §3.2.2 coordatt.py. Table 1 also compares SE at r = 24 with coordinate attention at r = 32, so the gap mixes the block and the ratio 2103.02907 tab. 1, tab. 4. Pick one behaviour for the bottleneck and the activation, and say which you followed.

XCiT's equation and its released code differ in two places. Equation 1 divides the cross-covariance by a temperature τ, and its softmax orientation is ambiguous as written. Algorithm 1 puts the softmax on the other axis 2106.09681 eq. 1, §3.2, app. C. The card's Paper vs code section records the released code's form, a per-head temperature multiplying the scores with the softmax over key channels, and advises following the code because it produced the published checkpoints. The same card records that the paper's stated local-interaction order is not given, and the code's order is convolution, GELU, then batch norm 2106.09681 §3.3 card. RTMDet's ablation text names a focal loss, while the released configs use a quality focal loss 2212.07784 §4.4. The paper says eight images are mixed per sample for Mosaic and MixUp, and the configs mix four and two 2212.07784 §3.3. The card's Paper vs code section lists these and the other differences against the mmdetection configs at the recorded commit card.

Some numbers disagree across papers. EfficientFormerV2 reports MobileNetV2×1.0 at 0.9 ms on iPhone 12 2212.08059 tab. 2, while EfficientFormer reports 1.3 ms on the iPhone 12 Neural Engine for the same model 2206.01191 tab. 1. The cards do not explain the gap, so do not mix the two as one baseline. ResNet-50's ImageNet top-1 is 76.0% in the detection survey 2104.11892 tab. 2, 78.5% in EfficientFormer's table 2206.01191 tab. 1 and 78.8% in FasterNet's 2303.03667 tab. 3, and 76.1% in PVT v2's 2106.13797 tab. 2. The sources do not share one training recipe. The survey copies its numbers from other papers and names no device for its latency column 2104.11892 tab. 4. Its MobileNetv2 row pairs 74.7% with 300 MFLOPs, while EfficientFormer and FasterNet give 71.8% and 72.0% for the model at about 0.3 GFLOPs 2206.01191 tab. 1 2303.03667 tab. 3.

HaloNet's learning rate is 1.6 in 2103.12731 §4.1 and 0.1 in 2103.12731 app. C.3. The card does not say which one the H0–H7 runs used, and weight decay is also given three ways (2e-5, 4e-5 and 8e-5) across the appendix. LocalMamba's text gives LocalVim-T a gain of 2.7 points over Vim-Ti 2403.09338 §5.1, while its abstract and table 2 give 3.1 2403.09338 tab. 2. This page uses the table values. RTMPose gives 69.7 AP for the CSPNeXt-m SimCC baseline in the text 2303.07399 §3.1 and 71.3 for SimCC* in its table 1 2303.07399 tab. 1. Its 75.8 AP headline for RTMPose-m with flip test is in the abstract only, and no table row matches it 2303.07399 §1, tab. 4. The released config uses 256 images per loader on eight GPUs, against the 1024 batch in the paper's table 2303.07399 tab. 7. The card has the full list.

EdgeNeXt's Table 1 gives XXS block counts of 3, 3, 9 and 3. The released code's XXS config uses 2, 2, 6 and 2, and its comment ties those shorter depths to the reported XXS accuracy 2206.10589 tab. 1. The card lists this and other code differences, such as GELU placement and the order of the channel split in SDTA (card). The EdgeNeXt text also claims more than its tables show. It says EdgeNeXt has fewer MAdds and parameters than MobileFormer, ViT-C and CoaT-Lite-T, but Table 2 gives S more MAdds than MobileFormer (1.30G against 214M) and more parameters than ViT-C (5.6M against 4.6M). It also says EdgeNeXt beats the lightweight models on detection and segmentation with fewer MAdds, but Table 5 shows more MAdds than all three MobileNets 2206.10589 §1, §4.3, tab. 2, tab. 5. For Lite-HRNet, the paper gives a pose batch size of 32 per GPU, while the released 256×192 configs use 64 per GPU 2104.06403 §4.1 (card).

The three papers added here also measure speed on different hardware. EdgeViT times ImageNet models on a Galaxy S21 CPU 2205.03436 §4.1. HiLo reports throughput on one RTX 3090 at batch 64 2205.13213 §5.1. Lightweight OpenPose reports end-to-end fps on an Intel NUC and an i7-6850K, with OpenVINO 1811.12004 §4.3. The three sets of numbers are not comparable.

Lightweight OpenPose's accuracy does not match its released code. The paper reports 42.8 AP for the two-stage network 1811.12004 tab. 5. The card records that the repository's README gives a checkpoint at about 40% AP on full COCO validation, and 38.6% on a validation subset, after three-stage training (card). The paper does not name the validation subset it uses 1811.12004 §3.2. Its fps come from OpenVINO code that the repository does not contain 1811.12004 §4.3, so the speed figures cannot be checked from the released Python.

EdgeViT's text and tables disagree in three places. The text gives +1.2 AP^m for EdgeViT-S over PVTv2-B1 on Mask R-CNN, while tab. 5 gives +2.2 2205.03436 §4.2.1, tab. 5. The Figure 1 caption gives 33.3 ms for the lightest model, while tab. 2 gives 32.8 ms for EdgeViT-XXS and 33.3 ms for MobileNet-v2 2205.03436 fig. 1, tab. 2. The text calls its efficiency column an average accuracy gain per mJ, but the values are top-1 divided by energy in mJ 2205.03436 §4.1, tab. 3. Separately, the released ImageNet code departs from the paper's Table 1 and complexity count. For XS and S the code's stage depths and head counts differ from the table, and it uses a 5×5 local-aggregation kernel where the complexity count uses k = 3 2205.03436 tab. 1, app. A.1 (card).

HiLo's paper and released code differ on training and on the head split. The paper gives a base learning rate of 1e-3 for classification 2205.13213 §5.1. The code default is 5e-4, and no LITv2 config overrides it (card). The paper says the global branch gets alpha N_h heads, which is not an integer for alpha 0.9 and 12 heads 2205.13213 §4.1. The code rounds down, so 12 heads give 10 global and 2 local heads (card). The paper's stage 4 has alpha 1.0 2205.13213 app. A, while the code keeps 0.9 but assigns every head to the global branch through window size 1 (card).

Papers

• Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks: frames latency as FLOPs / FLOPS, introduces partial convolution, and builds FasterNet, which is faster than mobile CNNs, mobile ViTs, Swin and ConvNeXt at similar accuracy on GPU, CPU and ARM.
• Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone: a road-damage dataset plus SSD MobileNet and SSD Inception V2 baselines, with GPU and Nexus 5X inference times.
• Rethinking Vision Transformers for MobileNet Size and Speed: EfficientFormerV2, a hybrid transformer whose sizes are chosen by a search on parameters and iPhone 12 latency; S0 reaches 75.7% top-1 at 0.9 ms.
• MambaVision: A Hybrid Mamba-Transformer Vision Backbone: a convolutional early-stage backbone whose late stages mix Mamba-style mixers with self-attention; Tiny reaches 82.3% top-1 with A100 throughput at batch 128.
• TinyViT: Fast Pretraining Distillation for Small Vision Transformers: a 5M/11M/21M hybrid transformer family pretrained on ImageNet-21k by distillation from stored teacher labels; the distilled 21M reaches 84.8% top-1 at 224px.
• YOLO-LITE: A Real-Time Object Detection Algorithm Optimized for Non-GPU Computers: a 7-layer, batch-norm-free detector found by trial and error from Tiny-YOLOv2; 33.57% VOC mAP at 21 FPS on a laptop without GPU.
• ResT: An Efficient Transformer for Visual Recognition: a four-stage transformer pyramid with reduced-key attention and a pixel-attention positional gate; ResT-Small reaches 79.6% top-1 on ImageNet-1k and 40.3 AP with RetinaNet.
• PVT v2: Improved Baselines with Pyramid Vision Transformer: a pyramid transformer with pooled-key attention, overlapping patches and a depthwise conv in the feed-forward block; B5 reaches 83.8% top-1 and B4 reaches 46.1 AP with RetinaNet.
• EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention: a sandwich-block transformer whose attention is cascaded across heads; M5 reaches 77.1% top-1 at 10,621 img/s on a V100, and is slower than MobileNetV3-Large in ONNX on CPU.
• Once-for-All: Train One Network and Specialize it for Efficient Deployment: one super-network whose sub-networks are specialized per device by a predictor and a latency table, with no per-device training; 76.0% top-1 at 58 ms on Pixel 1 without fine-tuning.
• YOLOv10: Real-Time End-to-End Object Detection: a YOLO-style detector trained with one-to-many and one-to-one heads, with no NMS at inference; S reaches 46.3 AP at 2.49 ms on a T4 with TensorRT FP16.
• Survey of edge deep learning for vision and medical diagnostics: a three-tier taxonomy of edge hardware and a review of pruning, quantisation, lightweight networks and NAS; survey only, with no experiments of its own.
• EfficientFormer: Vision Transformers at MobileNet Speed: a pure transformer with pooling blocks in early stages and attention only in the last stage, sized by a latency-driven search; L1 reaches 79.2% top-1 at 1.6 ms on the iPhone 12 Neural Engine.
• RepViT: Revisiting Mobile CNN From ViT Perspective: a pure CNN built from MobileNetV3-L with lightweight-ViT layout choices; M1.0 reaches 80.0% top-1 at 1.0 ms on iPhone 12 with distillation.
• Coordinate Attention for Efficient Mobile Network Design: an attention block that pools along height and width separately and gates rows and columns; in MobileNetV2 it reaches 74.3% ImageNet top-1, against 72.3% for the baseline.
• A Survey of Modern Deep Learning based Object Detection Models: a survey of detectors, backbones and lightweight networks that collects author-reported AP, FPS, top-1 and latency from source papers; it runs no experiments.
• XCiT: Cross-Covariance Image Transformers: a transformer whose attention runs across feature channels, so its cost is linear in the number of patches; XCiT-L24 reaches 86.0% top-1 at 384 px with distillation and fine-tuning.
• RTMDet: An Empirical Study of Designing Real-Time Object Detectors: a one-stage detector family with 5×5 depth-wise blocks and soft-IoU dynamic label assignment; RTMDet-s reaches 44.6 AP at 1.22 ms on an NVIDIA 3090 with TensorRT FP16.
• HaloNet: Scaling Local Self-Attention for Parameter Efficient Visual Backbones: a ResNet-like backbone built from blocked local self-attention with haloing and strided attention downsampling; H7 reaches 84.9% top-1 from scratch and is slower to train than matched EfficientNets.
• RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose: a top-down pose estimator with a CSPNeXt backbone and a SimCC head that predicts two bin distributions per keypoint; RTMPose-s runs at 13.89 ms on a Snapdragon 865 with four threads.
• LocalMamba: Visual State Space Model with Windowed Selective Scan: a Vision Mamba backbone with a windowed scan and a per-layer search over scan directions; LocalVMamba-T reaches 82.7% top-1 at 5.7 GFLOPs, and the paper reports FLOPs only.
• EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications: a mobile hybrid whose SDTA block combines multi-scale depth-wise convs with channel-wise attention; S reaches 79.4% top-1 at 1.3G MAdds, against 78.4% for MobileViT-S.
• Lite-HRNet: A Lightweight High-Resolution Network: HRNet built from shuffle blocks, with conditional channel weights in place of 1×1 convolutions; Lite-HRNet-30 reaches 67.2 COCO val AP at 0.31 GFLOPs without pretraining.
• YOLO advances to its genesis: a decadal and comprehensive review of the You Only Look Once (YOLO) series: a review of YOLOv1 to YOLOv12 and five variant detectors; it runs no experiments, so its numbers are carried over from the cited works.
• Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose: a bottom-up pose network with a dilated MobileNet v1 trunk and lighter refinement blocks; 28 fps end-to-end on an Intel NUC, with 42.8 AP reported by the authors.
• EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers: a light hierarchical transformer whose blocks mix neighbours with convolutions and attend over one sampled token per window; XXS reaches 74.4% top-1 at 32.8 ms on a Galaxy S21 CPU.
• Fast Vision Transformers with HiLo Attention: a head-split attention layer, with windowed attention for high frequencies and pooled-key attention for low frequencies, used in LITv2; LITv2-S reaches 82.0% top-1 at 1,471 img/s on an RTX 3090.