A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS

In plain words

YOLO is a family of object detectors that look at a picture in a single pass and output boxes with labels, which keeps them fast enough for real-time use. This paper is a guided tour of the family, from the first 2015 version to the 2023 releases. It explains how the score used to rank detectors (AP) and the step that removes duplicate boxes (NMS) work.

For each version it names the changes that mattered: better anchor boxes, new backbones, how training examples are matched to objects, and how the model is scaled to fit small or large hardware. The recurring theme is the trade-off between speed and accuracy, so the right version depends on the hardware and the application.

TL;DR

• A survey of YOLOv1 through YOLOv8, YOLO-NAS and YOLO-with-Transformer variants §1 §20, covering 16 versions §20.
• It explains AP and NMS first §3, then each version's architecture, loss and training tricks §4–§19.
• Reported COCO AP ranges from 33.0 (YOLOv3) to 56.8 (YOLOv7) in the summary table tab. 4; the text and the table disagree for several versions.
• Use it for orientation and as a reading list. Check the original papers for exact settings, because this review rarely gives the training schedule.

Problem

Object detection must find and label every object in an image. Before YOLO, the accurate detectors were two-stage: they proposed regions and then classified them, which was slow §1 §4. The authors note that earlier YOLO reviews stop at YOLOv3 or YOLOv4, so they cover the later versions too §1.

A reader who wants to pick or implement a YOLO variant must reconcile different metrics, datasets, input sizes and hardware across versions. The paper does not give a single benchmark setup, so its numbers cannot be compared directly §3.2 §20.1.

Method

This is a survey, so the "method" is the review's structure and its descriptions of each system. The steps below follow the paper's sections.

1. Metric and post-processing §3. Average Precision (AP), also called mAP, is the mean over classes of the area under each class's precision-recall curve. A detection counts as correct when its IoU with a ground-truth box passes a threshold §3.1. VOC uses 11-point interpolation at a single IoU; COCO uses 101 recall points and averages AP over IoU 0.50 to 0.95 in steps of 0.05 §3.2. NMS keeps the highest-scoring box and removes boxes that overlap it too much §3.3 alg. 1 fig. 4.

2. YOLOv1 (2015) §4. The image is split into an S×S grid. Each cell predicts B boxes (each with five values: a confidence and four box numbers) and C class probabilities §4.1. The output is S×S×(B·5+C); for VOC that is 7×7×30 with B=2 and C=20 §4.1 tab. 1. The network has 24 convolutional layers and two fully connected layers, and uses leaky ReLU except on the last layer §4.2. The loss is a sum of squared errors for localization, confidence and classification, with two weights, λcoord and λnoobj §4.3 fig. 6. The paper gives the loss only as a figure and does not reproduce its equation.

3. YOLOv2 (2016/2017) §5. Changes: batch normalization on all convolutions; a higher-resolution classifier; a fully convolutional network with no dense layers; anchor boxes with k-means-chosen priors (five per cell); direct location prediction relative to the cell; a passthrough layer that stacks 26×26×512 features into 13×13×2048; and multi-scale training §5 fig. 7 fig. 8 tab. 2. Box centres pass through a sigmoid offset from the cell, and width and height scale the prior by exponentials of the predicted values fig. 8. YOLO9000 jointly trains detection on COCO and classification on ImageNet §5.2.

4. YOLOv3 (2018) §6. Predicts an objectness score by logistic regression, assigns one anchor to each object, and uses independent binary classifiers instead of softmax §6. Backbone Darknet-53 has 53 convolutional layers with residual connections §6.1 fig. 9. Three detection scales (13, 26 and 52 grid sizes) use three priors each §6.2 fig. 10. The SPP variant with kernels 1, 5, 9 and 13 is YOLOv3-spp §6. Reported 36.2 AP and 60.6 AP50 at 20 FPS for YOLOv3-spp §6.3.

5. Backbone, neck and head §7. The paper frames later detectors as three parts: a backbone for features, a neck (for example FPN or PAN) that aggregates them, and a head that predicts boxes and classes, followed by NMS §7 fig. 11.

6. YOLOv4 (2020) §8. Keeps the Darknet framework. Backbone is CSPDarknet53 with Mish activation; neck uses a modified SPP and PANet with concatenation; head uses anchors. Bag-of-freebies include mosaic augmentation, DropBlock, label smoothing, CIoU loss, CmBN, self-adversarial training and cosine annealing. Genetic algorithms tune hyperparameters on the first 10% of periods §8 tab. 3 fig. 12. Reported 43.5 AP and 65.7 AP50 on COCO test-dev 2017, at more than 50 FPS on a V100 §8.

7. YOLOv5 (2020, Ultralytics) §9. A PyTorch implementation with a CSPDarknet-style backbone, SPPF, and a CSP-PAN neck. Uses AutoAnchor, a k-means-plus-genetic step that re-fits anchors to the dataset, with 1000 generations by default §9 §9.1 fig. 13. Five scales: n, s, m, l, x §9. Reports 50.7 AP at 640 px, and 55.8 AP with a 1536 px input and test-time augmentation §9.

8. Scaled-YOLOv4 (2021) §10. Adds scaling in both directions: YOLOv4-tiny for low-end devices (22 AP on COCO, 46 FPS on a Jetson TX2 and 440 FPS on an RTX 2080 Ti) and YOLOv4-large in P5, P6 and P7 sizes for cloud GPUs §10.

9. YOLOR (2021) §11. A multi-task model that learns one shared representation with implicit knowledge, and uses sub-networks for each task §11. Reports 55.4 AP, 73.3 AP50, at 30 FPS on a V100 §11.

10. YOLOX (2021) §12. Five changes to YOLOv3: an anchor-free head; multiple positives through center sampling of the central 3×3 area; a decoupled head with separate classification and regression branches fig. 14; SimOTA label assignment; and strong augmentation (MixUp and mosaic) §12. The paper credits each change with an AP gain: 0.9, 2.1, 1.1, 2.3 and 2.4 points, respectively §12. Reports 50.1 AP at 68.9 FPS on a Tesla V100 §12.

11. YOLOv6 (2022, Meituan) §13. EfficientRep backbone built from RepVGG blocks; a PAN neck with RepBlocks or CSPStackRep blocks; an efficient decoupled head; TOOD-style task-aligned label assignment; VariFocal and SIoU/GIoU losses; self-distillation; and quantization using RepOptimizer and channel-wise distillation §13 fig. 15. Eight scales, N to L6 §13.

12. YOLOv7 (2022) §14. E-ELAN aggregation blocks §14 fig. 16; model scaling that scales depth and width together for concatenation-based models; RepConvN without identity connections; auxiliary-head coarse label assignment with a fine lead head; and an exponential moving average of weights at inference §14. Compared with YOLOv4, the paper reports 75% fewer parameters, 36% less computation and 1.5 points more AP §14.1.

13. DAMO-YOLO (2022, Alibaba) §15. MAE-NAS-based architecture search; an Efficient-RepGFPN neck; a ZeroHead with one linear layer each for classification and regression; AlignOTA label assignment that uses IoU as a soft label; and two-stage distillation with an Align Module and channel-wise dynamic temperature §15. Reports 50.0 AP at 233 FPS on a V100 for the best model §15.

14. YOLOv8 (2023, Ultralytics) §16. Anchor-free, with a C2f block instead of CSP, a decoupled head for objectness, classification and regression, and CIoU plus DFL box losses §16 §16.1 fig. 17. Five scales, n to x. A YOLOv8-Seg variant is described as a semantic segmentation model §16. Reports 53.9 AP at 640 px and 280 FPS on an A100 with TensorRT §16.

15. PP-YOLO, PP-YOLOv2, PP-YOLOE (2020–2022, Baidu) §17. PP-YOLO applies ten existing tricks to YOLOv3, including a ResNet50-vd backbone with deformable convolutions, a larger batch size, parameter moving averages, IoU-aware branch, Matrix NMS and CoordConv §17. Augmentation includes MixUp with Beta(1.5, 1.5), random expand and crop, and multi-size training from 320 to 608 §17.1. PP-YOLOv2 switches to ResNet101, PAN and Mish in the neck, and larger inputs up to 768 §17.2. PP-YOLOE is anchor-free, uses RepResBlocks, TAL, an ET-head, and VFL with DFL §17.3 fig. 18. Reported: PP-YOLO 45.9 AP; PP-YOLOv2 49.5 AP; PP-YOLOE 51.4 AP §17.

16. YOLO-NAS (2023, Deci) §18. Neural architecture search (AutoNAC) with RepVGG-style blocks; QSP and QCI blocks for 8-bit quantization; hybrid quantization that quantizes only some layers; pretraining on Objects365, then COCO pseudo-labels, then COCO train §18 fig. 19. Released as S, M and L, in FP32, FP16 and INT8. Reports 52.2 AP on COCO at 16-bit precision §18.

17. YOLO with Transformers §19. YOLOS replaces the [CLS] token of ViT with 100 [DET] tokens and uses bipartite matching loss; reports 42.0 AP §19. ViT-YOLO, MSFT-YOLO, NRT-YOLO, YOLO-SD and DEYO are application-specific hybrids; DEYO reports 52.1 AP §19 fig. 20.

The discussion §20 notes trends: anchors dropped after YOLOX; the framework moved from Darknet to PyTorch after YOLOv3; backbones moved from plain convolutions to CSP, re-parameterization and NAS; and the speed-accuracy trade-off stayed central §20 §20.1 fig. 21.

Non-maximum suppression, in the form the paper gives it alg. 1:

# Non-maximum suppression  [alg. 1] [§3.3]
# Input:  boxes (N, 4), scores (N,), IoU threshold tau, score threshold T
# Output: kept boxes

def non_maximum_suppression(boxes, scores, tau, T):
    # Keep only boxes whose score reaches the threshold T  [alg. 1]
    candidates = [i for i in range(len(boxes)) if scores[i] >= T]
    # Visit the candidates from highest to lowest score  [alg. 1]
    candidates = sort_by_score_descending(candidates, scores)
    kept = []
    # Take the best remaining box, then drop boxes that overlap it  [alg. 1]
    while candidates is not empty:
        best = candidates.pop_first()
        kept.append(best)
        candidates = [
            other for other in candidates
            if iou(boxes[best], boxes[other]) < tau
        ]
    return kept

Hyperparameters

Values stated in the paper, by version. Most training schedules, learning rates and batch counts are not stated in this review.

Results

Author-reported. The table below is taken from tab. 4 and from the text of each section. Most values are on COCO; YOLO and YOLOv2 are on VOC 2007 §3 tab. 4. The paper does not run any of these models itself, and hardware differs between entries (V100, A100, T4, Jetson, RTX 2080 Ti) §8 §12 §16.

Other results in the text: YOLOS 42.0 AP §19; DEYO 52.1 AP on COCO §19; NRT-YOLO improves YOLOv5l by 5.4% on DOTA §19.

Paper vs code

No official code. The paper has no code link, no project page and no repository for the review itself, and its abstract and arXiv metadata do not point to one. A GitHub repository search for "yolo review terven" returned nothing. The code of the YOLO releases it describes is not part of this card.

Pitfalls

• Text and table disagree. Table 4 and the text give different AP for several versions: YOLOv3 33.0 vs 36.2 (spp) tab. 4 §6.3; YOLOv5 55.8 vs 50.7 at 640 px §9; YOLOv6 52.5 vs 57.2 §13; YOLOv7 56.8 vs 55.9 §14; YOLOX 51.2 vs 50.1 §12; PP-YOLOv2 50.3 vs 49.5 §17.2; PP-YOLOE 54.7 vs 51.4 §17.3. Use the text, which names the setting, and check the source paper.
• Year mismatch. Table 4 lists YOLOv2 as 2016; the text says CVPR 2017 §5 tab. 4.
• Mixed settings in one column. Table 4 combines VOC AP (YOLO, YOLOv2), COCO AP at different input sizes, and COCO AP with test-time augmentation (YOLOv5 at 1536 px) tab. 4 §9.
• Unstated COCO split. §6.3 says "COCO" without a split; some entries say test-dev 2017 and others do not say §8 §11 §13.
• Class/box wording in YOLOv1. §4.1 says B boxes "of the same class" and §4 says "at most 2 classes per grid element (B=2)". The output 7×7×30 fits B·5+C with B boxes and C classes per cell §4.1 tab. 1.
• Classification loss wording in YOLOv8. §16.1 says the class probabilities use softmax and also that the classification loss is binary cross-entropy §16.1. Check the official code before implementing.
• Segmentation label. §16 calls YOLOv8-Seg a semantic segmentation model, but §9 describes YOLOv5 variants for instance segmentation §9 §16. The paper does not explain the difference.
• Batch size units. PP-YOLO's batch size (64 to 192) and PP-YOLOv2's (24 to 12 per GPU) may not be in the same units, and the GPU count is not stated §17 §17.2.
• Losses are only figures. The YOLOv1 loss and the YOLOv8 box losses are described in words and figures. The equations are not reproduced fig. 6 §16.1.
• Hidden costs. Several gains come from training tricks (mosaic, genetic tuning, long schedules, pretraining on Objects365) that increase training cost, not inference cost §8 §18. The text says so for bag-of-freebies §8.
• Quantized results. The YOLO-NAS 52.2 AP is for 16-bit precision §18. The other numbers are for FP32 or FP16 unless stated.
• Version tag. The card is compiled from arXiv v7 (raw metadata). The title page says it is published in Machine Learning and Knowledge Extraction (DOI 10.3390/make5040083) [title page]. The published version may differ from this one; this card does not check that.
• Hardware is not controlled. Speeds come from V100, A100, T4, Jetson TX2 and RTX 2080 Ti in different papers §8 §10 §16. Do not compare FPS across rows.

Lineage

• Builds on: Fast R-CNN and Faster R-CNN (two-stage detection, §1); SSD, RetinaNet and EfficientDet are named as other single-shot or detection methods §1.
• Compared against: EfficientDet, RetinaNet, Fast R-CNN, SSD and Mask R-CNN in the introduction §1; YOLOv6 against YOLOv5, YOLOX and PP-YOLOE §13; YOLOv7 against YOLOv4 and YOLOR §14.1.
• Other reviews covered: Bhavya et al. (to YOLOv3), Diwan et al. (to YOLOv4), and Hussain et al. §1.
• Superseded by: not stated in this paper.