Paper · 2022-05-30 · cs.CV, cs.LG, eess.SP

Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles

Siyuan Liang, Hao Wu, Li Zhen et al.

Edge YOLO is a slimmed YOLOv4-style detector: a CSPDarknet53 backbone cut to 17 layers, an FPN neck and three 13/26/52 heads, run on a Jetson-class edge device while a cloud server retrains it. Author-reported: 26.6 FPS on Jetson Xavier and 0.473 mAP@0.5 on COCO2017 (YOLOv4: 0.657).

Object detection On-device inference Multi-scale feature fusion Bounding-box regression lossobject-detectionautonomous-drivingedge-computing

arXiv PDF Code Markdown14 min read

AI-compiled by claude-haiku-5-5 on 2026-11-01, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Self-driving and driver-assist cars have to spot people, vehicles and other objects in camera frames quickly. Sending every frame to a remote server adds delay, uses a lot of bandwidth and stops working when the network is weak. This paper keeps the detector on a small computer in the vehicle and lets a larger computer in the cloud do the heavy work of retraining it. The detector is YOLOv4 with a slimmer backbone, a lighter feature-merging path and three output scales. Overlapping boxes are pruned with a softer rule than the usual one. The authors report that the model runs at about 26.6 frames per second on a Jetson board and is much smaller than YOLOv4, but in their tests it is also less accurate than YOLOv4.

TL;DR

Edge YOLO is a YOLOv4-derived single-stage detector for 416 by 416 inputs. It is built to run on NVIDIA Jetson boards in a vehicle, while a cloud server retrains it and sends new weights down §III-A fig. 2. The backbone is trimmed, the neck is an FPN without PANet, and there are three output scales §III-B. Author-reported on COCO2017: 26.6 FPS on Jetson Xavier and 0.473 mAP@0.5 with a 25.27 MB model, against 0.657 for YOLOv4 tab. 5. The repository has configs and a compiled binary but no training or loss source, so most loss and NMS details cannot be checked against code.

Problem

Cloud inference for vehicle video depends on network quality and adds transmission delay and energy use §I, p. 2. The authors argue that edge devices often have low power budgets, weak GPUs and little RAM §I, p. 2, and that YOLOv4 needs desktop-class GPUs §II, p. 4, so a detector for the edge must be lighter. Their proposal is to split the work: the edge device runs inference and collects data, and the cloud labels data and trains §I, p. 2 §III-A, p. 4.

The experiments assume COCO2017 and KITTI 2D detection data at 416 by 416 inputs §IV-A, p. 10 §III-B, p. 7, and NVIDIA Jetson boards with TensorRT for deployment §IV-C, p. 11.

Method

Edge and cloud split. The camera captures frames at 30 FPS on a moving platform. The edge device runs the detector and saves the results locally. The cloud labels the collected data, retrains and sends updated weights back to the edge §III-A, p. 5 fig. 2. In the cloud-only comparison, the image goes to a cloud server instead §III-A, p. 5 §IV-D, p. 12.
Backbone. Start from the CSPDarknet53 backbone of YOLOv4 and trim it. Channel pruning reduces the residual stacks, and the backbone has 17 layers, down from 88 in YOLOv4 §III-B, p. 5 §III-B, p. 9 tab. 1. The last backbone layer (16) outputs 52 by 52 by 512 tab. 1.
CSP-style split. Half of the channels of a stage bypass the dense block and are merged again at the end of the stage. This reduces repeated gradients and memory traffic §III-B, p. 5–6 fig. 4. The config implements the split with a grouped route Edge_YOLO_coco.cfg.
Activation. Every convolution block is convolution, batch normalisation and Leaky ReLU (CBL) §III-B, p. 6 fig. 5. The leak slope is described only as a small constant and its value is not stated §III-B, p. 7. The config uses activation=leaky throughout Edge_YOLO_coco.cfg.
Neck: SPP and FPN. An SPP block max-pools the input with window sizes 5, 9 and 13 and concatenates the results with the input §III-B, p. 8 Edge_YOLO_coco.cfg. Feature fusion is an FPN with top-down and bottom-up paths, using stride-2 sampling and lateral connections §III-B, p. 8 fig. 8 tab. 2. Parts of PANet are cut: the bottom-up path augmentation, adaptive feature pooling and fully connected fusion §III-B, p. 9. The neck shrinks from 61 layers to 17 §III-B, p. 9.
Three detection heads. Each of the three scales (13 by 13, 26 by 26, 52 by 52) produces predictions with anchor boxes §III-B, p. 8 fig. 7. Each head ends in a linear 1 by 1 convolution. With 80 classes the output has 255 channels, which is 3 × (80 + 5) Edge_YOLO_coco.cfg. For KITTI with 9 classes the output has 42 channels, which is 3 × (9 + 5) edge_kitti.cfg.

Box decoding. The centre is the sigmoid of the predicted offset plus the cell index. Width and height are the anchor size scaled by the exponential of the prediction §III-B, p. 7 eq. 1.

$b_x = \sigma(t_x) + c_x, \quad b_w = p_w\, e^{t_w}$ eq. 1

Pseudocode

# Decode one predicted box in grid units  [§III-B] [eq. 1]
# Input: raw outputs t_x, t_y, t_w, t_h; cell index c_x, c_y;
#        anchor size p_w, p_h
def decode_box(t_x, t_y, t_w, t_h, c_x, c_y, p_w, p_h):
    # Centre: sigmoid offset inside the cell, plus the cell index  [eq. 1]
    b_x = sigmoid(t_x) + c_x
    b_y = sigmoid(t_y) + c_y
    # Size: anchor scaled by the exponential of the prediction  [eq. 1]
    b_w = p_w * exp(t_w)
    b_h = p_h * exp(t_h)
    return b_x, b_y, b_w, b_h

Anchors. The paper clusters ground-truth box sizes with k-means (Algorithm 1), following the YOLOv3 anchor scheme §III-B, p. 7 alg. 1. The number of clusters is not stated. The config lists nine anchor pairs, three per head Edge_YOLO_coco.cfg. These COCO values are identical to the anchor list in the repository's cfg/yolov4.cfg yolov4.cfg. The KITTI config uses a different list edge_kitti.cfg.

Pseudocode

# Anchor sizes by k-means over ground-truth box sizes  [alg. 1]
# The number of clusters K is not stated in the paper
def anchor_kmeans(boxes, num_anchors):
    # Normalise the box sizes  [alg. 1, line 3]
    points = normalise(boxes)
    # Random initial centres  [alg. 1, line 4]
    centres = random_choice(points, num_anchors)
    while centres change:
        # Assign each box to its nearest centre (squared distance)
        # [alg. 1, lines 5-9]
        clusters = assign_nearest(points, centres)
        # Move each centre to the mean of its cluster  [alg. 1, lines 10-15]
        centres = cluster_means(points, clusters)
    return centres

Loss. The total loss is the plain sum of three terms with no weights §III-B, p. 8 eq. 2:

term 1 is printed under the localisation heading but its terms are over classes: a squared difference of class probabilities and of their logarithms, summed over the cells that contain an object eq. 3;
term 2 is objectness: a squared error on confidence over object cells and, with a weight λ_noobj, over empty cells eq. 4;
term 3 is the CIoU box loss: one minus IoU, plus a centre-distance term ρ²/c², plus α·v, where v measures aspect-ratio mismatch and α = v / ((1 − IoU) + v) eq. 5 eq. 6 eq. 7.

Pseudocode

# Total loss per image, terms summed without weights  [eq. 2]
def total_loss(predictions, targets):
    # Term 1: class terms, printed under the localisation heading  [eq. 3]
    class_term = class_loss(predictions, targets)
    # Term 2: confidence; lambda_noobj for empty cells is not stated  [eq. 4]
    conf_term = confidence_loss(predictions, targets)
    # Term 3: CIoU box loss on matched boxes  [eq. 5]
    box_term = ciou_loss(predictions, targets)
    return class_term + conf_term + box_term

Pseudocode

# CIoU box loss between predicted and true box  [eq. 5] [eq. 6] [eq. 7]
def ciou_loss(pred, true):
    # Overlap term: IoU of the two boxes  [eq. 5]
    iou = intersection_over_union(pred, true)
    # Centre-distance penalty; the enclosing-box size c is not defined
    # in the text  [eq. 5]
    centre_term = squared_distance(pred, true) / diagonal_sq
    # Aspect-ratio mismatch v  [eq. 6]
    v = (4 / pi ** 2) * (atan(true.w / true.h)
                         - atan(pred.w / pred.h)) ** 2
    # Trade-off weight alpha  [eq. 7]
    alpha = v / ((1 - iou) + v)
    return 1 - iou + centre_term + alpha * v

Training. Plain SGD with the update θ ← θ − η ∇J, with η = 0.0002, for 100,000 batches on KITTI and COCO §III-B, p. 9 eq. 8. The input is fixed at 416 by 416 by 3 §III-B, p. 7.

$\theta = \theta - \eta \cdot \nabla_\theta J(\theta; x^{(i)}, y^{(i)})$ eq. 8

Non-maximum suppression. Algorithm 2, which the paper calls Soft-NMS, keeps the highest-scoring box and decays the other scores by an exponential of their IoU with it, scaled by σ. As printed, it then removes the boxes whose IoU is at or above T_nms and multiplies the scores of the rest by a function f of IoU §III-B, p. 8–9 alg. 2. The threshold value, σ and f are not stated (see Pitfalls).

Pseudocode

# Soft-NMS as printed in the paper  [alg. 2]
# T_nms, sigma and f are not given numerically
def soft_nms(boxes, scores, nms_threshold):
    kept = []
    while boxes:
        # Keep the highest-scoring box M  [alg. 2, lines 4-5]
        best = argmax(scores)
        kept.append(boxes.pop(best))
        best_box = kept[-1]
        scores.pop(best)
        # Decay all remaining scores by their overlap with M
        # [alg. 2, line 6]
        scores = [s * exp(-iou(best_box, b) / sigma)
                  for s, b in zip(scores, boxes)]
        # Remove boxes at or above the threshold; scale the rest by f
        # [alg. 2, lines 7-12]
        for i in reversed(range(len(boxes))):
            overlap = iou(best_box, boxes[i])
            if overlap >= nms_threshold:
                boxes.pop(i)
                scores.pop(i)
            else:
                scores[i] = scores[i] * f(overlap)
    return kept

Deployment. On the Jetson boards the network is run with TensorRT in FP16 §IV-C, p. 11 tab. 5 tab. 6. The paper lists "sparse training and model accelerated inference adaptation" as one of four parts of the network but does not describe a method for it §III-B, p. 5.

Hyperparameters

Name	Value	Where	Notes
Input size	416 × 416 × 3	§III-B, p. 7 fig. 3	Code also sets `resize=1.5` on each head (Edge_YOLO_coco.cfg); not described in the paper
Output scales	13×13, 26×26, 52×52	fig. 7 §III-B, p. 8	Three heads
Anchors per head	3 (9 pairs in total)	§III-B, p. 8 Edge_YOLO_coco.cfg	Paper text says 18 anchors are selected for one target; not reconciled with the nine pairs in the config
Grid cell size	1 × 1	§III-B, p. 7	Centre coordinates are relative to the cell
Optimiser	SGD, plain update	§III-B, p. 9 eq. 8	Momentum not stated in paper. Config: 0.9 (Edge_YOLO_coco.cfg)
Learning rate η	0.0002	§III-B, p. 9 eq. 8	COCO config 0.00261 (Edge_YOLO_coco.cfg); KITTI config 0.001 (edge_kitti.cfg)
Training length	100,000 batches	§III-B, p. 9	COCO config 30,000 (Edge_YOLO_coco.cfg); KITTI config 60,000 (edge_kitti.cfg)
LR steps	not stated	—	COCO config steps at 24,000 and 27,000 with scale 0.1 (Edge_YOLO_coco.cfg); KITTI at 40,000 and 50,000 (edge_kitti.cfg)
Burn-in	not stated	—	Config: 1000 (Edge_YOLO_coco.cfg)
Batch size	not stated	—	Config COCO: 16 with 1 subdivision (Edge_YOLO_coco.cfg); KITTI: 64 with 16 subdivisions (edge_kitti.cfg)
Weight decay	not stated	—	Config: 0.0005 (Edge_YOLO_coco.cfg)
Augmentation	not stated	—	Config: jitter 0.3, saturation 1.5, exposure 1.5, hue 0.1 (Edge_YOLO_coco.cfg)
Box loss	CIoU, unweighted sum	eq. 2 eq. 5	Config sets `iou_normalizer=0.07` on each head (Edge_YOLO_coco.cfg)
Confidence no-object weight λ_noobj	not stated	eq. 4	—
Activation slope (Leaky ReLU)	not stated	§III-B, p. 7	—
NMS	Soft-NMS	alg. 2	Config sets `nms_kind=greedynms`, `beta_nms=0.6` (Edge_YOLO_coco.cfg)
NMS threshold T_nms, σ, f	not stated	alg. 2	—
Evaluation IoU	0.5	§IV-C, p. 11	Used for TP/FP, mAP and AP (eq. 10)
Anchor clustering K	not stated	alg. 1	—

Results

Author-reported. Setup: COCO2017 (80 classes) and KITTI 2D (9 classes), NVIDIA Jetson Xavier and Nano as edge devices, a cloud workstation with an RTX 2080 Ti for training §IV-A–B, p. 10 tab. 3 tab. 4. Table V and Table VI give the numbers, read from the rendered PDF pages (no LaTeX source was available). The COCO baselines are YOLOv3, YOLOv4, MobileNetV3-SSD and YOLOv3-Tiny; the KITTI baselines are the same set.

COCO2017 tab. 5. The table columns are model size, mAP@0.5, FPS on Xavier and Nano, FLOPs, and the same metrics after TensorRT FP16 conversion.

Model	Size (MB)	mAP@0.5	FPS Xavier	FPS Nano	FLOPs	mAP@0.5 TensorRT	FPS Xavier TRT	FPS Nano TRT
YOLOv3	236.52	0.553	4.9	2.0	65.879G	0.373	31.2	4.93
YOLOv4	245.78	0.657	2.1	1.0	128.459G	0.459	26.1	4.62
MobileNetV3 SSD	115.09	0.362	10.3	9.2	17.863G	0.248	32.9	20.9
YOLOv3-Tiny	33.79	0.331	50.3	15.2	5.571G	0.077	67.5	35.8
Edge YOLO	25.27	0.473	26.6	11.4	10.250G	0.325	49.5	26.6

KITTI 2D tab. 6. The same columns. KITTI rows are the five networks retrained on the KITTI training set, and the mAP column is on KITTI.

Model	Size (M)	mAP	FPS Xavier	FPS Nano	BFLOPS	mAP@0.5 TensorRT	FPS Xavier TRT	FPS Nano TRT
YOLOv3	234.94	0.822	11.8	2.1	65.319G	0.731	29.8	5.03
YOLOv4	244.20	0.8473	5.2	5.1	127.365G	0.784	15.5	5.72
MobileNetV3 SSD	33.86	0.718	11.4	6.1	12.521G	0.653	56.3	26.4
YOLOv3-Tiny	33.11	0.7804	62.7	15.4	6.811G	0.704	66	36.6
Edge YOLO	24.48	0.8212	40.6	9.7	9.970G	0.726	68.2	25.5

Dataset sizes tab. 3 §IV-A, p. 10. COCO2017 has 80 classes, 106,450 training and 11,828 test images, for 118,278 in total. The paper says COCO is split so that 90% is used for training in Table V §IV-C, p. 11. KITTI 2D has 9 classes, 7,481 training and 7,518 test images.

Other reported numbers (in the text or figures):

Average power of the whole network is 16.8 W §V, p. 14; Edge YOLO's power curves sit below those of the other networks in the Jetson traces fig. 11.
Model load time on the two boards is about 5 s on Xavier and 10 s on Nano §IV-C, p. 12.
Cloud comparison §IV-D, p. 12–14 fig. 14: a simulated cloud workstation and an LTE link with 61.4 Mbps upstream, 20.35 Mbps downstream, 14 ms ping and 0% packet loss. Cloud processing time rises steeply with the number of uploaded pictures, while the two edge-cloud curves stay low fig. 14.
Training budgets differ: YOLOv3 and YOLOv4 were trained for about 150,000 batches and the other networks for about 100,000 §IV-B, p. 10.

Paper vs code

The repository is https://github.com/Adam123wu/Edge-YOLO at commit 3a81dbd365a65b298ee2176f26bb3ccac52e3df9. The paper's footnote links it (p. 1), which is the accepted basis for treating it as official; the GitHub owner is not checked against the author list. Its README is the AlexeyAB darknet README with Edge YOLO additions. The repository holds Edge YOLO configs, a compiled darknet binary, a TensorRT demo and a MobileNetV3-SSD folder README.md. There is no LICENSE file at the root; the only LICENSE is under tensorrt_demos/, which covers that subfolder. The README badge links the upstream darknet Unlicense README.md, but that does not license this repository, so it is treated as unlicensed and no code is quoted. The README says weight files are provided, but the clone contains no weight file.

Loss source is missing. The repository has no C or CUDA source for the YOLO layer, so the loss in code cannot be checked against eq. 2–7. Configs show iou_normalizer=0.07 on each head Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg, while eq. 2 sums the terms without weights §III-B, p. 8.
NMS. The paper uses Soft-NMS alg. 2. Every head in the config sets nms_kind=greedynms and beta_nms=0.6 Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg. The shipped inference path therefore does not match the paper's Soft-NMS, unless the binary does something else that cannot be verified.
Learning rate and schedule. Paper: η = 0.0002 and 100,000 batches §III-B, p. 9 eq. 8. COCO config: learning rate 0.00261, 30,000 batches, steps at 24,000 and 27,000 Edge_YOLO_coco.cfg. KITTI config: learning rate 0.001, 60,000 batches edge_kitti.cfg.
Input resizing and augmentation. Paper: fixed 416 by 416 input §III-B, p. 7. Config: resize=1.5 on each head Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg plus jitter and colour augmentation Edge_YOLO_coco.cfg. The paper describes none of this. The effect of resize with random=0 Edge_YOLO_coco.cfg cannot be checked without the source.
Anchors. Paper: "18 anchors" §III-B, p. 8. Config: nine pairs split three per head Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg. The COCO values match cfg/yolov4.cfg yolov4.cfg; the paper does not say the k-means output was used.
Typo in the backbone. One convolution has stride=1f Edge_YOLO_coco.cfg. Whether the parser accepts it cannot be checked without the source.
Classes and output channels. COCO config: 80 classes and 255 outputs Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg. KITTI config: 9 classes and 42 outputs edge_kitti.cfg edge_kitti.cfg. These match the paper's class counts tab. 3.
Config copies. cfg/Edge kitti.cfg and cfg/edge_kitti.cfg are identical. The KITTI anchor list edge_kitti.cfg differs from the COCO list.
Model size. The repository has no weight file, so the reported 25.27 MB and 24.48 MB sizes cannot be checked.

Pitfalls

Conflicting numbers in the paper. The abstract gives a 25.67 MB model, while Table V and the text give 25.27 MB [abstract] tab. 5 §IV-C, p. 11. The text gives KITTI mAP as [0.7904, 0.8473], but Table VI lists YOLOv3-Tiny at 0.7804 §IV-C, p. 12 tab. 6. The conclusion says "only 8 million parameters" §V, p. 14, but no table gives a parameter count.
Which split gives the headline COCO number is unclear. Table V says 90% of COCO is used for training and that IoU is 0.5 on the validation set, while the test results go to the COCO evaluation server §IV-C, p. 11. The 0.473 mAP@0.5 figure is not tied to a split in the text.
Loss terms contradict each other. Eq. 3 is titled a localisation offset loss, but its terms are class terms eq. 3. The text says the classification loss is binary cross-entropy §IV-B, p. 10, while eq. 3 and eq. 4 are squared errors. §III-B says CIoU is the box loss eq. 5, and §IV-B says DIoU §IV-B, p. 10. The variable u in eq. 3 is not defined, and eq. 5's c is not defined either.
λ_noobj and σ, f and T_nms are not given. Without them eq. 4 and Algorithm 2 cannot be implemented exactly.
Soft-NMS as printed is a hybrid. The loop removes boxes at or above T_nms, which is hard suppression, and decays only the others, through f. Line 6 applies a decay to every score before the loop, with an exponent that is hard to read in the PDF render. Check the exponent in the original PDF before implementing alg. 2.
Grid units are ambiguous. The paper says the centre coordinates are relative to a 13 by 13 grid and that "the size range of each grid is 1 × 1" §III-B, p. 7. The config's anchors appear to be in input pixels for width 416 Edge_YOLO_coco.cfg Edge_YOLO_coco.cfg. Compute the sigmoid offset in cell units and the anchor size in input pixels, and check against the paper's figures.
Accuracy drops under TensorRT FP16. COCO mAP@0.5 falls from 0.473 to 0.325, and KITTI mAP@0.5 falls from 0.8212 to 0.726 tab. 5 tab. 6. The 26.6 FPS headline on Xavier is the non-TensorRT figure; the TensorRT figure is 49.5 tab. 5. The text does not state the precision of the non-TensorRT runs.
Training budgets differ across the comparison. YOLOv3 and YOLOv4 were trained for about 150,000 batches, the others for about 100,000 §IV-B, p. 10. A fair comparison needs a common budget.
Four-part network with a missing part. The "sparse training and model accelerated inference adaptation" part is listed but not described §III-B, p. 5.
Cloud test is simulated. The authors "simulate" the cloud environment on their own workstation and link §IV-D, p. 12. The cloud curves in Fig. 14 are specific to that setup.

Lineage

Builds on: YOLOv4 (arXiv:2004.10934, ref. 19), with its CSPDarknet53 backbone and PANet; YOLOv3 (arXiv:1804.02767, ref. 20), cited for the anchor scheme. The paper also cites CSPNet (ref. 42) and FPN (ref. 39) §III-B, p. 5–9. Edge YOLO cuts parts of PANet §III-B, p. 9.
Soft-NMS is the paper's ref. 44 (Bodla et al.), which the reference list gives without an arXiv id.
Compared against: YOLOv3, YOLOv4, YOLOv3-Tiny and MobileNetV3-SSD (ref. 35, "Searching for MobileNetV3") tab. 5 tab. 6.
Superseded by: not known from content already in the wiki.

Source and license

Adapted from “Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles” by Siyuan Liang, Hao Wu, Li Zhen, Qiaozhi Hua, Sahil Garg, Georges Kaddoum, Mohammad Mehedi Hassan, Keping Yu (arXiv:2205.14942), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/Adam123wu/Edge-YOLO at commit 3a81dbd365a6 (none (no LICENSE file in repository)). Any quoted code is shown under that license.

In plain words

TL;DR

Problem

Method

Hyperparameters

Results

Paper vs code

Pitfalls

Lineage

Related

Source and license