HIC-YOLOv5: Improved YOLOv5 For Small Object Detection

In plain words

Drone photos contain many people and cars that are only a few pixels wide. Detectors in the YOLO family shrink the image step by step, and tiny objects become hard to pick out in the smallest feature maps. This paper changes a small YOLOv5 model in three places to keep them.

It adds one more output branch that works on a finer grid, so small objects keep more detail. It adds a block between the backbone and the neck that mixes channel information into neighbouring positions. And it adds a lightweight attention block at the end of the backbone that emphasises useful channels and locations. The authors train and test on the VisDrone-2019 drone dataset and report better accuracy than the base YOLOv5s model.

TL;DR

HIC-YOLOv5 modifies YOLOv5s for small-object detection on VisDrone-2019-DET: an extra 160x160 prediction head, an involution block before the neck and a CBAM block at the end of the backbone §3. The author-reported gain over YOLOv5s is +9.38 mAP@0.5 and +6.42 mAP@[.5:.95] tab. 6. The released code differs from the paper in its default batch size and epoch count, and it never calls the center-crop augmentation the paper describes (see Paper vs code). Treat the paper as a design sketch, not a reproduction recipe.

Problem

Object detectors built on feature pyramids lose resolution as depth grows, so tiny objects that cover few pixels are hard to find. YOLOv5's three heads work on 80x80, 40x40 and 20x20 maps for a 640x640 input §2.1. The authors say the smallest objects are hard for these heads §3.3.

Earlier fixes for small objects either replace the whole neck, which the authors say costs more computation §2.2, or add a transformer layer at the end of the backbone, which they say is expensive and hard to train at large input sizes §2.2. The paper assumes UAV scenes with many small, centrally located objects §4.1.2.

Method

The model is YOLOv5-6.0 with the small (s) configuration: depth multiple 0.33, width multiple 0.50 §4.2 yolov5s-p2-involution-cbam.yaml. Three changes are made to it §3.

1. Extra prediction head. A fourth head works on a 160x160 map, which is stride 4 for a 640 input §3.3. It takes its features from a higher-resolution level of the feature pyramid, through one more upsampling step and merge, and it has its own anchor group §3.3 tab. 3 yolov5s-p2-involution-cbam.yaml. The model therefore has four detection outputs yolov5s-p2-involution-cbam.yaml.
2. Involution between backbone and neck. The last backbone feature map passes through one involution block before the neck §3.2 fig. 3. An involution builds a separate kernel for each pixel from that pixel's own features, and the kernel is shared across the channels of a group §3.2 eq. 1. The released configuration uses a kernel of size 1 (see Paper vs code and Pitfalls).
3. CBAM at the end of the backbone. A CBAM block is applied to the 20x20 map, which is stride 32 for a 640 input, before the SPPF layer §3.1 fig. 4. The paper describes CBAM as channel attention followed by spatial attention §3.1. The released code wraps those two attention steps in a residual bottleneck (see Pseudocode).

The rest of the network is the YOLOv5-6.0 PANet neck and the anchor-based detection heads, which predict box, objectness and class §2.1 fig. 1. The loss is a weighted sum of objectness binary cross-entropy, class binary cross-entropy and CIoU box regression §3.4 eq. 2.

Training uses the YOLOv5 augmentations plus a center crop that the authors added for VisDrone's central object density §3.5 §4.2.

Key equations, as printed in the paper:

Involution output at position (i, j), channel k, with kernel H of size K and G groups eq. 1:

\mathbf{Y}_{i, j, k}=\sum_{(u, v) \in \Delta_K} \mathcal{H}_{i, j, u+\lfloor K / 2\rfloor, v+\lfloor K / 2\rfloor,\lceil k G / C\rceil} \mathbf{X}_{i+u, j+v, k} eq. 1

Total loss, with the three weights listed in §4.2 eq. 2:

\operatorname{Loss}=\alpha \operatorname{Loss}_{o b j}+\beta\text { Loss }_{b o x}+\gamma \operatorname{Loss}_{c l s} eq. 2

The paper's evaluation definitions are IoU eq. 3, precision eq. 4, recall eq. 5 and mAP §4.3.

Pseudocode for the released involution (kernel size 1, 16 channels per group). The paper leaves the kernel size K unstated, and the code uses 1 yolov5s-p2-involution-cbam.yaml common.py.

# Involution with kernel size 1, as configured  [eq. 1] [code: models/common.py#L562-L591]
# Input and output: features (B, C, H, W); C = 512 in the 's' model
def involution_k1(features):
    # One 1x1 conv turns C channels into G = C / 16 weights per pixel
    # [code: models/common.py#L574-L576] [code: models/common.py#L585]
    weights = conv1x1(features)                  # (B, G, H, W)
    # Channel c uses the weight of group c // 16 (16 channels per group)
    # [code: models/common.py#L570] [code: models/common.py#L587-L589]
    weights = repeat_each_group(weights, 16)     # (B, C, H, W)
    return weights * features                    # (B, C, H, W)

Pseudocode for the CBAM block as released. Spatial attention uses a 3x3 kernel in the configuration yolov5s-p2-involution-cbam.yaml common.py.

# CBAM bottleneck at the end of the backbone  [§3.1] [fig. 4]
# Input and output: features (B, 512, H, W), H = W = 20 at a 640 input
def cbam_bottleneck(features):
    # Conv = conv + batch norm + SiLU  [§2.1] [code: models/common.py#L536-L546]
    # 512 channels -> 256 -> 512, with 3x3 kernel in the second conv
    hidden = conv3x3(conv1x1(features))          # (B, 512, H, W)
    # Channel attention: shared two-layer MLP (reduction 16) on the
    # global average-pooled and max-pooled vectors, sum, sigmoid
    # [§3.1] [code: models/common.py#L497-L513] [code: models/common.py#L550-L552]
    channel_gate = channel_attention(hidden)     # (B, 512, 1, 1)
    hidden = channel_gate * hidden               # (B, 512, H, W)
    # Spatial attention: 2-channel map (mean and max over channels),
    # one 3x3 conv, sigmoid  [§3.1] [code: models/common.py#L516-L533]
    spatial_gate = spatial_attention(hidden)     # (B, 1, H, W)
    hidden = spatial_gate * hidden               # (B, 512, H, W)
    # Residual add, because input and output widths match
    # [code: models/common.py#L536-L559]
    return features + hidden

Pseudocode for the loss gains and optimizer, as the training script applies them. The paper lists only the base weights §4.2.

# Effective loss gains after train-time scaling  [§4.2] [code: train.py#L245-L247]
# nl = 4 detection layers, nc = 10 classes, image size 640
def scaled_loss_gains(num_layers, num_classes, image_size):
    # Base gains from the hyperparameter file  [§4.2]
    # [code: data/hyps/hyp.scratch-high.yaml#L13-L16]
    box_gain = 0.05 * 3.0 / num_layers
    cls_gain = 0.25 * num_classes / 80.0 * 3.0 / num_layers
    obj_gain = 0.5 * (image_size / 640.0) ** 2 * 3.0 / num_layers
    return box_gain, obj_gain, cls_gain          # 0.0375, 0.375, 0.0234

Hyperparameters

Values stated in the paper are cited to the paper. Values taken from the released code are cited to code. Where the paper gives nothing, the row says so.

Results

Author-reported, on VisDrone-2019-DET. Base model: YOLOv5s §4.4. Hardware and software as in §4.1.1. The paper does not state which split each table row uses, except where the row label says so.

Ablation over the three changes tab. 6:

Comparison with other methods on VisDrone2019 tab. 4:

Computational comparison with Wang et al. (TIA-YOLOv5) tab. 5:

Headline claims: mAP@0.5 +9.38 and mAP@[.5:.95] +6.42 over YOLOv5s §4.4 tab. 6. Precision +10.29 and recall +6.97 §4.4 tab. 6. The paper also reports that the extra head gives +8.31 mAP@0.5 and +5.51 mAP@[.5:.95] on its own, that involution on top adds +0.66 and +0.57, and that CBAM on top adds +0.41 and +0.34 §4.5 tab. 6.

Paper vs code

Official code: the repository https://github.com/Jacoo-ai/HIC-Yolov5 at commit a7e8b8196989abe8c4b74a027a4168b5869a4dce. Its owner account name is the paper's first author, and its README links this arXiv paper and gives this citation README.md README.md. It builds on YOLOv5 under GPL-3.0 LICENSE. No code is copied into this card; the points below are described in our own words.

• Batch size and epochs. The paper uses batch 128 and 300 epochs §4.2. The released defaults are batch 4 and 400 epochs train.py. The README sets only the weights, the config and the hyp file, so the paper's values are not in the release README.md.
• Center crop. The paper says center crop was added and sets the crop to half the image in each dimension §3.5 §4.2. The released code defines a center_crop function augmentations.py, but nothing calls it: the only reference in the dataset loader is its import datasets.py. The center_crop: 1.0 key in the hyp file is never read hyp.scratch-high.yaml. The paper's reported numbers may or may not include this augmentation; the paper does not say which code produced them.
• Involution kernel size. The paper describes a K x K kernel with a spatial neighbourhood §3.2 eq. 1 and does not state K. The configuration uses kernel size 1 and stride 1 yolov5s-p2-involution-cbam.yaml, so each pixel only scales its own group of 16 channels common.py.
• Involution reduction layer. The class defines a 1x1 reduction conv, but the forward pass computes the kernel from the input directly, and the reduction line is commented out common.py common.py.
• CBAM form. The paper describes CBAM as channel attention then spatial attention, applied at the end of the backbone §3.1. The code puts both inside a bottleneck: 1x1 conv, 3x3 conv, the two attention steps, then a residual add common.py. The block sits at backbone layer 9, before SPPF yolov5s-p2-involution-cbam.yaml.
• Loss weights. The paper lists 0.5, 0.05 and 0.25 as the loss weights §4.2. The training script multiplies them by 3/nl, scales the class gain by nc/80, and scales the objectness gain by (image size / 640)^2 train.py. With nl = 4 and nc = 10 at 640 px, the effective gains are 0.375 (object), 0.0375 (box) and 0.0234 (class). The paper does not mention this scaling.
• Weight decay. The paper states no weight decay. The code scales the base 0.0005 by batch x accumulate / 64 train.py hyp.scratch-high.yaml. At batch 128 the factor is 2, giving 0.001.
• Optimizer. The paper says Adam §4.2. The code also fixes beta1 to the momentum value 0.937 train.py, and the --adam flag cannot be disabled train.py.
• Pretrained start. The paper does not state whether training began from pretrained weights. The default weights are yolov5s.pt, and the README says to use it train.py README.md. When the weights are a .pt file, the script loads only the entries that match in name and shape, and leaves out the anchor entries train.py.
• Anchors. The paper says anchors come from k-means §3.3 and gives the values in Table III tab. 3. The config has those values fixed yolov5s-p2-involution-cbam.yaml. The script still runs the anchor check unless --noautoanchor is set train.py.
• Dataset paths. The dataset file uses Windows paths under D:/, which must be changed before training VisDrone.yaml. Its test path points to the test-dev folder with 1610 images VisDrone.yaml. The paper does not say which VisDrone test set its Test column uses.
• Early stopping and LR. The paper states patience 15 §4.2. The code matches this train.py. The LR schedule, warmup and final LR are not in the paper.

No further discrepancies were found for the head layout, the anchor values in Table III, the augmentation values in Table II, or the model width and depth.

Pitfalls

• Unknown training recipe. The paper gives batch 128 and 300 epochs; the code defaults are 4 and 400. Set both explicitly and record which one you used. Batch size also changes accumulation and weight decay (see Paper vs code).
• Center crop is not in the release. Anyone reproducing the paper from the released code will not get the center-crop augmentation. The function as written also swaps the crop dimensions for non-square images, since it passes half the width as the height. Treat it as an open question, and test with and without it.
• Involution is a per-group scale in the release. With kernel size 1, the block does not mix neighbouring positions, so the paper's spatial-neighbourhood description does not apply to the released model. If you implement K x K, you are not reproducing the released configuration.
• Loss gains are scaled at run time. The listed weights are not the values used in the loss. Derive the effective gains from the number of layers and classes, as the training script does train.py.
• Initialisation. It is unclear whether the reported numbers started from pretrained weights. The partial load means the new blocks and the P2 head start from random initialisation. Check which layers loaded before relying on the result.
• Percentages are absolute. The "%" gains in the text are differences of table entries, in absolute percentage points. For example, 36.95 - 27.57 = 9.38 tab. 6. They are not relative improvements.
• Speed claim not backed by a timing. The text says the CBAM choice reduces training time §4.4. Table V reports only layer, parameter and gradient counts, and its "Gradients" column repeats the "Parameters" column tab. 5. No timing or FLOP count is reported.
• Table IV mixes splits. Some rows are Test and some are Val tab. 4. The Ding et al. row is Val, and the HIC-YOLOv5 Val row is not the same as the Test row. The paper does not say whether other methods' numbers were reproduced or copied.
• Ablation split not stated. Table VI does not name its split. Its baseline and final rows equal the Test rows of Table IV tab. 4 tab. 6.
• Object-size wording. The text says 75% of objects are "0.001 times smaller than the image size" §4.1.2. Table I gives 0.001342 as the 75th percentile of object area as a fraction of the image tab. 1. Read the figure as an area fraction, not a linear scale.
• Code licence. The released code is GPL-3.0 LICENSE. Implementers should write their own code from this card, not copy it.

Lineage

• Builds on: YOLOv5-6.0 (Ultralytics), the base architecture §2.1 fig. 1. The extra high-resolution head is motivated by earlier small-object work the paper cites: Mul-BiFPN, PB-FPN and TPH-YOLOv5 §2.2 §3.3. Involution (Li et al., 2021) §3.2 fig. 5. CBAM (Woo et al., 2018) §3.1 fig. 4. No arXiv ids are given in the paper for these; they are not cards in this wiki.
• Compared against: Shang et al.; Liu et al. (GBS-YOLOv5 and SF-YOLOv5); Ding et al.; Wang et al. (TIA-YOLOv5) tab. 4 tab. 5.
• Superseded by: not known from the content of this wiki.