wikillm

Paper · 2023-04-13 · cs.CV

Real-time Multi-Class Helmet Violation Detection Using Few-Shot Data Sampling Technique and YOLOv8

Trains YOLOv8 to find motorcyclists and whether they wear helmets in traffic video. Training frames come from a sampling pipeline (background estimation, day/night/fog labels, rate-based frame selection), followed by six augmentations; test-time augmentation at inference. Author-reported: 0.5861 mAP on the 2023 AI City Challenge Track 5 test set (ranked 7th), 95 fps.

Object detectionData augmentationTest-time augmentationDataset curationobject-detectiontraffic-safetydata-samplingyolo

arXivPDFMarkdown9 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Checking by eye whether motorcyclists wear helmets does not scale to traffic cameras, so this paper builds a camera-based detector that runs in real time. It uses a single-stage detector, YOLOv8, that finds motorcycles, riders, and whether a rider wears a helmet.

Most of the work goes into choosing the training images. The authors pick a representative set of frames instead of re-checking every frame by hand, sort the videos by time of day and weather so that rare conditions are not drowned out, and create extra training images by flipping, rotating, scaling, cropping, blurring and changing the colours. At test time the model also runs on modified copies of each image and combines the answers.

On a public challenge leaderboard the system ranked 7th.

TL;DR

A YOLOv8 detector for helmet violations in traffic video, trained on frames chosen by a sampling pipeline (background estimation, day/night/fog labels, rate-based frame selection) plus six augmentations §3.2. Test-time augmentation raises validation mAP.05-.95 from 0.601 to 0.647 tab. 2. The author-reported test mAP is 0.5861, ranked 7th in the 2023 AI City Challenge Track 5 tab. 3 tab. 4. No official code is reachable, and the frame-rate formula and the sampling step have no ablation, so reuse needs care.

Problem

Manual helmet inspection is slow and error-prone, and most earlier automatic systems were not real time §1. Traffic video varies in weather, lighting and time of day, which changes how visible riders and helmets are §1.

The challenge ground truth has missing annotations. Fixing them by reviewing all 20,000 frames by hand would be too costly, so the authors look for a way to choose a smaller set of frames §3.2. Daylight videos also outnumber the others, which leaves the training data imbalanced across video types §3.2.

The paper does not state how many frames had to be re-annotated, or who annotated them.

Method

Data. The source is the 2023 NVIDIA AI City Challenge, Track 5. It has 100 training videos and 100 test videos, each about 20 seconds long, at 10 fps and 1920x1080 §3.1. The test videos are unannotated §5.

1. Background estimation. For each video, randomly sample frames within a 10-second window. Compute the median over 60 percent of the frames in the sample. The median is meant to remove short-term changes such as zoom and pixelation §3.2.

2. Video categorisation. Assign each video to day, night or foggy using the background image §3.2 alg. 1. A low maximum frequency marks night. Otherwise the skewness of the pixel-frequency distribution decides between day (skewness clearly non-zero) and foggy (skewness close to zero) §3.2 alg. 1. The frequency distributions are shown in fig. 5.

3. Frame sampling. Given the anticipated total number of frames, compute a sample rate for each category from the number of videos and their frame rates, then select frames from each video at that rate §3.2 alg. 2. The stated goal is to take more frames from under-represented categories §3.2.

4. Augmentation. Six transforms are applied to address occlusion, viewpoint change and lighting §3.2: horizontal flip, rotation, scaling, cropping, blurring, and brightness, contrast and saturation changes. No parameter ranges are stated.

5. Training and validation split. Models are trained on 4,500 training examples, split 0.7 : 0.3 into training and validation §3.3. Near-duplicate frames are removed with SCAN [van2020scan] §3.3 fig. 6. Sequences of consecutive frames are kept on one side of the split. Estimated background images, with their augmentations, are added as negative samples §3.3.

6. Detectors. Three single-stage YOLO detectors are compared: YOLOv5, YOLOv7 and YOLOv8 §4.

  • YOLOv5: backbone CSPDarknet53, an SPP block, and PANet feature fusion §4.
  • YOLOv7: E-ELAN layers, compound scaling, and auxiliary-head training §4.
  • YOLOv8: anchor-free detection head §4.

7. Training. Hyperparameters were produced by a genetic algorithm §4.1 and are listed in tab. 1. All models train for 400 epochs, batch size 16, at 832x832 input §4.1. Hardware was one NVIDIA GeForce RTX 3090 §3.3.

8. Test-time augmentation. Run the detector on transformed copies of an image and average the predictions §4.2. The transforms are described only generically (rotation, flipping, cropping) §4.2.

Metric. Mean average precision: mAP = (1/N) * sum over i of AP_i, where N is the number of queries eq. 1 §5. The leaderboard ranks entries by this score §5.1.2.

Pseudocode
# Categorize Videos  [alg. 1]
# Input:  background image of one video (H, W)
# Output: "day", "night" or "foggy"

def categorize_video(background):
    # Frequency of each pixel value in the background  [alg. 1]
    frequencies = pixel_frequencies(background)
    # Night test. The algorithm compares the max frequency with 150;
    # the text says the max frequency corresponds to a pixel value  [§3.2]
    if max(frequencies) < 150:
        return "night"
    # Foggy if skewness is close to zero, otherwise day  [§3.2] [alg. 1]
    # The paper gives no numeric threshold for "close to zero"
    if is_close_to_zero(skewness(frequencies)):
        return "foggy"
    return "day"
Pseudocode
# Frame Sampling Algorithm  [alg. 2]
# Input:  n_frames (anticipated total), videos_per_category,
#         fps_per_category
# Output: selected frames per category

def sample_frames(n_frames, videos_per_category, fps_per_category):
    # Denominator: sum over categories i of n_videos_i * fps_i  [alg. 2]
    # As printed, it does not depend on the category; see Pitfalls  [alg. 2]
    total_frame_slots = sum(
        videos_per_category[i] * fps_per_category[i]
        for i in categories
    )
    selected = {}
    for category in categories:
        # Sample rate for this category  [alg. 2]
        sample_rate = n_frames / total_frame_slots
        # Take frames from each video of the category at that rate  [alg. 2]
        selected[category] = select_frames(category, sample_rate)
    return selected
Pseudocode
# Test-time augmentation  [§4.2]
# Input:  trained detector, image (H, W, 3), list of transforms
# Output: combined prediction for the image

def predict_with_tta(detector, image, transforms):
    # Run the detector on each transformed copy of the image  [§4.2]
    predictions = []
    for transform in transforms:
        predictions.append(detector(transform(image)))
    # Average the predictions; for boxes the merge rule is not stated  [§4.2]
    return average(predictions)

Hyperparameters

Training settings from tab. 1 and §4.1, §3.3. The paper gives no learning-rate schedule, no rule for the warm-up epochs beyond the listed values, and no random seed.

NameYOLOv5YOLOv7YOLOv8WhereNotes
Initial learning rate0.01010.01010.0106tab. 1GA-tuned §4.1
OptimizerAdamAdamAdamtab. 1only source for the optimizer
Momentum0.9370.9650.971tab. 1
Weight decay0.000420.000510.00048tab. 1
Warmup epochs3.8625.5412.689tab. 1fractional values as printed
"IoU"0.8810.8670.912tab. 1column not defined in the text
Epochs400400400§4.1all models
Batch size161616§4.1all models
Image size832x832832x832832x832§4.1training input
Train/validation split0.7 : 0.30.7 : 0.30.7 : 0.3§3.3split unit (video or frame) not stated
Training examples4,5004,5004,500§3.3
Hardware (training)RTX 3090RTX 3090RTX 3090§3.3
Genetic-algorithm searchnot statednot statednot stated§4.1population, generations, fitness: not stated
Background window10 s10 s10 s§3.2sample count per video: not stated
Background median60 % of frames60 % of frames60 % of frames§3.2
Night threshold150150150alg. 1 §3.2unit ambiguous, see Pitfalls
Augmentation rangesnot statednot statednot stated§3.2
TTA transformsnot statednot statednot stated§4.2

Results

Author-reported. Setup: validation split from the training data §3.3; test scores on the 2023 AI City Challenge Track 5 test videos §5.1.2; baselines are the same detectors without TTA tab. 2 tab. 3. The inference hardware is not stated.

Validation tab. 2:

ModelmAP.05mAP.05-.95PrecisionRecall
yolov50.8230.4650.8920.811
yolov70.8460.5260.9120.854
yolov80.8580.6010.9230.898
yolov5+TTA0.9110.6130.9310.907
yolov8+TTA0.9230.6470.9530.918

Test dataset tab. 3:

ModelmAPfps
yolov50.3988160
yolov70.4061163
yolov80.5136162
yolov5+TTA0.5346100
yolov8+TTA0.586195

Public leaderboard, top 10 tab. 4. The paper's entry is rank 7:

RankTeam IDTeam nameScore
158CTC-AI0.8340
233SKKU Automation Lab0.7754
337SmartVision0.6997
418UT_He0.6422
516UT_NYCU_SUNY-Albany0.6389
645UT10.6112
7192Legends (Ours)0.5861
855NYCU-ROAD BEAST0.5569
9145WITAI-530.5474
1011AIMIZ0.5377

Paper vs code

No official code.

The paper's abstract links github.com/aboah1994/few-shot-Video-Data-Sampling. That URL returns HTTP 404, and git clone reports "Repository not found". The author account aboah1994 has 29 public repositories (listed on its repositories tab), none about helmets, few-shot sampling, YOLO or the AI City challenge. GitHub repository searches for few-shot-Video-Data-Sampling, helmet violation yolov8 aboah and few-shot data sampling yolov8 returned nothing or one third-party project (a different owner), which was not used.

Pitfalls

  • Night threshold, unit ambiguous. alg. 1 compares the maximum frequency with 150. §3.2 says the maximum frequency "corresponds to a pixel value" below 150. One reading is a count threshold, the other a pixel-intensity threshold. The paper does not resolve this, and no code was available to check it.
  • Pixel-frequency definition missing. alg. 1 computes a frequency for each pixel, but the text does not say whether that is a histogram over pixel values per frame, per video, or over the background image, and which skewness is taken.
  • "Close to zero" skewness has no threshold. §3.2 alg. 1 gives no number for the foggy test.
  • Sample rate does not depend on the category as printed. alg. 2 divides n_frames by a sum over all categories, so the same value appears for every category. The text says the goal is to take more frames from under-represented categories §3.2. The implementer must choose a category-dependent rule; the paper does not give one.
  • n_frames is not given. The anticipated frame budget alg. 2 is never stated, so the number of selected frames is unknown.
  • Background median sampling. The paper does not state the number of frames sampled per video or how the 60 percent subset is drawn §3.2.
  • Deduplication with SCAN is underspecified. §3.3 says SCAN is used to remove highly similar frames but gives no cluster count, similarity threshold or rule for which frame in a group is kept. SCAN is a clustering method, so the paper's use of it needs clarification.
  • Split details missing. The sizes of the training and validation subsets, whether the split is by video or by frame, and the random seed are not stated. The paper says consecutive frames were kept on one side of the split §3.3, but not how.
  • Augmentation parameters missing. Rotation angles, scaling and crop ranges, blur kernel, colour ranges, per-transform probabilities and whether augmentation is applied online or offline are all not stated §3.2.
  • Hyperparameter table. tab. 1 lists "IoU" with no definition, and "Adam" as the optimiser for all three detectors. Warm-up epochs are fractional (3.862, 5.541, 2.689) and are reported as printed. The genetic-algorithm search settings are not stated §4.1.
  • Backbone naming is inconsistent. §4 says YOLOv5 uses CSPDarknet53 but YOLOv8 uses "Darknet-53", without explaining the difference. This does not affect the numbers, but an implementer may be misled.
  • TTA is underspecified for detection. The paper says predictions are averaged §4.2, but averaging boxes from transformed copies needs a merge rule. Which transforms were used, and how flipped boxes were mapped back, are not stated. TTA cost is visible in tab. 3: yolov8 runs at 162 fps without TTA and 95 fps with it.
  • Metric names undefined. The text never defines mAP.05 or mAP.05-.95 tab. 2. The names suggest IoU thresholds, but this is not stated. Eq. (1) averages AP over N queries eq. 1, without defining AP per query or N further. Who computed the test-set scores is not stated.
  • Speed is not tied to hardware. The 95 fps figure tab. 3 has no hardware, batch size or input resolution. The 832x832 setting is stated for training §4.1, not for inference.
  • The main contribution has no ablation. The few-shot sampling and augmentation steps are not compared against training on the unsampled data. tab. 2 and tab. 3 compare detectors, and TTA on or off, only.
  • Robustness claims are not broken down. The introduction and results claim robustness across weather and time of day §1 §5, but no results are reported per condition.
  • "Fewer annotations" is not quantified. The paper claims fewer annotations are needed §1 but reports no count of annotations used or saved.
  • Leaderboard, not a win. §5.1.2 says the entry "ranked 7th"; the abstract says it "won 7th place". The score is a public-leaderboard rank.
  • Test labels are not public. The test score comes from the challenge leaderboard, so the test set cannot be re-run locally.

Lineage

  • Builds on: YOLOv7, cited as arXiv:2207.02696 §4. YOLOv5 and YOLOv8 are used without a cited source. SCAN [van2020scan] is cited without an arXiv id.
  • Compared against: YOLOv5, YOLOv7 and YOLOv8 without TTA, and YOLOv5 and YOLOv8 with TTA tab. 2 tab. 3. No YOLOv7+TTA run is reported.
  • Superseded by: not known from content in this wiki.

Source and license

Adapted from “Real-time Multi-Class Helmet Violation Detection Using Few-Shot Data Sampling Technique and YOLOv8” by Armstrong Aboah, Bin Wang, Ulas Bagci, Yaw Adu-Gyamfi (arXiv:2304.08256), used under CC BY-SA 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. This page is licensed under CC BY-SA 4.0.