wikillm

Paper · 2022-03-03 · cs.CV, eess.IV

A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions

Survey, no new method: computer vision for sports video, covering play-field extraction, detection, player and ball tracking, action recognition, trajectory prediction and tactics across about eight sports. Lists 17 datasets and GPU or embedded platforms; open issues include occlusion, identity switches, small fast balls and jersey numbers. Numbers are author-reported and not comparable.

Object detectionVisual object trackingTracking by detectionVideo summarizationsportssurveyplayer-trackingball-trackingsports-datasets

arXivPDFMarkdown8 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Watching sports video by hand is slow. Someone has to find every player and the ball in each frame and follow them over time. This review collects the computer-vision work that tries to automate this, sport by sport. The tasks it covers are finding the playing field, spotting players and the ball, following them from frame to frame, recognising actions such as shots or passes, and predicting where the ball or a player will go next. It also lists public video datasets and the small computers that can run such models close to the cameras.

The main message is that most methods still struggle with the same problems. Players hide behind each other, players in one team look alike, and the ball is small and moves fast. The reported results use different datasets and metrics, so they are hard to compare. Read the review as a map of the field and its open problems. Do not treat it as a recipe to implement.

TL;DR

  • A survey, not a method. It reviews computer vision for sports video, organised by task: play-field extraction, detection, tracking, trajectory prediction and action or event recognition §1 fig. 2.
  • It covers eight sport groups in per-sport tables: basketball, soccer, cricket, tennis, volleyball, hockey, badminton and a miscellaneous group §4 tab. 2–tab. 9.
  • It lists 17 public sports datasets tab. 10 and compares GPU and embedded boards tab. 11.
  • Use it as an entry point and a list of open problems. For implementation details, go to the primary papers it cites.

Problem

Automatic sports video analysis is wanted for player positions, ball trajectories, highlights, referee decisions and team tactics §1. The review says the work is hard because players look alike, occlude each other, move unpredictably, and appear small and blurred in distant views. The camera also moves, and broadcast edits add more noise. Detecting and tracking players and the ball together is harder still because the ball changes hands and is often occluded §1.

Method

This is a literature review, so its "method" is how it selects and groups studies.

  1. Selection. Studies were taken from "peer review" and "high impact factor online sources" §2. The review gives no search protocol, databases, date range or count of included papers §2. Figure 3 plots publications per publisher group for four periods fig. 3.
  2. Scope and taxonomy. Figure 1 splits sports into track and field games (swimming, cycling, running) and ball games, which are subdivided into indoor and outdoor and into individual and team fig. 1. Section 1 describes the split differently; see Pitfalls §1.
  3. Processing pipeline. Figure 2 shows the intended order: play-field extraction, then detection (team classification, referee gestures, jersey numbers, ball position, possession) and tracking (player performance, referee actions, player positions, ball trajectory, shot or goal classification), then highlight extraction fig. 2.
  4. Play-field extraction §3. Background subtraction, which the review attributes to a Gaussian model §3 fig. 5, gives a foreground mask. Colour-based field segmentation converts RGB to HSI, YCbCr or normalised RGB §3. Field lines, advertisements and non-field regions can be labelled and classified by a model trained on those three classes fig. 7. Camera setups shown are a ceiling-mounted camera, a bird's-eye view and a multi-camera layout with eight cameras fig. 6a–c §3.
  5. Per-sport review §4.1–§4.8. Each sport has a table with reference, problem, method, reported performance and limitations: basketball tab. 2, soccer tab. 3, cricket tab. 4, tennis tab. 5, volleyball tab. 6, hockey tab. 7, badminton tab. 8, miscellaneous tab. 9. The review does not re-run any of these studies.
  6. Machine and deep learning roadmaps fig. 8 fig. 9. Figure 8 covers classical machine learning: supervised, unsupervised, ensemble and reinforcement learning, with named methods and evaluation criteria. Figure 9 covers deep learning: architectures, optimisers, schedules, regularisation, transfer and curriculum learning, and metrics for detection, tracking and trajectory prediction.
  7. Deep-learning taxonomy fig. 10. Classification and detection: CNN backbones (AlexNet, ResNet, DenseNet and others), one-stage detectors (the YOLO series, SSD, RetinaNet, SqueezeDet, EfficientDet) and two-stage detectors (Region-CNN, Fast and Faster R-CNN, FPN, Mask R-CNN, YOLOv4). Unsupervised: RNN and deep extreme learning machines. Tracking: Deep-SORT, particle filter with CNN, FairMOT, Realtime-MOT, Deep-MOT, TransTrack, PointTrack++, DEFT, MOTR, EagerMOT and a monocular 3D tracker. Trajectory prediction: Stepwise Goal-Driven Net, Social-STGCNN, Trajectron++, graph-based prediction, CoverNet, Social GAN, OpenTraj and a spatio-temporal graph transformer.
  8. Applications §7 fig. 14. Chatbots and smart assistants §7.1; video highlights §7.2; training and coaching §7.3; virtual umpires, with VAR and Hawk-Eye as the current systems §7.4 §7.6; AI assistant coaches §7.5; commercial tracking systems such as SportVU and TRACAB §7.6.
  9. Open issues and directions §8 fig. 18. Open issues: camera calibration and viewpoints §8.1; ball detection and tracking §8.1; player tracking under occlusion and look-alike players §8.1; jersey-number reading §8.2 §8.3; action recognition as a non-linear problem §8.3; pose detection §8.3. Directions are grouped as player, ball and referee detection and tracking; pose estimation; and trajectory prediction fig. 18.

The one processing step the review describes as a procedure is the foreground mask of step 4. Threshold and channel handling are not stated fig. 5.

Pseudocode
# Foreground mask by background subtraction  [fig. 5] [§3]
# Input:  frame      (H, W, 3) current video frame
#         background (H, W, 3) play-field background image
#         threshold  scalar T, value not stated
# Output: mask       (H, W, 3) True where a moving object may be

def foreground_mask(frame, background, threshold):
    # Subtract the stored background from the current frame  [fig. 5]
    difference = frame - background
    # Keep values above T; how channels are combined is not stated  [fig. 5]
    mask = difference > threshold
    return mask

Datasets listed in tab. 10. The review says these are "publicly available" with annotations §5, but it gives no licence or URL for any of them. Check each dataset's licence before use.

Dataset (survey ref.)SportTaskAnnotationsSize as stated
Football keyword images [96]SoccerEvent classificationFree kick, penalty, tackle, red and yellow card5000 train, 500 test, 500 validation images
Basketball [267]BasketballAction recognitionDribbling, passing, shooting8 h video, 3399 annotations
Cricket strokes [116]CricketStroke localisationStrokes in untrimmed videoTelecast video at 25 FPS
TTNet [268]Table tennisBall detection, event spottingBall bounces, net hits, empty events5 videos of 10–25 min, 7 short test videos
SoccerNet and SoccerNet-v2 [269, 91]SoccerAction spottingGoal, yellow and red card, substitutionAbout 764 h, 6,637 moments
Volleyball group activity [160]VolleyballGroup activity recognitionPerson actions, group dynamics1525 annotated frames
S-HOCK [270]HockeySpectator analysisPosition, head pose, posture, action31 s, 30 FPS
Sports-1M [271]487 sport classesClassification, activity recognitionActivity labelsListed as 5 min 36 s
Soccer player positions [272]SoccerPlayer trackingPlayer trajectories45 min
Indoor volleyball [224]VolleyballActivity detectionSeven activities per player23 min, 25 FPS
Olympic Sports [273]VariousHuman activity recognitionPoses per sport16 classes, 50 sequences each
NCAA [32]BasketballEvent recognition11 event types1.5 h
ISSIA [274]SoccerPlayer and ball trajectoriesPlayer and ball positions2 min, 25 FPS
APIDIS [275]BasketballEvent summarisationPlayers, referees, ball positions16 min, 22 FPS
Martial arts, dancing and sports [276]Various, including martial arts and dance3D pose estimation5 action types53,000 frames
UCF Sports [225]VariousAction recognitionAction class and localisation13k clips, 27 h
SSET [277]SoccerShot segmentation, event detection, trackingShot types and events350 videos, 282 h, 25 FPS

Hardware tab. 11 lists boards by GPU, memory and storage. Table 12 lists studies that ran on GPU workstations or FPGAs tab. 12.

Hyperparameters

The review trains and evaluates no model, so there are no hyperparameters to implement. The values it names without giving them:

NameValueWhereNotes
background-subtraction threshold Tnot statedfig. 5compared against a per-pixel difference
Gaussian background model parametersnot stated§3 [ref. 12]only the technique is named
colour thresholds for field segmentationnot stated§3HSI, YCbCr and normalised RGB are named

Results

Author-reported, as the review restates them. Each row keeps the setup of its original study. Nothing here is comparable across rows.

ItemMetricValueSourceNotes
Share of reviewed studies by sportshare of studiesSoccer 26%, basketball 15%, cricket 13%, tennis 9%, hockey and ice hockey 6%, volleyball 6%, badminton 4%, all sports 21%fig. 4number of studies behind the shares is not stated
Basketball trajectory prediction (Bertugli et al. 2021, TrajNet++)ADE and FDE7.01 and 10.61tab. 2table labels are absent; order assumed ADE then FDE
Basketball referee signal recognition (Žemgulys et al. 2020)Accuracy95.6%tab. 2LBP and SVM
Soccer tracking, YOLOv3 and SORT (Naik et al. 2022, ref. 291)Tracking accuracy93.7%, 23.7 FPS detection, 11.3 FPS trackingtab. 3survey's first author
Soccer, DeepPlayer-Track (Naik et al. 2022, ref. 292)MOTA and GMOTA96% and 60%tab. 323 FPS detection; survey's first author
Tennis swing classification (ref. 157)Accuracy99.72% (best of several classifiers)tab. 5neural network
Yu Z et al. 2018, basketball trajectoriesAUC91%tab. 2 tab. 12GTX 960 GPU
L. Chen et al. 2020, player behaviourAccuracy76.5% for four action typestab. 2conflicts with 83% in tab. 12
Embedded boardsMemory and storageJetson TX1 4 GB, TX2 8 GB, AGX Xavier 16 GB; Raspberry Pi 1 GB; LattePanda 4 GB; Odroid Xu4 2 GBtab. 11as listed in the survey

Paper vs code

No official code. The paper is a survey with no method or released system. Its text (all pages read) and reference list link no repository. The arXiv metadata file has no comments field to check. A GitHub repository search for "computer vision sports survey" returned no results.

Pitfalls

  • Nothing to reproduce; numbers are not comparable. Each row restates a study's own result, measured on its own dataset and metric. The review says tracking algorithms cannot be compared on a common scale §9. Do not rank methods from this card.
  • Undocumented selection. Studies were chosen from "high impact factor" sources, with no search terms, databases or counts §2. Coverage gaps are likely.
  • Figure 3 caption and axis disagree. The caption says "past five years", but the axis covers 2001–15 through 2020–21 fig. 3.
  • Taxonomy wording differs. Section 1 says sports are split into player-centred and ball-centred, with analysis in Section 4 §1. Figure 1 splits them into track and field and ball games fig. 1.
  • Conflicting values for one study. L. Chen et al. 2020 is 76.5% in tab. 2 but 83% in tab. 12. The review does not explain the difference.
  • Off-topic text in the basketball section. The second paragraph of §4.1 talks about tennis §4.1.
  • Self-citation. Two soccer rows in tab. 3 (refs. 291, 292) are by the survey's first author. The review does not say whether those results were checked independently.
  • Shares without a base count. The percentages in fig. 4 have no stated denominator.
  • Dataset licences missing. The survey gives no licence, URL or access terms for the datasets in tab. 10. Under our licensing rules, do not rely on any dataset without confirming its licence.
  • Hardware figures unchecked. The memory and storage values in tab. 11 are as the survey lists them. They were not checked against vendor specifications.
  • Chess history. Section 7.5 and fig. 15 cover chess, not sports video. Ignore them for implementation.

Lineage

  • Builds on: not applicable, this is a review.
  • Compared against: the 13 earlier surveys and reviews in tab. 1 §1.1. Examples are the soccer player-tracking survey by Manafifard et al. (ref. 5) and the computer-vision-for-sports review by Thomas et al. (ref. 6).
  • Superseded by: not stated.

Source and license

Adapted from “A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions” by Banoth Thulasya Naik, Mohammad Farukh Hashmi, Neeraj Dhanraj Bokde (arXiv:2203.02281), used under CC BY-SA 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. This page is licensed under CC BY-SA 4.0.