Paper of the day, 2026-11-02

Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniques

Listen to the episode, or read on. The paper: arXiv:2005.01385.

If you are wiring a detector into a product this week, this 2020 paper is a useful cautionary tale. Its authors carefully benchmark how well the system finds people, but never check the score it was built to produce. That gap is easy to miss, and today you can reproduce it in a browser tab.

The problem in one picture

Drawing made for wikillm on 2026-11-02 by the AI editor. It is not taken from the paper.

How it works

The goal is modest and sensible: watch surveillance video and flag people standing too close together, so nobody has to do the watching by hand. The authors build this from two parts, a perception stack and a piece of geometry.

The perception stack comes first. A YOLO v3 detector, fine-tuned to answer one question (person or not?), draws a box around everyone in the frame. DeepSORT then keeps the same ID on each person from frame to frame. It predicts motion with a Kalman filter, compares appearance, and solves the matching with the Hungarian algorithm. A new track stays on probation for its first three frames. The gate settings and the track's maximum age are not stated.

The geometry comes next, and it is where the fun starts. Each person becomes three numbers: the box centre's x and y, plus a depth value worked out from the box's width and height. The formula has a printed shape but no derivation and no units, so treat it as a proxy the authors chose and not as depth in any physical sense.

The system then measures the straight-line distance between every pair of people in that three-number space. Any pair under a closeness threshold counts as neighbours. The threshold lies between 90 and 170 pixels and changes with position in the frame, but the rule for changing it is not stated.

Neighbours form groups, and the paper does not say exactly how. Our card assumes connected components, so friends of friends share a group, and flags that as its own assumption. Finally, each frame gets a violation index: people in close proximity divided by the number of groups.

The authors also offer an estimate of total violations, which is the index times the number of groups. Do that algebra at your desk and you may enjoy the moment when the groups cancel. The grand estimate is simply the count of people in proximity, wearing a nicer jacket.

The paper's epidemic equations, by the way, are scene-setting. The monitor never uses them.

Try it yourself

Over a coffee, build the geometric half with a made-up frame. No detector is needed: we hand-write the boxes, compute the printed depth value, group the neighbours and score the frame at both ends of the threshold range.

Python

import numpy as np
import matplotlib.pyplot as plt

# A made-up frame: centre x, centre y, box width, box height (pixels)
boxes = np.array([[100, 100, 30, 80], [170, 110, 30, 80], [280, 130, 35, 90],
                  [500, 300, 60, 160], [570, 320, 60, 160], [480, 180, 40, 110]], float)
x, y, w, h = boxes.T
d = (2 * 3.14 * 180) / (w + h * 360) * 1000 + 3   # the card's depth value, as printed
points = np.c_[x, y, d]
dist = np.linalg.norm(points[:, None] - points[None], axis=-1)   # pairwise L2

def group_labels(near):
    labels = np.arange(len(near))        # everyone starts alone
    for _ in near:                       # spread the lowest label along neighbour links
        labels = np.array([labels[row].min() for row in near])
    return labels

fig, axes = plt.subplots(1, 2, figsize=(9, 4), sharey=True)
for ax, threshold in zip(axes, [90, 170]):
    labels = group_labels(dist < threshold)     # includes each person's own zero distance
    ids, sizes = np.unique(labels, return_counts=True)
    n_g = len(ids)                              # a lone person counts as a group here
    n_p = sizes[sizes > 1].sum()
    v = n_p / n_g
    print(f"threshold {threshold}px: people in proximity {n_p}, groups {n_g}, index {v:.2f}")
    ax.scatter(x, y, s=h * 3, c=labels, cmap="tab10", vmin=0, vmax=9)
    ax.set_title(f"threshold {threshold}px, index {v:.2f}")
axes[0].invert_yaxis()
plt.show()

What you should see: two scatter plots of the same people, with larger dots for taller boxes and one colour per group. At 90 pixels there are two pairs and two loners, and the index is 1. At 170 pixels the crowd snaps into two trios, and the index is 3. Nobody moved. Only the threshold changed, and the threshold's update rule is the part the paper leaves out.

What to change: add a seventh row to boxes, placing someone far from everybody else, say in the empty bottom-left corner. At 170 pixels the index falls from 3 to 2. A stranger wandered through and the crowd's score improved, while nobody in it took a single step back. This follows from the card's pseudocode, which lets a lone person count as a group. The paper itself does not settle that point.

What the authors report, and what they leave open

The experiments use 800 manually filtered Open Images person pictures, split 8 : 2, with Oxford Town Centre frames added to the test set, on an NVIDIA GTX 1060. These are the authors' own numbers:

Detector	mAP	FPS	Training time (s)	Iterations
Faster R-CNN	0.969	3	9651	12135
SSD	0.691	10	2124	1200
YOLO v3	0.846	23	5659	7560

Faster R-CNN wins on accuracy and is the slowest. The authors prefer YOLO v3 as the best compromise. The total-loss column is left out here because the three losses are defined differently, which makes a side-by-side comparison meaningless. Sample Oxford Town Centre frames show indices of 3, 2, 2 and 2.33, but those are illustrations and not a measurement.

The open questions are the more interesting part:

The core output is unvalidated. Grouping and the index are never compared with ground truth. Distance error, grouping precision and identity switches are not measured either.
There is no bridge to the real world. The introduction cites the WHO norm of 6 feet, while the pipeline works in pixels mixed with the depth proxy. Camera calibration and a pixel-to-metre conversion are absent.
The paper disagrees with itself. One section pairs DeepSORT with Faster R-CNN, while later ones use YOLO v3. All detectors are said to use an Inception v2 backbone, yet YOLO v3 is described with Darknet-53.
Some of the measurement setup is missing. The IoU threshold behind mAP, the FPS measurement setup, the batch size, the learning rate and the video resolution are not stated.
Moving cameras are untested. The tracker's motion gate is said to struggle when the camera moves.
No official code was found, and the card's search is listed in full. Rights for the images and footage also need checking before anyone reuses them.

Ask yourself

Should a person standing alone count as a group? Rewrite the index both ways and decide which one you would be willing to show a building manager.
If you had one afternoon and one camera, how would you test the grouping against ground truth, and what would count as a pass?
Your detector has a fine mAP but the score downstream of it has never been checked. Which number in your own system is in that position today?