Density map regression

Overview

A density map model outputs one value per pixel, and the count in a region is the sum of those values over the region 1712.06679 §3.1. The training targets are built the same way. A Gaussian is placed at each annotated head and the results are summed, so the total of the map equals the head count 1712.06679 §3.1 eq. 1. The regression branch does not locate heads 1712.06679 §3.2. Only one card covers this slug so far, so the comparison below is an ablation inside one paper, not a survey of the field.

Variants

Regression map alone

A five-layer fully convolutional network produces the density map directly from the image patch, with a ReLU after the last layer 1712.06679 §3.2 eq. 2. Its ablation MAE is 3.37 on Mall and 42.85 on ShanghaiTech PartB 1712.06679 tab. 4. This is the simplest variant on the page, and it needs only point annotations.

Attention-weighted fusion with a detector (DecideNet)

A Faster R-CNN head detector also contributes a density map. Each detected box centre gets a fixed Gaussian, and the sum forms the detection map 1712.06679 §3.3 eq. 3. A small attention network reads both maps and the image, and outputs a per-pixel weight in (0, 1). The final density is the weighted sum of the two maps 1712.06679 §3.4 eq. 4. Card: DecideNet.

An extra loss term pushes the attention toward the detector's score map, which the paper calls quality-aware 1712.06679 §4 eq. 8. This term lowers the ablation MAE from 1.83 to 1.52 on Mall and from 24.93 to 21.53 on ShanghaiTech PartB 1712.06679 tab. 4.

When to use which

The page has one paper, so these rows follow the constraints the paper states, not a wider comparison.

Implementation sketch

The core is the target construction, the count as a sum, and a pixel-wise loss. The fusion variant adds a detector map and an attention blend on top.

# Target: a Gaussian at each annotated head, summed  [arxiv:1712.06679 §3.1]
target = zeros(H, W)
for (x, y) in head_points:
    # sigma 4.0 with a 15-pixel window, as in the paper's training setup [§5.1]
    target += gaussian_kernel(center=(x, y), sigma=4.0, window=15)
# The paper states that the total of the target equals the head count [§3.1] [eq. 1]

# Model: a non-negative density per pixel  [arxiv:1712.06679 §3.2]
density = model(image)          # output may be smaller than the input
# Count of a region: sum of the density over its pixels  [§3.1]
count = density[region].sum()

# Loss: pixel-wise squared error, summed and divided by the number of
# training images  [arxiv:1712.06679 §4] [eq. 6]
loss = ((density - target) ** 2).sum() / num_images

Open details the paper does not state, which an implementer must choose:

• How the output map is upsampled to image size before fusion 1712.06679 §3.4.
• The λ weight of the quality-aware term and the σ of the detector's Gaussian 1712.06679 §4 eq. 3 eq. 8.
• Optimizer momentum, weight decay, batch size and the validation metric 1712.06679 §5.1.
• Whether the detector's score map is detached from the gradient 1712.06679 §4.

Contradictions

The paper's headline claim conflicts with its own tables. The abstract and conclusion state the best results on all three datasets 1712.06679 §6. The tables show CP-CNN with a lower ShanghaiTech PartB MAE, 20.1 against 21.53 1712.06679 tab. 2. They also show Cross-scene with a lower MAE on WorldExpo'10 scenes S4 and S5, 9.30 and 3.10 against 17.40 and 4.75 1712.06679 tab. 3. The authors attribute the S4 gap to the detector working poorly on remote crowds 1712.06679 §5.4.

Papers

• DecideNet (arXiv:1712.06679, CC BY 4.0): fuses a detection density map and a regression density map with a learned per-pixel attention.