Anchor-free detection

Overview

Anchor-based detectors place a fixed set of candidate windows, called anchors, at every location, then classify and refine each one. Their results depend on how many anchors there are and how big they are 2103.13137 §1. An anchor-free detector drops the windows. Each location predicts its own start and end distances directly, so every location gives exactly one proposal 2103.13137 §3.1. The open question is how to get boundary features accurate enough to refine each proposal without anchors 2103.13137 §1.

Variants

Only one card in the wiki covers this concept so far. The design space below is therefore the one that paper explores, not a survey of anchor-free methods.

Boundary max-pooling refinement (AFSD)

Each pyramid location predicts a coarse start distance and end distance 2103.13137 §3.1. A refinement module takes each coarse proposal, builds a start region and an end region scaled by the proposal width, and takes the channel-wise maximum over each region 2103.13137 §3.2, eq. 3–4. Two consistency losses regularise the pooled features: an activation loss on the start and end features, and a triplet loss across a split action 2103.13137 §3.3. Reported numbers are 52.0 average mAP on THUMOS14 and 34.4 on ActivityNet 1.3, both author-reported 2103.13137 tab. 1. Card: Learning Salient Boundary Feature for Anchor-free Temporal Action Localization.

Design choices inside this variant, from the RGB-only ablation at mAP 0.3–0.7 average 2103.13137 tab. 2:

• Pooling: max 43.5, mean 42.7, conv 42.6, stack 42.9. Max is the best row.
• Quality score in place of a centerness target: 41.4, against 40.2 with centerness and 40.4 for the baseline.
• Consistency losses: 43.5 with both, 42.0 without the boundary contrastive term, 42.7 with either term alone.

When to use which

Implementation sketch

The core is the boundary pooling step. Variants differ in the pooling operator, the consistency losses and the training schedule. See the card for the full training step.

# Boundary max-pooling for one coarse proposal  [arxiv:2103.13137 §3.2] [eq. 3–4]
# Input:  coarse start psi and end xi (in frames); start_feat, end_feat (T, C)
# Output: pooled start and end features for this proposal, each of size C

def pool_boundaries(psi, xi, start_feat, end_feat, delta_a, delta_b):
    # Proposal width, taken as a positive length  [§3.2] (sign note on the card)
    w = xi - psi
    # Start region: part before the start point, part after  [eq. 3]
    s_lo = psi - w / delta_a
    s_hi = psi + w / delta_b
    # End region: part before the end point, part after  [eq. 3]
    e_lo = xi - w / delta_b
    e_hi = xi + w / delta_a
    # Channel-wise maximum over each region  [eq. 4]
    f_s = max_over_range(start_feat, s_lo, s_hi)
    f_e = max_over_range(end_feat, e_lo, e_hi)
    return f_s, f_e

Contradictions

• Region ratios. The paper's default values are δ_a = 4 and δ_b = 10 2103.13137 §4.1. The text says a larger share inside the action helps 2103.13137 §4.3. The best ablation row is labelled δ_a < δ_b, which under eq. 3 gives the background side the larger share 2103.13137 tab. 2. The paper does not reconcile these.
• Headline gains. The introduction claims 3.7 points at mAP@0.5 2103.13137 §1. Section 4.2 claims 7.7 points at mAP@0.6 2103.13137 §4.2. The main table gives 3.9 and 5.7 for the same comparisons 2103.13137 tab. 1. Use the table values.
• Paper and official code. The public repository differs from the paper in several settings that change results: the region ratios are reversed, the Soft-NMS "threshold" is used as a Gaussian width, and the refined positive threshold defaults to 0 rather than 0.5. The full list is on the card, under Paper vs code.

Papers

• Learning Salient Boundary Feature for Anchor-free Temporal Action Localization: AFSD. Per-location boundary prediction, max boundary pooling and two consistency losses for temporal action localization.