Coordinate classification

Overview

Keypoint methods usually output a heat map per joint and take its peak. A coordinate-classification method instead predicts two 1-D distributions per joint, one over the horizontal bins and one over the vertical bins. The joint position is the argmax of each distribution 2303.07399 §3.1 §3.4.

Training uses a Gaussian soft label over the bins around the true coordinate, and a KL divergence against the predicted distribution. The loss applies a temperature to both the outputs and the labels, and sets the label width per axis from the number of bins 2303.07399 §3.4 eq. 4 eq. 5 eq. 6. Since each keypoint yields two vectors rather than a map, the output stage is small, which is the basis of the paper's speed claim 2303.07399 §3.1 §4.3.

Variants

Only one card lists this slug, so the variants below are design choices the card's paper evaluates, not a survey of the field. The paper's development chain gives the COCO val AP of each step 2303.07399 §3.3 §3.4.

Full head: 7x7 convolution, GAU, temperature (RTMPose)

Each keypoint gets one map from a 7x7 convolution. The map is flattened, projected to 256 dimensions and passed through one gated attention unit (GAU). Two linear layers give the x and y logits. The paper uses a temperature tau of 0.1 in the loss and a separate sigma for each axis 2303.07399 §3.3 §3.4. Reported: 73.3 AP on COCO val after the 7x7 convolution, from a 72.8 step before it 2303.07399 §3.4 tab. 2. Card: RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose.

Head without the GAU block

The paper's chain has a step before the GAU, at feature dimension 256, that reaches 71.4 AP on COCO val. Adding the GAU gives 71.9 in the same chain 2303.07399 §3.3. The gain is small and comes from a cumulative sequence of changes, not from an isolated ablation. Card: RTMPose.

When to use which

The paper's speed tables are batch-1 timings on three setups: a Snapdragon 865 with ncnn, an Intel i7-11700 with ONNX Runtime on one thread, and a GTX 1660 Ti with TensorRT FP16 2303.07399 §4.3. The timings are for the pose model alone unless marked as a pipeline. Numbers are author-reported.

Detector cost is not in the pose-model numbers. With RTMDet-nano at 320x320 as the detector, the RTMPose-m pipeline takes 26.61 ms on the CPU and 4.31 ms on the GPU in the paper's setup 2303.07399 tab. 12. Budget the detector separately.

Implementation sketch

The common core is the same for all variants: one distribution per axis, a soft target around the true coordinate, a KL loss, and an argmax at test time. The head shape and the bin count depend on the input size, which is 256x192 in the paper's configs 2303.07399 §3.1.

# Target for one visible keypoint, per axis  [arxiv:2303.07399 §3.4] [eq. 4]
# Input:  keypoint (x, y) in crop pixels; sigma per axis  [eq. 6]
# Output: soft targets over the bins of each axis

def soft_target(x, y, sigma_x, sigma_y, bins_x, bins_y, ratio):
    # Map pixels to bins; the official code uses 2 bins per pixel,
    # which the paper does not state
    mu_x, mu_y = x * ratio, y * ratio
    # Gaussian centred on the true bin, unnormalised  [§3.4] [eq. 4]
    t_x = exp(-(bins_x - mu_x) ** 2 / (2 * sigma_x ** 2))
    t_y = exp(-(bins_y - mu_y) ** 2 / (2 * sigma_y ** 2))
    return t_x, t_y

# Loss on temperature-softened distributions  [arxiv:2303.07399 §3.4] [eq. 5]
# Input:  logits and soft targets per axis; tau = 0.1
# Output: scalar loss

def coord_loss(logits, targets, tau=0.1):
    # Softmax over bins, applied to both model output and target  [eq. 5]
    log_p = log_softmax(logits / tau)
    q = softmax(targets / tau)
    # KL divergence over the bins; the official code averages instead
    return sum(q * (log(q) - log_p))

# Decoding  [arxiv:2303.07399 §3.1] [§3.4]
# Input:  logits_x (K, bins_x), logits_y (K, bins_y)
# Output: keypoint (x, y) in crop pixels

def decode(logits_x, logits_y, ratio):
    # Argmax per axis, then undo the bin-to-pixel ratio
    x = argmax(logits_x) / ratio
    y = argmax(logits_y) / ratio
    return x, y

Contradictions

No cross-paper contradictions: only one card lists this slug. Within the card's paper, the abstract and the tables differ. The abstract gives 75.8 AP for RTMPose-m with flip test, and no table row has that value. The best table row for RTMPose-m is 75.7 2303.07399 §1 tab. 4. The abstract gives 72.2 AP for RTMPose-s. Table 10 gives 72.8 AP(GT) for the same model, and the paper does not define the GT column 2303.07399 §1 tab. 10.

Papers

• RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose: a top-down pose estimator that predicts each keypoint as two 1-D SimCC distributions, with a CSPNeXt backbone and one GAU head.