Correspondence (2-D candidates for 3-D keypoints)

Overview

A correspondence here links a fixed point on a 3-D model to a point in the image that might be that point. Each keypoint can have many candidates, so the data is a set of candidate clusters, one per keypoint 1911.08324 §3.1. With the camera matrix known, the task is to find the pose that explains the clusters 1911.08324 §3.1.

Classical pipelines solve this with RANSAC over random subsets of matches. The card's source paper argues that the RANSAC output depends on the order of the matches, and that a surrogate 2-D loss on the matches does not measure pose error 1911.08324 §1 fig. 1. Its remedy keeps the candidate generator and learns the solver instead 1911.08324 §3.1.

This is a different sense from dense correspondence, which matches pixels between two images. Here one side is a small, fixed set of 3-D points on a single object model. Solving the resulting problem is covered by perspective-n-point.

Variants

Learned pose from grouped candidates

The network takes, for each keypoint, a fixed number of candidate cells, each described by a grid position and an offset. A shared MLP scores each candidate, a per-keypoint max-pool summarises each cluster, and the cluster summaries are concatenated in keypoint order. Three fully connected layers output a quaternion and a translation 1911.08324 §3.1.2 eq. 3. The max-pool within a cluster ignores candidate order, and the fixed concatenation keeps the keypoint identity 1911.08324 §3.1.1. The card's source is Single-Stage 6D Object Pose Estimation. Training uses a loss on the 3-D keypoints under the predicted and true pose, not on 2-D positions 1911.08324 §3.2 eq. 7.

The paper attaches this head to two existing candidate generators, SegDriven and PVNet, without changing them 1911.08324 §3.2. Its reported numbers are author-reported and come from those two backbones tab. 1 tab. 2.

When to use which

The only card here describes one variant, so the table compares it with the approach it replaces. All numbers are author-reported.

Implementation sketch

The shape of the core step, from the card's pseudocode. Candidate features pass through a shared network, are pooled per keypoint, and are concatenated in keypoint order before the pose head.

# Grouped correspondence pooling  [arxiv:1911.08324 §3.1.2] [eq. 3]
# Input:  candidates (B, n, m, 4): per keypoint, m cells of (x, y, dx, dy)
# Output: quaternion (B, 4), translation (B, 3)

def pose_from_grouped_candidates(candidates):
    # One shared network scores every candidate  [arxiv:1911.08324 §3.1.2]
    features = shared_mlp(candidates)               # (B, n, m, D)
    # Pool each keypoint's cluster, ignoring candidate order  [eq. 3]
    per_keypoint = max_over_candidates(features)    # (B, n, D)
    # Keep the keypoint order when flattening  [arxiv:1911.08324 §3.1.1]
    flat = flatten(per_keypoint)                    # (B, n * D)
    # Map to pose; the released code decodes translation to a fixed box
    # [code: train.py#L143-L145]
    return pose_head(flat)

Without the fixed keypoint order, a single max-pool over all candidates discards which cluster each value came from. The paper reports a clear accuracy drop for that variant 1911.08324 §3.1.2 fig. 7.

Contradictions

None across cards, since only one card is in the wiki. Within that card, the paper and the released code differ on training schedule, sampling of candidates, noise and the error metric. The card lists each gap with its citation 1911.08324 §4.1 train.py. The gaps are not resolved here.

Papers

• Single-Stage 6D Object Pose Estimation: replaces the RANSAC PnP step with a learned, order-aware head that pools each keypoint's candidate correspondences.