---
kind: concept
slug: correspondence
title: Correspondence (2-D candidates for 3-D keypoints)
summary: >-
  Matches between fixed 3-D model keypoints and candidate 2-D image points, used to recover an object's pose. One card replaces the RANSAC-based PnP step with an order-aware network that pools each keypoint's candidates. Author-reported: faster than PVNet's pipeline on the same backbone, and higher ADD-0.1d averages on Occluded-LINEMOD.
tags: [pose-estimation, 6d-object-pose, perspective-n-point, point-set-pooling]
papers: ["1911.08324"]
related: [perspective-n-point, dense-correspondence, local-feature-matching]
status: draft
updated_at: 2026-11-08
---

## Overview

A correspondence here links a fixed point on a 3-D model to a point in the
image that might be that point. Each keypoint can have many candidates, so the
data is a set of candidate clusters, one per keypoint [arxiv:1911.08324 §3.1].
With the camera matrix known, the task is to find the pose that explains the
clusters [arxiv:1911.08324 §3.1].

Classical pipelines solve this with RANSAC over random subsets of matches. The
card's source paper argues that the RANSAC output depends on the order of the
matches, and that a surrogate 2-D loss on the matches does not measure pose
error [arxiv:1911.08324 §1] [fig. 1]. Its remedy keeps the candidate
generator and learns the solver instead [arxiv:1911.08324 §3.1].

This is a different sense from [dense correspondence](/concepts/dense-correspondence),
which matches pixels between two images. Here one side is a small, fixed set of
3-D points on a single object model. Solving the resulting problem is covered
by [perspective-n-point](/concepts/perspective-n-point).

## Variants

### Learned pose from grouped candidates

The network takes, for each keypoint, a fixed number of candidate cells, each
described by a grid position and an offset. A shared MLP scores each candidate,
a per-keypoint max-pool summarises each cluster, and the cluster summaries are
concatenated in keypoint order. Three fully connected layers output a
quaternion and a translation [arxiv:1911.08324 §3.1.2] [eq. 3]. The max-pool
within a cluster ignores candidate order, and the fixed concatenation keeps the
keypoint identity [arxiv:1911.08324 §3.1.1]. The card's source is
[Single-Stage 6D Object Pose Estimation](/papers/1911.08324). Training uses a
loss on the 3-D keypoints under the predicted and true pose, not on 2-D
positions [arxiv:1911.08324 §3.2] [eq. 7].

The paper attaches this head to two existing candidate generators, SegDriven
and PVNet, without changing them [arxiv:1911.08324 §3.2]. Its reported
numbers are author-reported and come from those two backbones [tab. 1] [tab. 2].

## When to use which

The only card here describes one variant, so the table compares it with the
approach it replaces. All numbers are author-reported.

| Situation | Use | Why |
|---|---|---|
| You have a candidate generator with per-keypoint outputs and need a pose at inference time | Learned pose head on grouped candidates [arxiv:1911.08324 §3.2] | The paper reports it as a drop-in replacement for the RANSAC step, with gains on both backbones it tested [tab. 1] |
| Frame rate matters | Learned head on the PVNet backbone | Reported at 45 FPS against 25 FPS for PVNet's own pipeline, same GPU [tab. 3] |
| The keypoints change between objects or runs | Not this variant as released | The network works for a fixed keypoint set, and the released model hard-codes eight keypoints [arxiv:1911.08324 §4.3] [code: model.py#L26] |
| Very precise correspondences are available | A geometric solver such as PnP | The paper reports its method is less accurate than geometric PnP there [arxiv:1911.08324 §4.3] [fig. 6] |
| The pose must come from a fully differentiable pipeline | Learned pose head | The paper argues the RANSAC pipeline is not end-to-end trainable [arxiv:1911.08324 §1] |

## Implementation sketch

The shape of the core step, from the card's pseudocode. Candidate features
pass through a shared network, are pooled per keypoint, and are concatenated
in keypoint order before the pose head.

```pseudocode
# Grouped correspondence pooling  [arxiv:1911.08324 §3.1.2] [eq. 3]
# Input:  candidates (B, n, m, 4): per keypoint, m cells of (x, y, dx, dy)
# Output: quaternion (B, 4), translation (B, 3)

def pose_from_grouped_candidates(candidates):
    # One shared network scores every candidate  [arxiv:1911.08324 §3.1.2]
    features = shared_mlp(candidates)               # (B, n, m, D)
    # Pool each keypoint's cluster, ignoring candidate order  [eq. 3]
    per_keypoint = max_over_candidates(features)    # (B, n, D)
    # Keep the keypoint order when flattening  [arxiv:1911.08324 §3.1.1]
    flat = flatten(per_keypoint)                    # (B, n * D)
    # Map to pose; the released code decodes translation to a fixed box
    # [code: train.py#L143-L145]
    return pose_head(flat)
```

Without the fixed keypoint order, a single max-pool over all candidates
discards which cluster each value came from. The paper reports a clear
accuracy drop for that variant [arxiv:1911.08324 §3.1.2] [fig. 7].

## Contradictions

None across cards, since only one card is in the wiki. Within that card, the
paper and the released code differ on training schedule, sampling of
candidates, noise and the error metric. The card lists each gap with its
citation [arxiv:1911.08324 §4.1] [code: train.py#L80-L87]. The gaps are not
resolved here.

## Papers

- [Single-Stage 6D Object Pose Estimation](/papers/1911.08324): replaces the RANSAC PnP step with a learned, order-aware head that pools each keypoint's candidate correspondences.