Perspective-n-point (camera pose from 2-D/3-D correspondences)

Overview

Given 3-D object points or lines, their image projections and known intrinsic camera parameters, find the rigid transform from the object frame to the camera frame 2610.19734 §1 §2. The unknowns are six: three for rotation and three for translation. Closed-form solutions exist for small numbers of correspondences; for larger sets the problem is nonlinear and needs an iterative solver 2610.19734 §1 §1.1.

The first paper names three choices that decide the result: how the pose is represented, which error function is minimised, and which optimiser is used 2610.19734 §1. Its variants differ in the first two, and all use the same optimiser, a trust-region method 2610.19734 §4. It links no code 2610.19734 §1 §6.

The second paper uses PnP differently. It does not refine a pose from known correspondences. A learned matcher produces the 2-D/3-D pairs, and PnP inside RANSAC gives the pose from them 2205.12257 §3.2. Its PnP step is the last stage of a larger pipeline, so the paper does not measure PnP alone.

Two reuse-licensed papers cover this concept so far. Their numbers come from different setups, a calibration object with added noise and a video dataset of real objects, so they are not directly comparable 2610.19734 §5 2205.12257 §4.1. Both are author-reported.

Variants

Point-based joint estimate

Each image point gives two collinearity constraints, which are quadratic in the dual-number quaternion (r, s). They are summed with penalties that keep r unit-length and orthogonal to s 2610.19734 §3 eq. 22. Rotation and translation are estimated together. Card: Dual-quaternion pose from points and lines. Reported: with 10 noise-free points the translation error is 0.0171 cm, against 2.4277 cm for the linear method 2610.19734 tab. 1 tab. 7. At 460 points the run takes 4.83 s and 10 iterations 2610.19734 tab. 7.

Line-based joint estimate

Each line correspondence gives two constraints, so three lines are the minimum for six unknowns 2610.19734 §2. The line error sums these constraints with the same penalties, and rotation and translation are solved together 2610.19734 §2.2 eq. 13. Reported: 230 noise-free lines take 0.67 s and 8 iterations 2610.19734 tab. 5.

Line-based rotation-then-translation

The rotation is fitted first, with a rotation-only error in r [arxiv:2610.19734 eq. 14]. Then s is found by linear least squares, which gives the translation 2610.19734 §2.2 eq. 15 eq. 7. This is the cheapest variant the paper reports: 30 noise-free lines take 0.03 s and 6 iterations 2610.19734 tab. 3. The paper does not say whether r is refitted after s is solved 2610.19734 §2.2 §5.

PnP inside RANSAC on learned matches

OnePose localizes a query image against a sparse 3-D model of an object, built from a short video scan 2205.12257 §3.1. A graph attention network matches the query's 2-D features to the 3-D points, keeping pairs whose confidence is above a threshold theta 2205.12257 §3.2. PnP with RANSAC then gives the pose from those 2-D/3-D pairs 2205.12257 §3.2. The paper does not name the PnP solver, the RANSAC settings or theta. The official code uses EPnP with a 5-pixel reprojection threshold; the card's Paper vs code section records this 2205.12257 §3.2. Card: OnePose: One-Shot Object Pose Estimation without CAD Models. Author-reported: about 58 ms per frame, against 618 ms for HLoc with SuperPoint and SuperGlue, with accuracy mixed by object size [arxiv:2205.12257 tab. 1] §4.3. Those times exclude 2-D detection and PnP [arxiv:2205.12257 §4.3], so they do not measure this variant's PnP step.

Linear baseline

The paper's reference method is the linear Faugeras-Toscani calibration, run on the same subsets 2610.19734 §5. It is not a variant the paper proposes. The paper reports it as unreliable for translation once image noise is above about half a pixel 2610.19734 tab. 2. At noise 5.00, the translation error is 61.16 cm, against 0.38 cm for the point-based estimate with 100 points 2610.19734 tab. 2 tab. 8.

When to use which

The rows follow the constraints a project has. Results are the paper's, on its own data.

Timings come from an old workstation (Sun SPARC 10), and the paper does not state hardware for the other tables 2610.19734 tab. 3. Compare ratios between variants, not absolute times. Also, the joint point and line runs use different data sizes and error functions, so their timings are not like for like 2610.19734 §6 tab. 7. The OnePose times were measured on an NVIDIA TITAN RTX and exclude PnP, so they do not compare with the first paper's timings 2205.12257 §4.3.

Implementation sketch

The core is the same for every variant: turn each correspondence into constraints on the unknown pose, add the penalties, minimise, then read the pose out. The variants differ in the constraint function and in whether the rotation and translation are solved together.

# Common core of a PnP refinement  [arxiv:2610.19734 §2] [§3] [§4]
# objects: 3-D points or lines; images: their 2-D measurements  [§1]
# intrinsics: known camera parameters  [§2] [eq. 1]
def refine_pose(objects, images, intrinsics, start):
    # Map pixels to camera-frame coordinates  [§2] [eq. 1]
    rays = normalise(images, intrinsics)
    # One or two constraints per correspondence: points or lines  [§2.2] [§3]
    def residuals(x):
        return stack(constraints(obj, ray, x)
                     for obj, ray in zip(objects, rays))
    # Penalties for unit length of r and r.s = 0  [§2.2] [eq. 13]
    # Minimise with a trust-region loop  [§4.1]
    x = trust_region(start, residuals, penalty_weight=50)
    # Read rotation and translation from (r, s)  [eq. 6] [eq. 7]
    return pose_from_dual_quaternion(x)

The RANSAC variant has a different outer shape. It needs no start pose. It fits candidate poses on sampled subsets of the matches and keeps the best 2205.12257 §3.2:

# Robust pose from learned 2-D/3-D matches  [arxiv:2205.12257 §3.2]
# matches: pairs whose confidence is above theta; theta is not stated  [§3.2]
def pose_from_matches(matches, intrinsics):
    # Fit candidate poses on sampled subsets and keep the best  [§3.2]
    # Sample size, threshold and iterations are not stated in the paper
    return pnp_ransac(matches.pts_2d, matches.pts_3d, intrinsics)

Where the paper leaves a choice open, use one convention throughout:

• Normal of an image line. Figure 1 calls n a unit normal, but eq. 3 takes n as (a, b, c) from the image line 2610.19734 fig. 1 eq. 3. The error functions scale with |n|² eq. 13 eq. 22, so the penalty weight of 50 acts differently for each normalisation. Normalise n and state it.
• Factor of one half. The least-squares form carries ½ eq. 23, but the error functions in eqs. 13, 14, 15 and 22 do not. The fixed shift 0.001 on the Hessian depends on this scale 2610.19734 §4 §4.1.
• Initial pose and radius. Neither is given, and the paper's claim that the result does not depend on initialisation is not tested by any experiment 2610.19734 §4.1 §6.

For the representation itself, see Dual-number quaternion.

Contradictions

None on the same quantity. The two papers do not measure the same thing: the first reports translation error on a calibration object with added image noise 2610.19734 §5, and the second reports pose accuracy and runtime on its own video dataset 2205.12257 §4.1 §3.3. The second paper does not compare its PnP step with the linear or nonlinear methods of the first.

Papers

• Dual-quaternion pose from points and lines: a dual-quaternion, trust-region estimator from points, lines or both, with a rotation-first line variant; author-reported gains over a linear method under noise.
• OnePose: One-Shot Object Pose Estimation without CAD Models: a learned matcher from a video-scanned object model to query features, with PnP inside RANSAC; author-reported runtime and accuracy against HLoc and PVNet.