---
kind: concept
slug: hand-eye-calibration
title: Hand-eye calibration
summary: >-
  Finds the fixed transform between a camera and a robot hand from several hand motions. Compares the classical AX = XB formulation with a projection-matrix formulation MY = M'YB, and linear, closed-form and joint non-linear solvers. Author-reported: the joint solver is most accurate in simulation, and the new formulation lowers translation error on the real data sets that test it.
tags: [robotics, calibration]
papers: ["2311.12655"]
related: []
status: draft
updated_at: 2026-10-18
---

## Overview

A camera mounted on a robot hand moves with the hand, but its pose relative to
the hand is fixed and unknown. Hand-eye calibration estimates that fixed
transform by moving the hand several times and recording how the camera and the
hand each moved [arxiv:2311.12655 §1] [§2.1]. Every variant splits the problem
into a rotation part and a translation part [arxiv:2311.12655 §4] [§5]. At
least two independent motions are needed to fix the unknown uniquely
[arxiv:2311.12655 §3]. The intrinsic camera parameters are assumed constant
during the calibration [arxiv:2311.12655 §8].

## Variants

A method has two independent choices: the equation that relates the motions,
and the solver for that equation.

### Formulations

#### Classical: AX = XB

The sensor motion A, the hand motion B and the unknown offset X satisfy
AX = XB [arxiv:2311.12655 §2.1]. For a camera, A comes from two calibrations,
and each calibration is split into intrinsic and extrinsic parameters
[arxiv:2311.12655 eq. 5]. The paper argues that this split is unstable
[arxiv:2311.12655 §2.2]. Card: [Hand-Eye Calibration](/papers/2311.12655).

#### New: MY = M'YB

This form works directly on the 3x4 projection matrices M and M' of the camera
at two positions, relative to a calibration frame. The unknown Y is the
hand-to-calibration-frame transform [arxiv:2311.12655 §2.2]. Because the
intrinsic and extrinsic parts are never separated, it avoids the split that
the paper calls unstable [arxiv:2311.12655 §2.2] [§8]. The paper shows
the two forms have the same mathematical structure, so a solver written for
one applies to the other [arxiv:2311.12655 §8]. Card:
[Hand-Eye Calibration](/papers/2311.12655).

### Solvers

#### Linear, Tsai-Lenz (TL)

The earlier solvers decouple rotation from translation and solve each with
linear algebra [arxiv:2311.12655 §3]. The paper uses the Tsai-Lenz method as
its linear baseline in all three real data sets [arxiv:2311.12655 tab. 1]
[tab. 2] [tab. 3]. Card: [Hand-Eye Calibration](/papers/2311.12655).

#### Closed-form rotation, then translation (CF)

The rotation is a unit quaternion. The rotation residual becomes a quadratic
form in the quaternion, and its minimiser is the eigenvector of the smallest
eigenvalue of a 4x4 matrix [arxiv:2311.12655 §5.1] [eq. 29]. Translation then
follows from a linear least-squares problem with the rotation fixed
[arxiv:2311.12655 §5.1]. The rotation step is the closed form of Faugeras and
Hebert, named here only [arxiv:2311.12655 §8]. Card:
[Hand-Eye Calibration](/papers/2311.12655).

#### Joint non-linear (NL)

Rotation and translation are estimated together, as seven parameters: four for
the quaternion and three for the translation [arxiv:2311.12655 §5.2]
[eq. 30]. The cost sums rotation and translation residuals and adds a penalty
that keeps the quaternion near unit length. The penalty weight is 2 × 10^6 in
all the paper's experiments. The minimisation uses Levenberg-Marquardt
[arxiv:2311.12655 §5.2]. The initial guess and the stopping rule are not
stated [arxiv:2311.12655 §5.2]. Card:
[Hand-Eye Calibration](/papers/2311.12655).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Camera calibration gives 3x4 projection matrices and intrinsics can be treated as constant | New formulation, with any solver | On the two real data sets that test both forms, translation error is lower with the new form for every solver [arxiv:2311.12655 tab. 2] [tab. 3]. Rotation error is not always lower: classical TL has the lowest rotation error on set 3 [arxiv:2311.12655 tab. 3]. |
| Only extrinsic parameters are available | Classical formulation | The paper's first real data set has extrinsic parameters only, and there it tests only the classical form [arxiv:2311.12655 §7] [tab. 1]. |
| Noisy measurements with few motions, in the simulated setting | Joint NL | With two motions and 1% to 6% noise it is the most accurate in all reported settings [arxiv:2311.12655 §6]. With four motions and Gaussian noise, translation error is about 4% for NL and about 6.5% for the other two methods [arxiv:2311.12655 §6]. |
| Limited CPU time | TL or CF | The only timing reported is on a Sparc-10 workstation, on set 3. TL and CF take 0.06 to 0.08 s and NL takes 0.21 to 0.25 s [arxiv:2311.12655 tab. 3]. |
| Only rotation accuracy matters, and the robot's kinematic parameters are not well calibrated | TL or CF for rotation | On real data the linear and closed-form methods can estimate rotation better, because robot translation errors do not follow the simulated noise models [arxiv:2311.12655 §7]. The authors still prefer NL in practice [arxiv:2311.12655 §7]. |
| No trustworthy starting guess for an iterative solver | TL or CF | Both are linear or closed-form and need no starting values. NL needs q0 and t0, the paper does not give them, and the cost is non-linear, so results may depend on the start [arxiv:2311.12655 §5.2]. |

## Implementation sketch

The rotation and translation equations come from pairs of motions. For the
classical form, each motion gives a unit-vector pair (v, v'), a translation
pair (p, p') and a matrix K. The paper states the general forms
[arxiv:2311.12655 eq. 25] [eq. 26]. The mapping of the classical and new
formulations onto these symbols is our reading, not written out in the paper
[arxiv:2311.12655 eq. 16] [eq. 18] [eq. 22] [eq. 23] [eq. 24].

```pseudocode
# Closed-form rotation, then translation  [arxiv:2311.12655 §5.1]
# Input: per motion, v, v', p, p' and K  [eq. 25] [eq. 26]
# Output: unit quaternion q and translation t
def closed_form(motions):
    # 1. Accumulate the 4x4 matrix A over all motions  [eq. 29]
    A = zeros(4, 4)
    for m in motions:
        D = Q(m.v_prime) - W(m.v)            # Q, W  [app. A]
        A = A + transpose(D) @ D
    # 2. Rotation: eigenvector of A for its smallest eigenvalue  [§5.1]
    #    q is defined up to sign, which does not change R
    q = smallest_eigenvector(A)
    R = rotation_matrix(q)                   # [app. A]
    # 3. Translation: stack (K - I) t = R p - p'  [eq. 26], solve linearly
    M = vstack([m.K - I for m in motions])
    b = concat([R @ m.p - m.p_prime for m in motions])
    t = least_squares(M, b)                  # [§5.1]
    return q, t
```

```pseudocode
# Joint non-linear estimate  [arxiv:2311.12655 §5.2]  [eq. 30]
# Input: the same motion data, starting values q0 and t0 (not stated)
# Output: unit quaternion q and translation t
def joint(motions, q0, t0):
    # 1. Cost over the seven parameters  [eq. 30]
    def cost(q, t):
        total = 0
        for m in motions:
            # rotation residual, weight 1  [§5.2]
            total += norm2(m.v_prime - rotate(q, m.v))
            # translation residual, weight 1  [§5.2]
            total += norm2(rotate(q, m.p) - (m.K - I) @ t - m.p_prime)
        # penalty keeping q near unit length, lambda = 2e6  [§5.2]
        total += 2e6 * (1 - dot(q, q)) ** 2
        return total
    # 2. Levenberg-Marquardt  [§5.2]
    return levenberg_marquardt(cost, q0, t0)
```

Here `rotate(q, x)` is the quaternion product q x q-bar [app. A]. The paper
notes that a trust-region method would also suit the joint problem
[arxiv:2311.12655 §5.2].

## Contradictions

- **Real-data rotation of the NL method.** The paper's text describes the
  rotation cost of NL on real data as small [arxiv:2311.12655 §7]. In
  [arxiv:2311.12655 tab. 3], classical NL has a rotation error of 0.258 on
  set 3, against 0.014 for classical TL. The paper does not discuss this gap.
- **Overall ranking.** The discussion says the non-linear method gives the most
  accurate results [arxiv:2311.12655 §8]. On real data, the linear method has
  the lowest classical rotation error on set 3 [arxiv:2311.12655 tab. 3], and
  the paper notes that linear and closed-form rotation can be better on real
  data [arxiv:2311.12655 §7].
- **Trade-off within one table.** On set 2, classical NL has a higher rotation
  error than TL and CF (0.0017 against 0.0014), but a lower translation error
  (0.015 against 0.023 and 0.036) [arxiv:2311.12655 tab. 2].

## Papers

- [Hand-Eye Calibration](/papers/2311.12655): a second formulation, MY = M'YB, built
  on projection matrices, and two solvers: closed-form rotation then translation,
  and joint non-linear optimization.