Robot Hand-Eye Calibration

Move a wrist-mounted camera around a stationary target. The camera-to-wrist transform must explain every view with the same rigid mounting geometry.

One rigid camera mount makes both wrist poses agree on the stationary target.

Why Does This Exist?

A camera locates a bolt accurately in its own coordinates, yet the arm reaches beside it. The missing quantity may be the camera's position and orientation relative to the wrist. Hand-eye calibration estimates that fixed rigid transform from synchronized robot poses and camera observations. Naming frames correctly does not measure the mount.

A transform rotates and translates point coordinates between frames; composition chains those changes. Coordinate frames introduces that algebra. Here the transform is unknown. Intrinsic camera calibration instead measures projection and distortion inside the camera; dynamics identification measures another part of the system entirely.

Think of It Like This

A torch taped to a hinged ruler

Tape a torch at an unknown offset from a ruler's hinge. Turn and move the ruler while aiming at one mark on the wall. The offset rotates with the ruler, while the wall mark stays still. Find the attachment geometry that makes all the sightings agree. The ruler maps to the robot wrist, the torch to the camera and the wall mark to a calibration target. The analogy stops at the measurements: a camera estimates a full target pose from several known marks, not just the direction of a light beam.

How It Actually Works

Which transform is unknown?

Write ${}^aT_b$ for the homogeneous transform that takes coordinates from frame $b$ into frame $a$ . Let $G_i={}^bT_{g_i}$ be gripper-to-base pose at capture $i$ , $X={}^gT_c$ camera-to-gripper mount, and $Y_i={} ^{c_i}T_t$ target-to-camera pose. The superscripts $b,g,c,t$ identify base, gripper, camera and target frames. In an eye-in-hand setup, the target is fixed in the base frame, so

$G_iXY_i=G_jXY_j.$

Multiplying by inverses gives $AX=XB$ , with $A=G_i^{-1}G_j$ and $B=Y_iY_j^{-1}$ . Both represent relative motion, in different frames. Reversing both motion-pair directions consistently gives another valid convention; reversing just one does not.

OpenCV's calibrateHandEye contract accepts gripper-to-base and target-to-camera pose arrays and returns camera-to-gripper. Its documentation derives the relative-motion equation and requires rotations about nonparallel axes for a spatial calibration. An eye-to-hand setup has a fixed camera and moving target; derive its chain before reusing an API call.

Recover an offset in a planar fixture

For a transparent calculation, restrict motion to a plane and suppose the mount rotation $R_X$ is already known to be identity. Let the relative wrist motion have rotation $R_A=R_{90}$ , a counterclockwise quarter-turn, and translation $t_A=(0.3,0)$ m. Observations imply $R_B=R_{90}$ and $t_B=(0,-0.1)$ m. Expanding the translation part of $AX=XB$ gives

(R_A-I)t_X=R_Xt_B-t_A, \qquad \begin{bmatrix}-1&-1\\1&-1\end{bmatrix} \begin{bmatrix}t_x\\t_y\end{bmatrix} =\begin{bmatrix}-0.3\\-0.1\end{bmatrix}.

The two rows say $t_x+t_y=0.3$ and $t_x-t_y=-0.1$ , giving $t_X=(0.1,0.2)$ m. The determinant is 2, so this restricted translation problem has a unique solution. It does not solve an unknown spatial mount from one motion.

Set absolute wrist poses $G_1=I$ and $G_2=A$ . The camera origins become $(0.1,0.2)$ and $(0.1,0.1)$ m; the second camera rotates 90 degrees. A fixed target at $(0.4,0.4)$ m appears at translation $(0.3,0.2)$ in camera 1 and $(0.3,-0.3)$ in camera 2. Composing each chain recovers $(0.4,0.4)$ . The diagram uses equal x/y scales: equal-length solid mounts rotate with the wrist, and dashed observations meet at the same diamond. The line segments show origins and sightings; camera optical directions are omitted.

What makes real captures informative?

The rotational equation is $R_AR_X=R_XR_B$ ; translation is solved after or jointly with rotation. Pure translation leaves $(R_A-I)t_X$ zero and cannot determine the mount offset this way. Repeated turns about one axis leave spatial ambiguity. Acquire diverse, well-separated rotations with the target visible, rather than many nearly identical poses. More pairs from the same captures do not become independent measurements just because a solver accepts them.

Evaluate held-out captures by comparing $G_iXY_i$ across views, separating translation error in metres and rotation error in radians. Do not add them without an explicit weight or characteristic length. Then check a physical target in the working volume: kinematic error, target dimensions, focus changes and wrist flex can produce a consistent but physically wrong solution. Extra calibration effort buys a measurable mapping; a bolted fixed fixture measured directly can be simpler for a narrow task.

Code

This dependency-free Python solves the restricted translation fixture and exposes an incorrect mount through its closure residual. Rotation is given, not estimated.

1from math import hypot2
3
4def solve2(a: float, b: float, c: float, d: float,5           u: float, v: float) -> tuple[float, float]:6    det = a*d - b*c7    if abs(det) < 1e-12:8        raise ValueError("Translation is not identifiable from this pair")9    return ((d*u-b*v)/det, (a*v-c*u)/det)10
11
12def closure_error(tx: float, ty: float) -> float:13    # AX translation is R90*tX+tA; XB translation is tX+tB.14    return hypot(-ty+0.3-tx, tx-ty+0.1)15
16
17tx, ty = solve2(-1.0, -1.0, 1.0, -1.0, -0.3, -0.1)18print(f"mount: {tx:.3f}, {ty:.3f} m")19assert closure_error(tx, ty) < 1e-1220print(f"wrong mount residual: {closure_error(tx+0.02, ty):.6f} m")

The recovered mount is 0.100, 0.200 m; the offset perturbation produces a 0.028284 m relative-motion closure residual. That residual is not directly the tool's target-position error.

Watch Out For

A plausible inverse solves the wrong frame chain

Residuals depend strongly on pose or the mapped point moves when the stationary target does not. Write each transform's source and destination beside it. Check API directions, robot flange versus tool frame, units and pose conventions. A transform and its own inverse always round-trip; that alone cannot verify physical calibration.

Many captures hide uninformative motion

The solver changes dramatically when one view is removed, despite many samples. Add rotations around distinct axes and inspect held-out chain disagreement. Pair each image with the pose at exposure time; motion during capture creates a timing error that more static-pose fitting cannot remove.

The Quick Version

Hand-eye calibration estimates a rigid sensor mount from matched robot and target poses.
The fixed-target chain yields $AX=XB$ only with consistent relative-motion directions.
The planar example recovers a 0.1 m by 0.2 m offset with known mount rotation.
Spatial calibration needs informative rotations, synchronized captures and held-out checks.
Low algebraic residual does not establish physical targeting accuracy.