Essential Matrix for Calibrated Cameras

The essential matrix is the calibrated cousin of the fundamental matrix: with intrinsics known, five point pairs recover the rotation and the direction of travel between two views, up to scale.

With calibration known, E recovers rotation and travel direction from five matches, up to scale

Why Does This Exist?

The fundamental matrix works without calibration but needs seven or eight matches and never yields motion directly. Once intrinsic parameters are known, pixels become true rays and the geometry tightens: five matches determine the relative pose, and decomposing $E$ hands over the rotation plus the travel direction. This is how structure from motion bootstraps its first camera pair and how visual SLAM initializes motion before the map exists.

Think of It Like This

A steel ruler with no units marked

The fundamental matrix is a knotted string: it orders positions along a line but measures nothing. Calibration upgrades it to a steel ruler, straight and true, so angles and directions read exactly. But the ruler has no numbers printed: it says "that way" without saying "how far."

That missing scale is permanent to pure vision. Two journeys with identical headings but different lengths produce the identical $E$ . The analogy stops at sensor fusion: an IMU, GPS, or known object writes the numbers on the ruler from outside.

How It Actually Works

With calibrated ray coordinates $x$ and $x'$ (pixels premultiplied by $K^{-1}$ ), the constraint keeps its shape: $x'^T E x = 0$ , where $E = [t]_{\times} R$ . Here $[t]_{\times}$ is the skew-symmetric cross-product matrix of the translation direction $t$ , and $R$ is the relative rotation. Freedom counting gives $3$ (rotation) $+ 2$ (direction, since scale is free) $= 5$ degrees of freedom, hence five-point solvers. The two matrices connect through $E = K'^T F K$ .

Decomposing $E$ yields four $(R, t)$ combinations, of which exactly one places the triangulated points in front of both cameras. That visibility test is the cheirality check, and it is the only disambiguation needed.

Worked numbers

Take $R$ as identity and $t = (1, 0, 0)$ , so $E$ has rows $(0,0,0)$ , $(0,0,-1)$ , $(0,1,0)$ . With $x = (0.5, 0.2, 1)$ :

E x = (0,\ -1,\ 0.2)

and with $x' = (1.5, 0.2, 1)$ the residual is $1.5 \times 0 + 0.2 \times (-1) + 1 \times 0.2 = 0$ . Decomposing this $E$ returns the identity rotation and the $+x$ travel direction among its four candidates, and cheirality keeps that physical one.

Code

1def essential_residual(xp: tuple[float, float], x: tuple[float, float]) -> float:2    e = ((0.0, 0.0, 0.0), (0.0, 0.0, -1.0), (0.0, 1.0, 0.0))  # skew t with R as identity3    ex = (e[0][0] * x[0] + e[0][2], e[1][2], e[2][1] * x[1])  # E x in closed form4    return xp[0] * ex[0] + xp[1] * ex[1] + ex[2]5
6
7print(essential_residual((1.5, 0.2), (0.5, 0.2)))  # -> 0.0

Watch Out For

Reading absolute scale out of E

By convention $\lVert t \rVert = 1$ , so the translation is a direction with an arbitrary unit length. Treating it as meters yields trajectories with perfect shape and wrong size. This is the same scale ambiguity structure from motion documents: fuse an IMU, GPS, or a known-size object to pin the gauge.

Feeding it a pure rotation

With no translation, $t = 0$ makes $E$ the zero matrix and the constraint $0 = 0$ accepts everything. Decomposition output is then garbage with a confident face. Detect the rotation-only case (negligible translation, high rotational consistency) and switch to a homography, which owns pure rotations via $H = K R K^{-1}$ .

The Quick Version

The essential matrix acts on calibrated rays and satisfies $x'^T E x = 0$ with $E = [t]_{\times} R$ .
Five matches suffice, versus seven or eight for the fundamental matrix.
Decomposition gives four $(R, t)$ candidates; the cheirality check keeps the physical one.
Translation comes out as a direction only; absolute scale is unobservable.
Pure rotation zeroes $E$ out, so rotation-only pairs belong to a homography.
It relates to $F$ through $E = K'^T F K$ .