Multi-View Geometry from Many Views

Two views give lines to search; three or more views pin a point down. Multi-view geometry is the shared rulebook for turning many photos into one 3D shape.

Three cameras triangulate one point where pairwise epipolar lines agree in the third view

Why Does This Exist?

Two cameras recover depth along one baseline, but one pair cannot tell a good match from a lucky one. A third camera breaks the tie: a false match that satisfies one epipolar geometry constraint almost never satisfies the second. That is why structure from motion and city-scale reconstruction use many views instead of stopping at stereo.

This page covers the joint constraint across three or more calibrated or uncalibrated views. Pairwise tools live in the fundamental matrix and the essential matrix; per-pixel depth lives in stereo vision. The camera calibration and parameters page holds the intrinsics both need.

Think of It Like This

Three surveyors with compass bearings

Three surveyors stand around a field and each shouts the compass bearing to a flag. Two bearings already cross at one spot, but a misheard number still gives a crossing somewhere. The third bearing is the check: only the true flag position agrees with all three lines at once.

Cameras are surveyors and epipolar lines are bearings. The analogy stops at noise: surveyors hear exact numbers while pixels quantize, so three rays never meet perfectly and the crossing becomes a least-squares point.

How It Actually Works

Each camera maps a 3D point $X$ to a pixel $x_i = P_i X$ , where $P_i = K_i [R_i | t_i]$ holds intrinsics $K_i$ , rotation $R_i$ , and translation $t_i$ . For calibrated pairs the essential constraint is $x_2^T E x_1 = 0$ ; for uncalibrated pairs it is $x_2^T F x_1 = 0$ . With three views the trifocal tensor $T$ plays the same role: given a match in views one and two, $T$ predicts the epipolar line in view three, and the true third match must lie on it.

Cheirality adds one more rule: a valid $X$ must sit in front of every camera that sees it ( $Z > 0$ in each camera frame). Points behind any camera are rejected even when their reprojection error is small.

Worked numbers

Take two calibrated cameras with $f = 800$ , baseline $0.2$ m along $x$ , both looking forward. A world point at $(X, Y, Z) = (0.4, 0.0, 2.0)$ projects to $u_1 = 800 \times 0.4 / 2.0 + 320 = 480$ in view one and $u_2 = 800 \times (0.4 - 0.2) / 2.0 + 320 = 400$ in view two, so disparity is $80$ px. A third camera shifted $0.2$ m vertically predicts $v_3 = 240 - 800 \times 0.2 / 2.0 = 160$ . A candidate third match at $(400, 160)$ satisfies all three rays; one at $(400, 170)$ misses by $10$ px and is rejected as inconsistent.

Code

1def predict_third_row(u1: float, disparity: float, v_center: float, f: float, base_y: float, z: float) -> float:2    u2 = u1 - disparity  # rectified stereo gives the second view for free3    v3 = v_center - f * base_y / z  # third camera shifted vertically4    return u2, v35
6
7print(predict_third_row(480.0, 80.0, 240.0, 800.0, 0.2, 2.0))  # -> (400.0, 160.0)

Watch Out For

Trusting two views when three disagree

A pair with low residual feels done, yet mismatches pass pairwise tests regularly on repeating windows and fences. The symptom is a clean two-view model that collapses when a third view arrives. Always check the third-view transfer error before accepting a match, and drop points that miss by more than two pixels.

Keeping points behind a camera

Triangulation without the cheirality check returns points with $Z < 0$ that reproject perfectly. They look fine in pixel error and poison everything downstream. After solving, transform $X$ into each camera frame and keep only points with positive depth in all views.

The Quick Version

Multi-view geometry joins pairwise two-view constraints across three or more cameras.
The trifocal tensor predicts the third-view line from a match in the first two views.
A true 3D point must satisfy every pairwise epipolar constraint, not just one.
Cheirality requires the point to sit in front of each camera that observes it.
Two-view inliers that miss the third view by pixels are mismatches to drop.