Epipolar Lines Shrink Stereo Search

Two cameras looking at the same point are tied by geometry, so the match for any left pixel must sit on one specific line in the right image. Stereo search walks that line instead of the whole picture.

One left pixel generates a single epipolar line in the right view, so the match is found on that row and depth reads 2.0 metres from a disparity of 50 pixels

Why Does This Exist?

Two cameras watching one scene must agree on which pixels show the same 3D point, and the naive approach compares every left patch against every right patch. On a 640 by 480 pair that is hundreds of thousands of candidates per pixel, slow and full of lookalikes: repeated windows and fences match everywhere. The geometric relationship between the views exists to delete that work. Because both cameras see the same point through known centers, the match is trapped on one line, and search drops from an area to a row.

This page covers the constraint and the matching that uses it. The algebra behind the constraint lives in fundamental matrix for uncalibrated pairs and essential matrix for calibrated ones, while the depth stage that follows matching lives in stereo vision and depth estimation.

Think of It Like This

Two guards watching one runner

Two guards stand at opposite ends of a field watching the same runner. The left guard radios, "my runner is along my line of sight," and the right guard sees that whole line of sight edge-on, as a single stripe across her own view. She never scans the field; she scans the stripe.

The guards are the two cameras, the line of sight is the ray through a pixel, and the stripe is the epipolar line. Where it stops: guards also recognize faces, while cameras only compare pixel patches, so geometry narrows the search but appearance still decides the match, and blank walls defeat it.

How It Actually Works

Name the parts. The baseline joins the two camera centers. Where the baseline pierces each image plane sits an epipole. A 3D point plus the two centers defines a plane, the epipolar plane, and that plane slices each image in an epipolar line. A left pixel $x$ back-projects to a ray, and the right camera sees that entire ray edge-on as the line $l'$ , so the partner pixel $x'$ must satisfy the epipolar constraint $x'^T F x = 0$ , where $F$ is the fundamental matrix for uncalibrated cameras or folds into $E$ once calibration is known.

Rectified stereo warps both images so every epipolar line becomes a horizontal row and both epipoles move to infinity. Matching then walks one row: for a left coordinate $x_L$ and its partner $x_R$ , the disparity is $d = x_L - x_R$ , and depth is $Z = f B / d$ , with focal length $f$ and baseline $B$ . Large disparity means close, small disparity means far, and the division is why distant objects get noisy depth first.

Worked example

Take $f = 500$ pixels and $B = 0.2$ metres. A left pixel at $x_L = 320$ matches on its epipolar row at $x_R = 270$ , so $d = 320 - 270 = 50$ and $Z = 500 \times 0.2 / 50 = 2.0$ metres. A second pixel matches at $x_R = 275$ against $x_L = 300$ , so $d = 25$ and $Z = 500 \times 0.2 / 25 = 4.0$ metres. Halving the disparity doubles the depth, which is exactly the inverse relationship the diagram draws as two arrows of length 50 and 25 pixels pointing at 2.0 and 4.0 metres.

Code

The snippet below replays the worked example with stdlib Python only: two disparities, two depths, no images needed.

1f_px = 500.02baseline = 0.23
4d1 = 320 - 2705print(d1, round(f_px * baseline / d1, 2))6# -> 50 2.07
8d2 = 300 - 2759print(d2, round(f_px * baseline / d2, 2))10# -> 25 4.0

Disparity 50 gives 2.0 metres and disparity 25 gives 4.0 metres, confirming the diagram by hand. Real matchers add patch comparison along the row, as feature matching describes, but the row itself comes from the geometry above.

Watch Out For

Searching horizontal rows on unrectified images

Rectification is what makes epipolar lines horizontal. The symptom is a matcher that finds nothing along rows on raw pairs and gets blamed for bad features. Fix: calibrate and rectify first with camera calibration, or intersect the true epipolar line $l' = F x$ before searching.

Blank walls and hidden points returning confident matches

A textureless wall offers identical patches all along the line, and an occluded point has no partner at all. The symptom is holes or smeared stripes in the depth map where walls and edges sit. Fix: mark low-confidence matches as unknown instead of filling them, and fuse several views through triangulation where occlusion is expected.

The Quick Version

Each left pixel matches on exactly one epipolar line in the right view, never the whole image.
The constraint reads $x'^T F x = 0$ , with $F$ uncalibrated and $E$ calibrated.
Rectified pairs search one horizontal row per pixel, giving disparity $d = x_L - x_R$ .
Depth is $Z = f B / d$ , so disparity 50 gives 2.0 metres and disparity 25 gives 4.0 metres here.
Unrectified rows, blank walls, and occlusions are the three failures to check first.