Disparity Estimation in Stereo Pairs

Slide the right photo over the left until each patch lines up. The size of that slide per pixel is the disparity, and big slides mean close things.

Rectified patches slide along one epipolar row and the best match sets the disparity

Why Does This Exist?

Epipolar geometry and stereo matching cuts the search to one row, but something still has to pick the winning pixel on that row. That picker is disparity estimation: for every left pixel it scores candidate right pixels and keeps the shift with the lowest cost. Without it there is no disparity map, and without a disparity map the depth formula $Z = f B / d$ has no input.

This page covers the matching cost and the search itself. The geometry behind the rows lives in epipolar geometry; the depth conversion and its failure modes live in stereo vision.

Think of It Like This

Sliding tracing paper over a drawing

Lay tracing paper over a drawing and slide it right until the outlines line up. The distance you slid is the answer, and flat blank areas never snap into place because any position looks the same.

Pixels are the drawing and the matching cost is your eye judging alignment. The analogy stops at lighting: paper never changes brightness while real cameras auto-expose, which is why raw brightness costs fail across views.

How It Actually Works

After rectification every match sits on the same image row, so disparity $d = x_L - x_R$ is a one-dimensional shift. A data cost such as sum of absolute differences over a small window scores each candidate $d$ ; semi-global matching adds a smoothness penalty that charges $P1$ for a one-pixel disparity jump and $P2$ for larger jumps, then optimizes along several path directions. The winner-takes-all disparity is $\arg\min_d C(d)$ , followed by a left-right consistency check that keeps only matches both views agree on.

Depth follows from similar triangles as $Z = f B / d$ , where $f$ is focal length in pixels and $B$ is baseline in meters. Error grows with $Z^2$ : differentiating gives $\Delta Z \approx Z^2 \Delta d / (f B)$ .

Worked numbers

Take $f = 800$ px, $B = 0.12$ m, and measured disparity $d = 80$ px. Then $Z = 800 \times 0.12 / 80 = 96 / 80 = 1.2$ m. A one-pixel mistake to $d = 79$ gives $Z = 96 / 79 \approx 1.215$ m, off by about $0.015$ m. At $d = 8$ px (about $12$ m out) the same one-pixel error swings depth by roughly $1.7$ m, which is why far stereo is brittle.

Code

1def depth_from_disparity(d: float, f: float, baseline: float) -> float:2    return f * baseline / d  # similar triangles on a rectified pair3
4
5print(depth_from_disparity(80.0, 800.0, 0.12))  # -> 1.26print(depth_from_disparity(79.0, 800.0, 0.12))  # -> 1.2151898734177216

Watch Out For

Matching textureless walls by cost alone

On blank paint every candidate shift scores nearly the same, so winner-takes-all returns noise with high confidence. The symptom is speckled disparity across walls and floors. Fix it with a larger window, a smoothness penalty, or an active texture projector instead of trusting the raw argmin.

Reading far disparity as exact depth

One pixel spans $Z^2 / (f B)$ meters, so a single-pixel disparity error at $12$ m outweighs a ten-pixel error at $1$ m. Clamp the usable range by baseline, and never feed far stereo depth into grasping without inflating its uncertainty.

The Quick Version

Disparity estimation scores candidate shifts along each rectified epipolar row.
The disparity map holds $d = x_L - x_R$ per pixel for the depth formula.
Semi-global matching adds smoothness penalties $P1$ and $P2$ to tame noise.
Depth is $Z = f B / d$ , with error growing as range squared.
With $f$ 800 and baseline $0.12$ m, disparity $80$ means depth $1.2$ m.
Blank regions and far range are the two standard failure modes.