Stereo Vision

Two cameras separated by a known baseline see the same point shifted by a disparity, and that shift triangulates depth with no active sensor.

Two cameras 12 centimeters apart image one point with a 35 pixel shift, and dividing focal length times baseline by that shift triangulates 2.4 meters of depth.

Why Does This Exist?

One camera flattens the world: a toy car nearby and a real car far away can project to identical pixels, and no amount of staring at a single image recovers the missing scale. Active sensors answer with lasers or projected patterns, but they cost more, fail in bright sunlight, and need power budgets a small robot may not have. Stereo vision answers with a second ordinary camera. Two views separated by a known baseline distance disagree by a measurable shift, and that disagreement triangulates depth passively, outdoors, on cheap hardware.

The geometry behind the search lives in epipolar geometry and stereo matching, the single-camera counterpart in depth estimation, and the moving-camera generalization in structure from motion. This page covers calibration plus rectification, the matching step, the triangulation formula with a hand-computed example, and the two conditions that break it.

Think of It Like This

The thumb and eye experiment

Hold a thumb at arm's length, close one eye, and note which background object it covers. Switch eyes and the thumb jumps sideways against the background. Bring the thumb to your nose and the jump grows huge; aim at a far wall and it nearly vanishes. Big jump means close, small jump means far.

The two eyes are the cameras, the jump is the disparity in pixels, and the brain's distance sense is the triangulation formula below.

The analogy stops at the brain. Eyes swivel, focus, and fuse effortlessly, while cameras need calibration plus rectification before any matching works, and a blank wall that your brain still places correctly leaves the algorithm with nothing to match.

How It Actually Works

Calibrate, rectify, then match

Before any depth math, both cameras are calibrated: the focal length $f$ in pixels and the baseline $B$ between camera centers must be known. Images are then rectified, projected onto a common plane so matching points share horizontal rows. Rectification shrinks correspondence from a 2D whole-image search to a 1D walk along one row. Matching then finds, for each left-image pixel at $x_L$ , the same feature in the right image at $x_R$ , using block comparison or learned matchers, and records the disparity $d = x_L - x_R$ in pixels.

The depth formula

Similar triangles through the two camera centers give $Z = f \cdot B / d$ , where $Z$ is depth, $f$ is focal length in pixels, $B$ is baseline in world units, and $d$ is disparity in pixels. Depth runs inverse to disparity: double the shift, halve the distance.

Worked example: one point, two depths

Take $f = 700$ pixels and $B = 0.12$ meters, so $f \cdot B = 84$ . A point with disparity $d = 35$ pixels sits at $Z = 84 / 35 = 2.4$ meters. A second point with $d = 7$ pixels sits at $Z = 84 / 7 = 12.0$ meters. The nonlinearity bites at range: a one-pixel matching error on the far point gives $84 / 6 = 14.0$ or $84 / 8 = 10.5$ meters, a spread of $3.5$ meters from a single pixel. Close work is precise; distant work degrades fast, which is why baselines grow with the operating range.

Code

1def stereo_depth(disparity_px, focal_px=700.0, baseline_m=0.12):2    """Triangulate depth in meters from disparity in pixels."""3    return focal_px * baseline_m / disparity_px4
5
6for d in (35, 7, 6, 8):7    print(d, stereo_depth(d))8# -> 35 2.49# -> 7 12.010# -> 6 14.011# -> 8 10.5

Watch Out For

Matching on textureless surfaces

Blank walls, smooth floors, and clear skies give the matcher uniform neighborhoods with no distinctive feature to lock onto. The symptom is holes or wild spikes in the disparity map exactly over the plainest regions. Confirm scenes carry texture, or project artificial pattern onto them, before trusting the depth output.

Trusting far-field depth like near-field depth

Because $Z$ runs as $1/d$ , error balloons with distance: the worked example turns one pixel into $3.5$ meters of spread at $12$ meters. The symptom is confident-looking distant obstacles that drift meters between frames. Size the baseline to the operating range and down-weight or discard far-field readings in planning.

The Quick Version

Two calibrated cameras turn a pixel shift $d = x_L - x_R$ into depth $Z = f \cdot B / d$ .
Rectification reduces matching from a 2D search to a 1D walk along image rows.
With $f = 700$ and $B = 0.12$ , disparities of $35$ and $7$ pixels mean $2.4$ and $12.0$ meters.
One-pixel errors cost centimeters nearby but meters far away.
Textureless regions and darkness starve the matcher, so verify texture before trusting depth.