Depth Estimation from Images Guide

Turn flat photos into meters per pixel. Depth estimation is the umbrella that splits into stereo, single-image, and filled-sparse methods.

One umbrella splits into stereo, monocular, and completion branches with different scale answers

Why Does This Exist?

Robots need meters, not just labels, yet cameras record brightness. Depth estimation exists to bridge that gap in all its forms: stereo vision triangulates from two views, monocular nets regress from one, and completion densifies sparse lidar. This overview maps the family so readers pick the right branch; each branch then gets its own page.

Detail lives in monocular depth estimation, stereo depth estimation, and depth completion. Representation basics are depth maps.

Think of It Like This

Judging distance three ways

Cover one eye and judge by size memory; open both and feel the stereo snap; then tap a cane to fill the gaps. All three give distance, with different guarantees.

One eye is monocular, two eyes are stereo, and the cane is sparse lidar. The analogy stops at learning: people carry size memory while nets need training data for every new domain.

How It Actually Works

Stereo fixes scale through $Z = f B / d$ with known baseline $B$ . Monocular predicts either metric depth $Z$ or affine-invariant disparity $1/Z$ up to unknown scale $s$ and shift $t$ , resolved later by median alignment $s \times \hat{Z} + t$ against sparse truth. Completion takes sparse truth plus the image and regresses dense depth with a sparsity-aware encoder. Evaluation reports absolute relative error $\mathrm{AbsRel} = |Z - \hat{Z}| / Z$ and root mean square error over valid pixels.

Worked numbers

A pixel with truth $Z = 4.0$ m and prediction $\hat{Z} = 4.4$ m gives $\mathrm{AbsRel} = 0.4 / 4.0 = 0.10$ . Over four pixels with errors $0.4$ , $0.2$ , $0.0$ , $0.6$ m, RMSE is $\sqrt{(0.16 + 0.04 + 0 + 0.36)/4} = \sqrt{0.14} \approx 0.374$ m. Stereo with $f$ $800$ and $B$ $0.12$ m at disparity $24$ px gives $Z = 4.0$ m exactly, no alignment needed.

Code

1def abs_rel(pred: float, gt: float) -> float:2    return abs(pred - gt) / gt  # scale-aware error per pixel3
4
5print(abs_rel(4.4, 4.0))  # -> 0.09999999999999987

Watch Out For

Comparing metric and affine outputs raw

Affine models output disparity up to scale and shift, so raw numbers look broken against meters. The symptom is near-perfect shapes with absurd AbsRel. Median-align predictions to ground truth before scoring, and never deploy affine depth without a scale source.

The Quick Version

Depth estimation predicts meters per pixel from images plus optional sparse range.
Stereo gives metric depth by baseline; monocular gives shape up to scale.
Completion densifies sparse lidar guided by the image.
AbsRel and RMSE over valid pixels are the standard scores.
A $0.4$ m error at $4$ m is AbsRel $0.10$ .
Align affine outputs before comparing or deploying.