Depth Maps from Single Images

A depth map stores one distance per pixel, which turns a photo into a 2.5D point cloud but leaves occlusions unknown.

Why Does This Exist?

Full 3D is heavy, but many tasks need only how far each pixel sits. Stereo vision triangulates depth from two views; monocular methods regress it from one. A depth map $D(u, v)$ in meters plus intrinsics unprojects into a point cloud, which is why BEVDet lifts on depth. The limit is one layer: occluded backs have no pixel.

Think of It Like This

A theater seating chart with row numbers

Each seat has a row number but no note about who hides behind tall hats. You know how far every visible head sits, yet the hidden heads have no entry at all.

Pixels are seats and depth is the row. The analogy stops at scale: row numbers are exact while monocular depth needs a scale fix.

How It Actually Works

Pinhole unprojection is $X = (u - c_x) \cdot Z / f$ , $Y = (v - c_y) \cdot Z / f$ with depth $Z = D(u, v)$ , focal $f$ , center $(c_x, c_y)$ . Monocular nets predict inverse depth $1/Z$ for stable near-far balance. Error grows near quadratically because one pixel spans more meters far away.

Worked example

Take $f = 800$ , $(c_x, c_y) = (640, 360)$ , pixel $(840, 360)$ , depth $Z = 10$ m. Then $X = (840 - 640) \times 10 / 800 = 2.5$ m, $Y = 0$ m. A one-pixel error to $u = 841$ gives $X = 2.5125$ m, off by $0.0125$ m. At $Z = 50$ m the same pixel error gives $0.0625$ m, five times worse. That growth is why far BEV cells smear.

Watch Out For

Trusting monocular scale across cameras

One camera's $10$ m is another's $11$ m without calibration, so fused clouds split. The symptom is doubled walls in BEVDet grids. Fix scale per camera on known baselines first.

The Quick Version

Depth maps store meters per pixel and unproject with intrinsics.
One pixel covers more meters far away, so far error grows fast.
Use depth when one visible layer suffices for driving or lifting.
Do not use it for backs of objects; occlusions have no pixels.
Predict inverse depth to balance near precision against far range.