Visual explainer
Camera Projection and Pinhole Model
How one hole turns 3D depth into flat 2D size, and why size alone lies.
1. Why Depth Disappears
A pinhole camera turns directions in 3D into positions in a 2D image. Size and distance become entangled: double an object's height and its depth, and its projected size stays unchanged. The two arrows occupy the same viewing angle.
2. The Physical Camera
Trace the upper ray through the hole: it reaches the bottom of the rear sensor. This is physical inversion. To keep the following equations positive, we'll place a virtual image plane in front of the camera instead.
3. Similar Triangles
The small triangle has the same slope as the large one: x/f = X/Z. Here X is the point's transverse coordinate, Z its depth, and f the plane's distance. With consistent units, 2 × 1 / 4 = 0.5.
4. Change The Focal Length
Keep the sensor width and scene fixed. Doubling focal length doubles the point's offset while narrowing the angle between the sensor's edge rays. This changes magnification; it doesn't move the camera closer.
5. Change The Depth
Now keep focal length and object size fixed. Depth is in the denominator, so moving from 4 to 8 halves the image height. Both sensor drawings use the same scale.
6. Measure In Pixels
Calibration often expresses focal length in pixels. A scene ratio of one quarter and fx = 800 px produce a 200-pixel horizontal offset. That offset is measured from the principal point, not the left border.
7. Locate The Pixel
OpenCV's camera model adds the principal point: u = fx X/Z + cx, and similarly for v. Here cx = 320, so u = 520. Pixel axes run right and down; the principal point needn't be exactly centered.
8. Account For The Lens
The ideal model maps straight lines to straight lines. Real lenses can add radial and tangential distortion. This illustrative grid shows barrel distortion; calibration estimates additional coefficients to correct it.
9. A Hole Has Limits
A wider opening sends light from one point across a larger sensor patch. Narrowing it reduces geometric blur and light throughput. At very small sizes, diffraction blurs the image again. There is no universally best hole size.
10. Where It Breaks
Every point along the ray has the same projection. One image coordinate cannot determine depth without additional information: another view, measured size, or scene assumptions. A learned depth estimate supplies prior knowledge; the projection equation alone does not. In robotics, active range sensors and stereo pairs resolve this scale ambiguity by measuring external depth.
The Quick Version
- Projection preserves direction while discarding depth along a ray.
- A physical rear sensor inverts the image.
- A virtual front plane gives
x = fX/Z. - Longer focal length enlarges projections and narrows the view.
- Doubling depth halves size when object height stays fixed.
- Focal length in pixels converts ratios to pixel offsets.
- Adding the principal point gives image coordinates.
- Real lenses need distortion terms beyond ideal projection.
- Pinhole size trades light, geometric blur, and diffraction.
- Recovering scale needs information beyond a single projected point.