Image Formation and Camera Model

How a 3D physical world is mathematically projected into a 2D pixel array, forming the foundation of computer vision and multi-view geometry.

A 3D point at depth 15 projects through focal length 800 onto pixel (373.33, 346.67), showing how division by depth shrinks distant objects

Why Does This Exist?

Images are flat, but the world is not. Every photograph crushes an entire three-dimensional dimension down to a two-dimensional grid of light intensities, and that lost depth is exactly what vision systems need back.

To build autonomous vehicles, 3D reconstructors, or augmented reality systems, you must understand precisely how that loss happens. The camera model is the mathematical bridge relating a pixel on screen to a physical location in the real world. Without it, a self-driving car sees a pedestrian but has no geometric basis for telling ten feet from a hundred. This page teaches the projection itself; recovering the lost depth belongs to depth estimation, and finding a real camera's numbers belongs to camera calibration.

Think of It Like This

A laser pointer shining through a keyhole

Imagine standing outside a dark room holding a laser pointer, aiming through a tiny keyhole at the opposite wall inside. The point where the beam lands depends purely on your position, your angle, and the distance from the keyhole to the wall.

In a pinhole camera, light from the world acts like millions of tiny lasers. They all travel through one keyhole (the camera centre) and strike a wall (the image sensor). Move the wall further out (a longer focal length) and the image stretches, making objects look larger; move it closer and you capture a wider field of view.

The analogy stops at the sensor. A wall records every landing point continuously, while a real sensor rounds each landing onto a fixed photosite grid and clips brightness beyond its range. Geometry decides where each ray lands, and sampling plus quantization decide what number that landing is worth.

How It Actually Works

Projection runs as a sequence of coordinate changes. Start with an object in global 3D space, $P_{world} = (X_w, Y_w, Z_w)$ , where each component is a distance in world units such as metres.

1. World to camera (extrinsics). The camera sits somewhere in that world, pointing some direction. Its rotation matrix $R$ and translation vector $T$ move the point into the camera's own frame: $P_{cam} = R \cdot P_{world} + T$ . The result $P_{cam} = (X, Y, Z)$ is the same point measured relative to the camera centre, with the $Z$ -axis pointing straight out of the lens.

2. Perspective projection. Light travels in straight lines through the camera centre to the image plane. Under the pinhole model, similar triangles give the continuous image point $(x, y)$ at focal length $f$ : $x = f X / Z$ and $y = f Y / Z$ . Division by $Z$ is the whole perspective effect: doubling depth halves image size.

3. Image plane to pixels (intrinsics). The intrinsic matrix $K$ converts metric coordinates into pixel indices $(u, v)$ , folding in focal lengths in pixel units ( $f_x, f_y$ ) and the principal point $(c_x, c_y)$ where the optical axis meets the sensor: $u = f_x X / Z + c_x$ and $v = f_y Y / Z + c_y$ .

4. One matrix. In homogeneous coordinates the pipeline is a single multiplication, $x = K [R \mid T] X$ , where $[R \mid T]$ stacks rotation beside translation and $X$ carries a trailing $1$ .

Worked example

Take $K$ with $f_x = f_y = 800$ , $c_x = 320$ , $c_y = 240$ , no rotation ( $R$ is identity), translation $T = [0, 0, 5]$ , and world point $[1, 2, 10]$ . Extrinsics give $P_{cam} = [1, 2, 10 + 5] = [1, 2, 15]$ , so depth $Z = 15$ . Then $u = 800 \times 1 / 15 + 320 = 53.33 + 320 = 373.33$ and $v = 800 \times 2 / 15 + 240 = 106.67 + 240 = 346.67$ . The point lands at pixel $(373.33, 346.67)$ , and the same point at twice the depth would land half as far from the centre.

Code

Pinhole projection with the worked example's numbers, standard library only.

1def project_point(point: tuple, k: tuple, t: tuple) -> tuple:2    """Project a world point to pixels with identity rotation."""3    xc, yc, zc = point[0] + t[0], point[1] + t[1], point[2] + t[2]4    fx, fy, cx, cy = k5    return (fx * xc / zc + cx, fy * yc / zc + cy)6
7
8print(project_point((1.0, 2.0, 10.0), (800.0, 800.0, 320.0, 240.0), (0.0, 0.0, 5.0)))9# -> (373.3333333333333, 346.6666666666667)

Identity rotation keeps the snippet short; a real pipeline multiplies by $R$ first.

Watch Out For

Ignoring lens distortion

The pinhole model assumes light travels perfectly straight. Real lenses add radial distortion (straight lines bowing near edges, the fisheye look) and tangential distortion. Running geometry on distorted pixels makes the $K$ equations wrong exactly where wide lenses see most.

Un-distort first with the calibration coefficients, then project. Never absorb distortion into $K$ ; it is not linear and the matrix cannot hold it.

Projecting points behind the camera

The formula $x = f X / Z$ happily projects points with $Z \lt 0$ into valid pixels, mirroring what sits behind the lens into the frame. A renderer or tracker that skips the sign check hallucinates objects that are not there.

Check $Z \gt 0$ in the camera frame before rendering or matching any point. Behind-camera points are invisible by definition, not by accident.

The Quick Version

Image formation collapses a 3D world into a 2D pixel array, losing depth.
Extrinsics ( $R$ , $T$ ) move the world into the camera frame; intrinsics ( $K$ ) map it to pixels.
The pinhole divides $(X, Y)$ by depth $Z$ , so distant objects shrink (perspective).
Our fixture lands at $(373.33, 346.67)$ from depth $Z = 15$ with focal length $800$ .
Calibrate for $K$ , un-distort before geometry, and reject points with $Z \lt 0$ .