Camera Models: Pinhole, Intrinsic/Extrinsic

A camera turns each 3D point into a pixel by shifting it into the camera frame and dividing by depth. One matrix per step does the work.

$A 3D point at X 1 Y 2 Z 4 projects through the pinhole to 125 pixels right and 250 up from the center, landing at pixel 445 490$

A 3D point at X 1 Y 2 Z 4 projects through the pinhole to 125 pixels right and 250 up from the center, landing at pixel 445 490

Why Does This Exist?

Every photo, video frame, and robot depth map starts as 3D points and ends as 2D pixels. Without a precise map between the two, a self-driving car cannot tell whether a pedestrian is 4 meters or 40 meters away, and a 3D reconstruction pipeline cannot fuse two views of one statue.

The pinhole model is that map. It says each 3D point travels in a straight line through a single center point and lands on the image plane, scaled by focal length over depth. Two matrices hold the numbers: the extrinsic matrix records where the camera sits in the world, and the intrinsic matrix records how the camera turns landed rays into pixel rows and columns.

This page covers one camera and its two matrices. Two cameras looking at one scene add a new constraint between their images, which belongs to epipolar geometry.

The one idea you need first

A linear transformation is a matrix times a vector: it rotates, scales, or shears the input but keeps the origin fixed and lines straight. Camera projection is almost one, with a single extra step. After the matrices multiply, you divide by depth, which is what makes far things small. That division is the only nonlinear part.

Think of It Like This

Tracing a statue on a window

Stand a statue behind a glass window and trace its outline with a marker while keeping your eye perfectly still. Your eye is the pinhole, the glass is the image plane, and each traced dot sits exactly where the straight line from your eye through the statue pierces the glass.

Move the statue twice as far and its tracing shrinks to half the size, because the piercing point slides toward the middle. That slide is the divide-by-depth step. A longer focal length is a window held further from your eye: the same statue traces larger.

Where the analogy stops: your eye sees in all directions at once while the model covers one rectangular frame, and real lenses bend rays the pinhole keeps straight. Calibration software estimates that bending separately.

How It Actually Works

Write a world point as $X = [X, Y, Z]$ and its pixel as $x = [u, v]$ . Projection runs in two matrix steps plus one division:

x = K \, [R \mid t] \, X

In plain words: $R$ is a $3 \times 3$ rotation matrix saying which way the camera looks, $t$ is a 3-vector saying where the camera sits, and together $[R \mid t]$ moves the point from world coordinates into camera coordinates. $K$ is the $3 \times 3$ intrinsic matrix holding focal length $f$ and optical center $(c_x, c_y)$ , and it turns camera coordinates into pixels. $X$ here means the homogeneous form $[X, Y, Z, 1]$ so the translation $t$ can apply.

The two matrices split the unknowns cleanly. Extrinsics change every time the camera moves and stay fixed when it does not. Intrinsics stay fixed for one camera and change only when you zoom, crop, or switch lenses.

The divide that makes depth

Multiplying $K[R \mid t]X$ gives three numbers $[a, b, c]$ where $c$ equals the depth $Z$ in front of the camera. The pixel is the division:

u = \frac{a}{c} = f\frac{X}{Z} + c_x, \qquad v = \frac{b}{c} = f\frac{Y}{Z} + c_y

In plain words: each coordinate is focal length times the ratio of lateral position to depth, shifted by the center. Depth $Z$ sits in the denominator, so doubling depth halves the offset from the center. That single division is why one photo cannot recover depth: many $(X, Z)$ pairs give the same ratio.

Worked example

Take a camera with focal length $f = 500$ , center $(c_x, c_y) = (320, 240)$ , no rotation ( $R$ is the identity), and no translation ( $t = [0, 0, 0]$ ). The world point is $X = 1$ , $Y = 2$ , $Z = 4$ .

Skip the matrices since pose is the identity and go straight to the division:

u = 500 \times \frac{1}{4} + 320 = 125 + 320 = 445, \qquad v = 500 \times \frac{2}{4} + 240 = 250 + 240 = 490

So the point lands at pixel $(445, 490)$ : 125 pixels right and 250 pixels above the center. Move the same point to $Z = 8$ and the offsets halve to $62.5$ and $125$ , giving $(382.5, 365)$ . Same statue, twice as far, half the size on screen.

Code

This runs the worked example with plain Python so the two steps stay visible: pose first, then intrinsics with the divide.

1def project(point: tuple[float, float, float], f: float, cx: float, cy: float) -> tuple[float, float]:2    x, y, z = point  # camera frame already: R is identity, t is zero3    u = f * x / z + cx4    v = f * y / z + cy5    return (u, v)6
7
8print(project((1.0, 2.0, 4.0), 500.0, 320.0, 240.0))  # -> (445.0, 490.0)9print(project((1.0, 2.0, 8.0), 500.0, 320.0, 240.0))  # -> (382.5, 365.0)

Both prints match the hand calculation. Doubling $Z$ from 4 to 8 halves each offset from the center $(320, 240)$ , which is the divide-by-depth law in one line of output.

Watch Out For

Forgetting the divide after the matrix multiply

$K[R \mid t]X$ returns $[a, b, c]$ , not pixels. Using $(a, b)$ directly as $(u, v)$ looks almost right for points near $Z = 1$ and drifts further off everywhere else, because you skipped dividing by $c$ . The symptom is a projection that scales with the scene instead of staying in the frame: points at $Z = 4$ land four times too far from the center.

Always divide by the third coordinate before comparing against pixels. If your reprojection error grows linearly with depth, the divide is the first line to check.

Reading the pose backwards

$[R \mid t]$ moves world points into the camera frame, not the camera into the world. Inverting the direction (or transposing $R$ without negating $t$ properly) places the scene behind the camera, where $Z$ is negative and the image flips. The symptom is pixels mirrored across the center or a solver that converges to a camera looking away from every point.

The round-trip test catches it: send a known world point through $[R \mid t]$ and confirm its camera-frame $Z$ is positive and its depth matches the tape measure. For reasoning about vectors and frames, see vectors, matrices, and tensors.

The Quick Version

A pinhole camera maps $X$ to pixels with $x = K[R \mid t]X$ followed by a divide by depth.
Extrinsics $[R \mid t]$ record camera position and direction; they change when the camera moves.
Intrinsics $K$ hold focal length and optical center; they change only with zoom, crop, or a new lens.
Pixel offsets from the center equal focal length times position over depth: $fX/Z$ and $fY/Z$ .
With $f = 500$ , point $(1, 2, 4)$ lands at $(445, 490)$ , which is 125 right and 250 up from center.
Doubling depth halves image size, so one view alone cannot recover depth.
Always divide by the third coordinate, and check the pose sends points in front of the camera.