Camera Calibration and Parameters

Camera calibration is the process of estimating a camera's internal characteristics and its physical position in the world, enabling us to mathematically map 3D world coordinates to 2D image pixels.

Extrinsic parameters transform world coordinates into camera coordinates, and intrinsic parameters project them onto the 2D image plane.

Why Does This Exist?

A camera captures a 3D world and crushes it down into a flat, 2D grid of pixels. In doing so, we lose depth information, and the image geometry gets warped by the specific lenses we use. If you want to use Computer Vision to measure the real physical size of an object, calculate the distance to a pedestrian for a self-driving car, or seamlessly render 3D augmented reality graphics onto a video feed, you cannot just look at raw pixels. You need a mathematical model that maps the 3D space to the 2D plane.

Camera calibration provides this mathematical mapping. By determining the exact characteristics of the camera (like focal length and lens distortion) and its position in space, we establish a strict geometric relationship between a 3D point in the real world and a 2D pixel in the image.

Think of It Like This

The Artist's Setup

Imagine you are a painter trying to paint a landscape on a canvas through a glass window.

The Extrinsic Parameters describe where you put your chair and which way you are facing relative to the landscape. If you move your chair left, right, or rotate it, the landscape will look completely different through the window, even though the window itself hasn't changed.

The Intrinsic Parameters describe the window itself. How far is your eye from the window (focal length)? Is the window perfectly flat, or is it curved like a fishbowl (lens distortion)? Where is the exact center of the window (principal point)?

To mathematically predict exactly where a specific mountain peak will appear on your window, you need to know both where your chair is (extrinsics) and the physical properties of the glass (intrinsics).

How It Actually Works

The projection of a 3D world point $(X, Y, Z)$ into a 2D image pixel $(u, v)$ is traditionally modeled using the Pinhole Camera Model, supplemented by lens distortion models. This transformation happens in two distinct steps:

1. Extrinsic Parameters (World $\rightarrow$ Camera)

Extrinsics define the camera's location and orientation in the 3D world. They convert coordinates from the global "World Coordinate System" into a local "Camera Coordinate System" where the camera lens is the origin $(0,0,0)$ and the camera is looking straight down the Z-axis.

Extrinsics consist of:

A Rotation Matrix ( $R$ ): A $3 \times 3$ matrix representing how the camera is tilted, panned, or rolled.
A Translation Vector ( $t$ ): A $3 \times 1$ vector representing the camera's physical $x, y, z$ offset from the world origin.

2. Intrinsic Parameters (Camera $\rightarrow$ Image)

Once the point is relative to the camera itself, we must project it onto the 2D sensor. Intrinsics are entirely internal to the specific camera and do not change when you move the camera around.

Intrinsics are represented by a $3 \times 3$ Camera Matrix ( $K$ ), containing:

Focal Length ( $f_x, f_y$ ): The distance between the camera sensor and the optical center. It dictates the field of view. Usually, $f_x$ and $f_y$ are very similar unless the sensor pixels are not perfectly square.
Principal Point ( $c_x, c_y$ ): The exact point where the optical axis intersects the image plane. Ideally, this is the exact center of the image (e.g., pixel $(320, 240)$ for a $640 \times 480$ image), but manufacturing imperfections often shift it slightly.

3. Lens Distortion

The pinhole model assumes light travels in perfectly straight lines onto a flat sensor. Real lenses are curved, causing straight lines in the real world to appear curved in the image. The calibration process estimates distortion coefficients ( $k_1, k_2, p_1, p_2$ ) to mathematically "undistort" the image before applying the pinhole model.

Radial Distortion: Causes straight lines to bow outward (barrel distortion, common in wide-angle/GoPro lenses) or inward (pincushion distortion).
Tangential Distortion: Occurs when the lens and image sensor are not perfectly strictly parallel.

Code

In practice, camera calibration is done by taking multiple pictures of a known geometric pattern (most commonly a black-and-white checkerboard) from different angles. OpenCV can detect the corners and solve for all parameters.

1import numpy as np2import cv23
4# Prepare object points, like (0,0,0), (1,0,0), (2,0,0) ...5# for a checkerboard where Z=06objp = np.zeros((6*7, 3), np.float32)7objp[:,:2] = np.mgrid[0:7, 0:6].T.reshape(-1, 2)8
9# Arrays to store object points and image points from all the images.10objpoints = [] # 3d points in real world space11imgpoints = [] # 2d points in image plane12
13# ... (assume you loop through multiple images of the checkerboard)14# ret, corners = cv2.findChessboardCorners(gray, (7,6), None)15# objpoints.append(objp)16# imgpoints.append(corners)17
18# Perform camera calibration19ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(20    objpoints, imgpoints, gray.shape[::-1], None, None21)22
23# mtx contains Intrinsics (focal length, principal point)24# dist contains Distortion coefficients25# rvecs, tvecs are Extrinsics (rotation, translation) for each image

Watch Out For

Assuming Constant Intrinsics with Autofocus

Intrinsic parameters are only static if the physical optics of the camera do not change. If your camera has autofocus enabled or a variable zoom lens, the focal length is constantly shifting, rendering a single static intrinsic matrix useless. For calibrated computer vision tasks, autofocus and auto-zoom must be disabled.

Insufficient Calibration Diversity

When calibrating with a checkerboard, the math relies on seeing the board in a wide variety of orientations and locations across the frame. If you only hold the checkerboard dead center and take 20 pictures without tilting it or moving it to the corners, the calibration algorithm will fail to accurately estimate the distortion coefficients or focal length.

The Quick Version

Camera Calibration: The process of mathematically defining how a 3D real-world point projects onto a 2D image pixel.
Extrinsic Parameters ( $R, t$ ): Where the camera is located in the world and how it is rotated. Changes when the camera moves.
Intrinsic Parameters ( $K$ ): The internal properties of the camera (focal length, optical center). Static for a fixed lens.
Distortion Coefficients: Mathematical correctors for the physical curving of light caused by real-world glass lenses.