Image Transformations and Homography

Why Does This Exist?

In computer vision, the cameras that capture images are rarely perfectly aligned with the objects they are viewing. If a camera takes a picture of a document from an angle, the document appears skewed or distorted, turning a rectangle into a trapezoid. If we want our algorithms to read that document, we first need a reliable, mathematical way to "flatten" it out. The same applies to simpler edits like rotating a photograph, resizing a thumbnail, or shifting an object's position. All these spatial modifications rely on geometric image transformations.

Image transformations exist to project pixels from one coordinate system to another. This is the cornerstone of tasks ranging from basic photo editing to complex augmented reality and panoramic image stitching. Without mathematical transformations to align and distort pixels systematically, our models would lack spatial reasoning about camera angles and perspectives, treating a tilted square as an entirely different shape than a straight one. By formalizing these transformations mathematically, computer vision pipelines can standardize their inputs, correct lens distortions, and align multiple images taken from slightly different viewpoints.

Whenever your smartphone stitches together a panorama, it relies heavily on image transformations. The phone calculates how the camera moved between each snapshot and applies a specific mathematical mapping to warp the images so they overlap perfectly. Similarly, in medical imaging, scans taken at different times or from different machines must be spatially registered, meaning one image is transformed to perfectly overlay the other for accurate comparison. Thus, the existence of transformations provides the essential spatial glue that ties disparate images back to real-world physical geometry.

Think of It Like This

How It Actually Works

At its core, a geometric transformation maps each pixel coordinate $(x, y)$ in the original source image to a new coordinate $(x', y')$ in the destination image. Because iterating through millions of individual pixels in Python loops is painfully slow, computer vision libraries rely on linear algebra to perform these mappings via highly optimized matrix multiplication.

1. Affine Transformations

Affine transformations include translation (shifting), scaling (resizing), rotation, and shearing. The crucial property of an affine transformation is that it preserves collinearity (straight lines remain straight) and parallelism (parallel lines remain parallel). A rectangle might turn into a parallelogram, but it will never turn into a trapezoid.

To represent all affine transformations uniformly, we use a $2 \times 3$ transformation matrix $M$. We represent pixel coordinates in homogeneous form by appending a $1$ to the $(x, y)$ coordinate vector, resulting in $(x, y, 1)$. This neat mathematical trick allows us to encode translation (which is an addition operation) as a multiplication operation alongside scaling and rotation.

In this matrix, $m_{11}, m_{12}, m_{21},$ and $m_{22}$ control the rotation, scaling, and shearing effects, while $t_x$ and $t_y$ dictate the translation along the x and y axes. Because the last row is implicitly understood as $[0, 0, 1]$, an affine matrix has 6 degrees of freedom. You need exactly three pairs of corresponding points between a source and a target image to calculate an unknown affine transformation matrix.

2. Perspective Transformations (Homography)

While affine transformations keep parallel lines parallel, perspective transformations do not. A homography represents the projection of a planar surface from one camera perspective to another. It maps planes to planes, and requires a $3 \times 3$ transformation matrix $H$.

To get the actual 2D coordinates in the new image plane, we must divide the resulting $x'$ and $y'$ values by the scaling factor $w'$:

This critical division by $w'$ is precisely what creates the perspective effect, making objects shrink as they recede into the distance. A homography matrix contains 8 degrees of freedom. Since the matrix can be multiplied by any non-zero scalar value without changing the final 2D projection (due to the division by $w'$), $h_{33}$ is typically normalized and set to 1. To calculate a homography matrix between two planes, you need at least four corresponding point pairs.

In practice, to stitch two photos together, an algorithm like SIFT or ORB first finds dozens of matching feature points between the two images. It then uses an algorithm called RANSAC to aggressively discard incorrect matches (outliers) and calculate the optimal homography matrix from the remaining correct points, allowing one image to be perfectly warped into the perspective of the other.

3. Interpolation

When a coordinate $(x, y)$ is transformed to $(x', y')$, the resulting values are almost never perfect integers. For instance, a pixel might land at $(14.3, 8.9)$. Since digital images only exist on a discrete, integer-based grid, we must determine what color or intensity should be assigned to the new coordinate. This mathematical estimation process is called interpolation.

Common interpolation methods vary in speed and quality:

• Nearest-Neighbor: Snaps to the closest integer pixel. Extremely fast, but leads to heavily jagged, blocky edges.
• Bilinear: Takes a weighted mathematical average of the 4 closest pixels surrounding the point. It is the industry standard default due to its excellent balance of speed and smoothness.
• Bicubic: Takes a weighted average of the 16 closest surrounding pixels. It produces the smoothest edges and sharpest details, but is computationally heavier.

Code

Here is a short OpenCV snippet to apply an affine rotation and a perspective homography. Notice how both rely on calculating a matrix first, then calling a dedicated warp function.

1import cv22import numpy as np3
4# Load image5img = cv2.imread("document.jpg")6rows, cols = img.shape[:2]7
8# 1. Affine Transformation: Rotate by 45 degrees around the center9center = (cols / 2, rows / 2)10# Get a 2x3 affine rotation matrix11rotation_matrix = cv2.getRotationMatrix2D(center, angle=45, scale=1.0)12# Warp the image using the affine matrix13rotated_img = cv2.warpAffine(img, rotation_matrix, (cols, rows))14
15# 2. Homography: Perspective warp using 4 corresponding points16pts_source = np.float32([[56, 65], [368, 52], [28, 387], [389, 390]])17pts_dest = np.float32([[0, 0], [300, 0], [0, 300], [300, 300]])18
19# Calculate the 3x3 homography matrix (requires 4 points)20homography_matrix = cv2.getPerspectiveTransform(pts_source, pts_dest)21# Warp the image using the homography matrix22warped_img = cv2.warpPerspective(img, homography_matrix, (300, 300))

Watch Out For

The Quick Version

• Image transformations map pixel coordinates from one space to another using linear algebra matrices, crucial for alignment and distortion correction.
• Affine transformations (using a $2 \times 3$ matrix) handle shifting, scaling, rotating, and shearing. They keep parallel lines parallel and require 3 points to solve.
• Homographies or perspective transformations (using a $3 \times 3$ matrix) handle viewpoint changes where parallel lines may converge, mimicking 3D depth on a 2D plane. They require 4 corresponding points to calculate.
• Calculating transformations requires homogeneous coordinates, a math trick adding an extra $1$ to coordinate vectors so translations can be multiplied like rotations.
• Non-integer target coordinates require interpolation (like bilinear or bicubic) to smoothly guess pixel values based on neighboring grid colors.