Image Stitching

Find the landmarks two photos share, keep only the matches that agree on one camera motion, and warp every frame onto a single canvas.

Stitching keeps the 140 inlier matches of 200 (70 percent) that agree on one homography and routes one seam through their overlap

Why Does This Exist?

No affordable sensor captures a mountain range or a whole factory floor in one frame, yet inspectors and hikers want the whole scene at full resolution. Stitching manufactures wide views from overlapping narrow ones, keeping every pixel sharp instead of shrinking the world to fit. Phone panoramas, satellite mosaics, and microscope slide maps all run this pipeline.

Two prerequisites carry the load: repeatable landmarks from keypoint detection, and the warp math from transformations and homography. This page covers detection, matching, robust estimation, and compositing. Viewable wide projection is panorama creation; invisible seams are image blending.

Think of It Like This

Tiling with overlapping transparencies

Print a map across overlapping transparencies and reassemble it by sliding sheets until roads continue unbroken across every overlap, then trim the doubled regions away. Shared landmarks are the road crossings, geometric agreement is the unbroken road, and trimming is compositing.

It stops holding when sheets bend differently. Transparencies that stretched in the heat never align everywhere at once, the same way parallax and lens distortion defeat a single global warp. Flat-planed, well-calibrated captures tile; violated assumptions wrinkle.

How It Actually Works

Detect, describe, match

Each frame yields keypoints with descriptors (SIFT or ORB classically) that survive rotation and exposure shifts. Nearest-neighbour matching in descriptor space proposes pairs, and Lowe's ratio test keeps only pairs whose best match clearly beats the runner-up, typically at a $0.75$ ratio. A run might propose $200$ raw matches with $140$ surviving as inliers later, a $70\%$ keep rate that signals a solid overlap; under $20$ surviving inliers means rejection, not a panorama.

One motion from noisy votes

True matches all obey one geometric motion (a homography for planar scenes or pure rotation), while mismatches scatter. RANSAC samples minimal point sets, fits candidate motions, and keeps the motion with the most supporters, discarding the rest as outliers. Four points determine a homography, so even heavy mismatch rates resolve when the true motion commands a plurality.

Warp and composite

The winning homography warps each frame onto a shared canvas, usually anchored to a central reference frame to spread distortion. Overlapping pixels then need a seam decision (which frame owns each pixel) before blending, and the canvas crops to the valid region. Bundle adjustment refines all motions jointly for sequences longer than pairs, stopping drift from accumulating frame to frame.

Code

The two-call OpenCV stitcher for a tested image set:

1import cv22
3stitcher = cv2.Stitcher_create(cv2.Stitcher_PANORAMA)4status, pano = stitcher.stitch([img1, img2, img3])5# status OK (0) means enough inliers chained all frames; 1-3 name the failure

Status codes distinguish success from too-few-matches, estimation failure, and camera-parameter failure, so check the code before trusting the canvas.

Watch Out For

Parallax from translating through close scenes

Symptom: near objects double or shear while distant mountains align perfectly, and no parameter fixes both at once. One homography models one plane or pure rotation; close foreground violates both. Shoot rotation-only captures about the lens nodal point, keep scenes distant, or mask moving foreground before matching.

Visible exposure steps between frames

Symptom: alignment is perfect but every frame boundary shows as a brightness stripe from auto-exposure or vignetting. Geometry succeeded and photometry failed. Enable gain compensation and vignette correction before compositing, and let multi-band blending (the blending page) dissolve whatever steps remain.

The Quick Version

Stitching chains detect, match, robustly estimate, warp, and composite across overlapping frames.
Ratio-tested descriptor matches plus RANSAC turn noisy pairs into one trustworthy camera motion.
Four points determine a homography; under about $20$ surviving inliers, reject rather than force a result.
Anchor warps to a central frame and refine long chains with bundle adjustment against drift.
Parallax and exposure steps are the two classic failures, needing capture discipline and photometric correction.