Visual explainer

Feature Matching and RANSAC

Turn descriptor neighbors into candidate correspondences, reject ambiguity, fit geometric hypotheses, count inliers and refit without mistaking consensus for truth.

Propose Appearance Matches

Descriptor similarity proposes correspondences before geometric verification.

Feature matching searches for similar local appearance across images. RANSAC then tests whether enough proposed correspondences agree with a chosen geometric model. The stronger solid link identifies the preferred appearance candidate; it does not prove that both patches depict the same scene point. If the point is occluded, even its nearest neighbor may be wrong. SIFT and ORB descriptors supply these local signatures.

Choose The Right Distance

Hamming distance counts the bit positions where two binary descriptors disagree.

For A=1010 and B=1001, XOR gives 0011: two positions differ, so Hamming distance is 2. The two outlined columns identify those disagreements. Treating the bit strings as integers and subtracting gives a different quantity with no useful relationship to the number of changed tests. Standard two-point ORB uses Hamming; SIFT commonly uses L2 over its vector components. OpenCV requires Hamming2 for ORB with WTA_K=3 or 4, as specified in its matcher reference.

Reject A Near Tie

A nearest match is less convincing when the runner-up is almost equally close.

Request two nearest neighbors for each query descriptor. With the illustrative rule d1/d2 < 0.75, distances (20,50) pass because 20/50=0.4. Distances (20,22) fail because 20/22≈0.909. The nearest distance is identical; only its separation from the runner-up changes. Bars use four drawing units per distance unit. This ratio cutoff is a tuning choice, not a universal confidence probability. A zero-distance tie supplies no distinctive winner and should be rejected rather than divided blindly.

Check Both Search Directions

Mutual nearest-neighbor checking keeps a correspondence only when both searches agree.

Both A and B choose target X, but X chooses A when searched in reverse. Cross-checking retains A–X and discards B–X. The double-ended link means mutual preference; the dashed single arrow means a one-way preference. This is a different filter from the ratio test: it checks reciprocity rather than separation from a runner-up. Either can still accept a repeated-pattern mistake. They operate on descriptors before the geometric model is checked.

Sample A Geometric Hypothesis

Four nondegenerate point pairs can propose a homography hypothesis.

A homography maps image coordinates on one plane to coordinates in another view. It has eight independent parameters, so four exact point pairs in general position provide a minimal hypothesis. Sampled points must not collapse into a degenerate configuration such as a line. The marker shapes identify the four pairings.

Our source rectangle has corners (0,0),(40,0),(40,30),(0,30) pixels. Target coordinates add (20,10) to every corner. Both panels use three drawing units per pixel with separate drawing origins. This particular homography is a translation, but RANSAC is testing the full chosen model. More samples are needed for other models; four is not a generic RANSAC requirement. See OpenCV's homography workflow.

Measure Reprojection Error

Reprojection error measures the distance from the model prediction to the observed target.

Map source point (10,10) through the translation to predict (30,20). Suppose the candidate actually lands at (33,24). The residual is (3,4) pixels and its Euclidean length is sqrt(3²+4²)=5 pixels. The hollow marker is the prediction; the solid marker is the observation. This magnified target-image view uses 40 drawing units per pixel and image y downward. The dashed legs expose the components. This is one-way reprojection error, not descriptor distance or symmetric transfer error.

Count Model Support

A geometric error threshold selects the matches that support a hypothesis.

For four candidate errors [1,2,5,12] pixels and a three-pixel cutoff, only 1 and 2 support the hypothesis. Stem lengths share a ten-drawing-units-per-pixel scale; crosses mark rejected matches. The bottom numbers identify error values for separate candidates, not horizontal coordinates. RANSAC scores hypotheses against the whole candidate set, not only their sampled pairs. A wide cutoff can reward an inaccurate fit; a narrow one can reject genuine but noisy observations. Use units consistent with the coordinates sent to the estimator.

Repeat The Sampling

More independent RANSAC trials increase the chance of drawing an all-inlier sample.

Assume an inlier fraction w=0.5 and sample size s=4. One trial draws only inliers with probability w⁴=0.0625. Under independent trials, the chance of at least one such draw after N trials is 1−(1−wˢ)ᴺ. For a target of 0.99:

N\geq \frac{\log(1-0.99)}{\log(1-0.5^4)},\qquad N=72\text{ after rounding up}.

The plotted sample probabilities at 1, 16 and 72 trials are approximately 0.0625, 0.6439 and 0.9904; straight segments only join those computed samples. The x axis is linear in trial count. Real implementations estimate the inlier rate and may adapt stopping. Correlated sampling, degeneracy and a wrong model invalidate a naive confidence interpretation. An all-inlier draw is not itself a guarantee of a useful model.

Refit Accepted Evidence

Refitting uses all accepted correspondences to reduce dependence on one minimal sample.

The final estimate should use the accepted correspondences, rather than retaining only one sampled fit. To expose that idea with arithmetic, temporarily restrict the model to a single horizontal translation. Three accepted shifts [18,20,22] pixels have least-squares estimate (18+20+22)/3=20; residuals are −2,0,2, with squared sum 8. The plot uses 20 drawing units per shift pixel, with separate rows for observation identity. A full homography refit solves a projective estimation problem and may refine geometric error; it is not a component-wise average of shifts or matrices. OpenCV's calibration module documents homography estimation and refinement.

Where It Breaks

A dominant-plane consensus can reject genuine matches on another scene depth.

Suppose the background moves 20 pixels and a nearer foreground moves 40 because the camera translates. Fitting the background's translation leaves a 20-pixel foreground residual. The overlaid wall aligns, while the dashed predicted foreground and solid observed foreground remain 80 drawing units apart at four units per pixel. A tight threshold rejects that foreground match even if its appearance correspondence is physically correct.

RANSAC finds support for the model supplied to it. Consensus does not prove that every rejected match is false or that one plane describes the whole scene. Inspect spatial coverage and residual structure, and choose a model appropriate to the task. Homography and stitching follows this limitation into image alignment.

The Quick Version

Appearance similarity proposes candidates rather than proving scene identity.
Hamming distance counts differing binary test outcomes.
A ratio test rejects poorly separated nearest neighbors.
Mutual checking requires agreement from both search directions.
Homography hypotheses require four nondegenerate correspondence pairs.
Reprojection error measures geometric disagreement in target coordinates.
An error threshold defines each hypothesis's consensus set.
Repeated trials improve all-inlier sampling probability under stated assumptions.
Refitting uses accepted evidence beyond the original minimal sample.
A dominant model can reject valid matches on another depth.