Harris Corner Detector

Harris turns every pixel into a corner score from local gradients, so you can keep points that stay put when the camera shifts.

Harris builds a structure tensor from gradients and scores each pixel for corner strength.

Why Does This Exist?

Tracking and stitching need points you can find again. Flat patches all look alike, and edges slide along their own direction. Corners lock in two directions at once, so they survive small viewpoint changes.

Harris gave this a cheap test. It uses only first derivatives in a small window, no explicit eigenvalue solver, and one response value per pixel. Read the overview in corner and keypoint detection first, then come back for the exact score.

Think of It Like This

A tent corner in the wind

Picture a tent stretched on a lawn. Push the middle of a flat sheet and you cannot tell where you pushed. Push along a seam and the fabric slides. Pull the corner stake and the whole shape answers at once.

Harris looks for tent stakes in the image. Flat means no pull in any direction. Edge means pull in one direction. Corner means pull in every direction. The analogy stops at physics: pixels do not stretch, only gradient energy counts.

How It Actually Works

Start with horizontal and vertical derivatives $I_x$ and $I_y$ at each pixel, usually from Sobel filters. Around each pixel, form the structure tensor over a Gaussian window $w$ :

$M = \sum w \begin{bmatrix} I_x^2 & I_x I_y \\ I_x I_y & I_y^2 \end{bmatrix}$

$M$ summarizes gradient energy in the neighborhood. Let its eigenvalues be $\lambda_1$ and $\lambda_2$ . Both small means flat. One large and one small means edge. Both large means corner.

The response shortcut

Eigenvalues cost too much per pixel, so Harris uses determinant and trace:

$R = \det(M) - k \cdot \mathrm{trace}(M)^2$

Here $\det(M) = \lambda_1 \lambda_2$ and $\mathrm{trace}(M) = \lambda_1 + \lambda_2$ , with $k$ near $0.04$ . You threshold $R$ and keep local maxima.

Worked numbers

Take a window where $\sum I_x^2 = 5.0$ , $\sum I_y^2 = 4.2$ , and $\sum I_x I_y = 0.6$ . Then $\det = 5.0 \times 4.2 - 0.36 = 20.64$ and trace $= 9.2$ . With $k = 0.04$ , $R = 20.64 - 0.04 \times 84.64 = 20.64 - 3.39 = 17.25$ . That large positive $R$ marks a corner. An edge with sums $5.0$ , $0.2$ , and $0.3$ gives $\det = 0.91$ , trace $= 5.2$ , and $R = 0.91 - 1.08 = -0.17$ , so it is rejected.

Harris is rotation invariant because rotating the image rotates both eigenvectors together. It is not scale invariant: zoom in and the window sees a curve instead of a corner.

Watch Out For

Threshold picked on one image

A threshold that looks clean on a textured poster fires everywhere on gravel and nowhere on soft faces. Tune $k$ and the threshold on your own contrast range, then add non-maximum suppression. A raw threshold without local maxima gives clusters, not points.

Expecting scale invariance

Harris has no pyramid, so matching a close-up against a wide shot fails. If scale changes, move to scale invariant features or a pyramid detector instead of raising the threshold.

The Quick Version

Harris builds a $2 \times 2$ structure tensor from $I_x$ and $I_y$ in each window.
Response $R = \det(M) - k \cdot \mathrm{trace}(M)^2$ avoids an eigenvalue solver.
Large positive $R$ means corner, negative means edge, small means flat.
Rotation invariant but scale sensitive, so it suits tracking at fixed distance.
Threshold plus non-maximum suppression turns scores into usable points.