Speeded-Up Robust Features

SURF keeps the SIFT idea of scale proof points but swaps costly filters for box approximations, so detection runs much faster.

SURF approximates the Hessian with box filters and describes patches with wavelet sums.

Why Does This Exist?

SIFT works but drags on large images because Gaussian derivatives scale with kernel size. SURF asked how to keep scale invariance while making filter cost independent of size. The answer is integral images plus box filters.

Any rectangle sum then costs four memory reads, so large kernels cost the same as small ones. That speedup made near real time retrieval possible on older CPUs. Read SIFT first, since SURF mirrors its detect, orient, describe chain.

Think of It Like This

Cookie cutters instead of sculpting

SIFT sculpts each scale with smooth Gaussian tools. SURF stamps it with boxy cookie cutters. The stamp is cruder per cut, but each press costs the same whatever its size.

Integral images are the trick that keeps every press cheap. The analogy stops at quality: boxes approximate second derivatives well enough for speed, but they lose some SIFT accuracy under strong rotation and blur.

How It Actually Works

Detection uses the Hessian determinant with box approximations $D_{xx}$ , $D_{yy}$ , $D_{xy}$ :

$\det(H) \approx D_{xx} D_{yy} - (0.9 D_{xy})^2$

Scale space grows the box size instead of shrinking the image. Maxima over space and scale become keypoints. Orientation comes from Haar wavelet responses in $x$ and $y$ inside a circular window, summed over sliding sectors to find the dominant direction.

Worked numbers

A $9 \times 9$ box filter at scale $\sigma = 1.2$ needs four integral reads per rectangle, same as a $27 \times 27$ box at higher scale. Suppose wavelet sums in the best $60$ degree sector total $(dx, dy) = (42, 18)$ . Orientation is $\arctan(18/42) \approx 23$ degrees. The standard descriptor splits a $20\sigma$ window into $4 \times 4$ cells with $4$ sums each ( $\sum dx$ , $\sum dy$ , $\sum |dx|$ , $\sum |dy|$ ), giving $16 \times 4 = 64$ floats. U-SURF skips orientation for upright scenes like aerial photos and runs faster.

Watch Out For

Patent and license history assumed current

SURF was long patent encumbered, which pushed many teams to ORB. Patent status changed by country and year, so check current counsel before shipping, or pick a free binary feature.

U-SURF on rotated photos

U-SURF assumes upright images. Feed it rotated phone photos and matches collapse because orientation was skipped. Use full SURF or SIFT when roll varies.

The Quick Version

Integral images make box filter cost independent of kernel size.
Hessian determinant with $0.9$ weighting finds scale tuned points.
Haar wavelet sectors assign orientation.
Sixty four floats describe each patch, half the SIFT size.
Faster than SIFT, weaker under large rotation and blur.