Convolution and Image Filtering

Combine A Neighborhood

Linear image filtering computes each output from a weighted neighborhood of input pixels. The kernel supplies the weights. A three-by-three mean uses nine weights of 1/9, so the pictured patch gives (90 + eight zeros)/9 = 10. The mint outline marks the whole contributing patch, not just its center. This is the local weighted-sum mechanism behind OpenCV's smoothing filters.

The output goes at the kernel's anchor location. Compute every result from the original input, not from outputs already written earlier in the scan. A normalized mean preserves constant regions away from border effects; derivative kernels instead respond to changes.

Slide The Same Weights

Now enlarge the input to five-by-five. The dot is the sole nonzero pixel, still 90; every unmarked cell is zero. The mint window is the first valid placement. Moving it one column at a time and then one row at a time gives nine placements, each containing the center impulse once. All nine outputs are therefore 10.

For input width n, kernel width k, padding p and stride s, output width is floor((n + 2p − k)/s) + 1. Here (5 − 3)/1 + 1 = 3; the same calculation holds vertically. This drawing uses valid filtering, with no invented border pixels. Same-size library output normally requires a border policy.

Check The Kernel Orientation

Zoom to a one-row example to expose orientation. The upper weights pair directly with [0,0,10]: −1×0 + 0×0 + 1×10 = 10. This is cross-correlation. Mathematical convolution reverses the weights first; the lower row gives 1×0 + 0×0 − 1×10 = −10.

In two dimensions, convolution flips a kernel along both axes, equivalently rotating it 180 degrees. The filter2D API contract explicitly specifies correlation despite the function's common convolution terminology. Symmetric kernels, including our box mean, give identical results under either convention. Preserve signed responses in an appropriate signed or floating output type; unsigned storage can clip negative derivatives.

Decide What Lies Outside

This new one-dimensional fixture is a constant row of 9s. A three-sample mean centered on its first pixel needs an unavailable left neighbor. The dashed cell is that invented sample. Zero padding gives (0+9+9)/3 = 6; replication gives (9+9+9)/3 = 9. The solid vertical line is the image boundary.

Padding therefore changes numerical results, even where the original image is constant. Reflection is another policy; name the exact convention because reflection variants differ at the edge. Padding, stride and anchor are part of the operation, not incidental implementation details.

Change The Weight Pattern

These bars represent kernel weights, not pixel values. The uniform kernel [1,1,1]/3 assigns equal influence; the binomial kernel [1,2,1]/4 assigns half to the center and one quarter to each neighbor. Both sum to one. For the same row [0,9,0], the first outputs 3 and the second 4.5. This small binomial filter is Gaussian-like; a general Gaussian kernel has weights determined by its spread and sampling. Center weighting still mixes neighbors rather than preserving every center unchanged.

Factor A Separable Kernel

Place an impulse of 16 in an otherwise zero image. Filtering rows with [1,2,1]/4 spreads it into [4,8,4]. Filtering columns with the same weights then gives the pictured three-by-three output: rows [1,2,1], [2,4,2], [1,2,1]. Its sum is still 16. Other output locations are zero; the drawing crops to the nonzero support.

This is equivalent to a two-dimensional kernel equal to the outer product, [[1,2,1],[2,4,2],[1,2,1]]/16. The OpenCV separable filtering API applies separate row and column kernels. For a separable width-k kernel, two passes require roughly 2k multiply-accumulates per output instead of k², ignoring boundary and implementation costs. Not every kernel admits a single such factorization.

Sharpening Can Overshoot

Use the symmetric one-dimensional sharpening kernel [-1,3,-1]. An isolated 9 surrounded by zeros produces [-9,27,-9] around its position: the center becomes 3×9=27, while each adjacent output gets −1×9=−9. Both panels share a signed vertical scale; downward bars are negative. The weights sum to one, so a constant region is preserved, but local extremes can grow. Sharpening can amplify noise and introduce halos. Clamping to a display range changes the linear result.

Choose Which Anchors To Evaluate

The seven cells mark input locations; dots below mark the centers of width-three kernels that fit without padding. Stride one evaluates centers 1, 2, 3, 4 and 5, giving five outputs. Stride two evaluates centers 1, 3 and 5, giving three. The count formula gives floor((7−3)/2)+1=3. Dots encode evaluated positions, not response magnitudes. A larger stride changes spatial sampling; blur before decimation may be needed to control aliasing.

Preserve Signed Responses

The flipped derivative example earlier produced −10. A signed or floating output can retain it. Saturating conversion to an unsigned 8-bit image maps it to 0, losing the falling-edge response. Both panels mark zero at the same height. Choose a suitable output depth during filtering, then perform any magnitude calculation, normalization or display conversion deliberately. Taking an absolute value is also a transformation: it preserves strength while discarding polarity.

Where It Breaks

For [0,0,9,0,0], the three valid width-three means are [3,3,3]. Both panels use the same height scale. An isolated noise spike becomes less intense, but a real thin bright feature with those same samples does too. The filter cannot infer which interpretation is intended. Averaging trades fine spatial detail for smoothing; a larger kernel increases the region being mixed.

The Quick Version

• A linear filter combines neighboring values using fixed kernel weights.
• Valid filtering shrinks the output when the kernel exceeds one pixel.
• Convolution flips the kernel; correlation pairs weights directly.
• Border policies change pixel values near image boundaries.
• Kernel weights determine how much each neighbor contributes.
• Separable kernels can use successive horizontal and vertical passes.
• Sharpening can produce negative responses and overshoots.
• Stride controls which kernel anchors produce output locations.
• Signed output preserves derivatives that unsigned storage may erase.
• Smoothing can erase real detail along with noise.