Convolution Mathematics

Convolution blends two functions by flipping a kernel and sliding it across a signal, so each output is a weighted average of its neighbours, and CNN layers compute its unflipped twin.

With input 1, 4, 1, 2, 0 and kernel 3, 1, 2 flipped to 2, 1, 3, the middle output is 4 times 2 plus 1 times 1 plus 2 times 3 equals 15, flanked by 9 and 4

Why Does This Exist?

Images, audio and sensor traces are not bags of independent numbers: neighbours belong together. A dense layer treats pixel 5 and pixel 6 as strangers; convolution treats each output as a weighted average of a small neighbourhood, so edges, slopes and textures emerge from the arithmetic rather than from memorised positions.

The same operation answers two needs. In signal processing it filters: smooth, sharpen, or extract an edge by choosing the weights. In deep learning it detects: each kernel becomes a learned pattern matcher sliding across the input, and stacking such layers builds a hierarchy from edges to objects. The mathematics is one formula; the CNN reading of it is in convolution operation.

The one idea you need first

Each output is a dot product: multiply overlapping pairs and add. If $a = [4, 1, 2]$ overlaps weights $[2, 1, 3]$ , the output is $4(2) + 1(1) + 2(3) = 8 + 1 + 6 = 15$ . Everything below is that product, repeated at every position with one twist: the flip.

Think of It Like This

A frosted window sliding along a picket fence

Imagine a picket fence with numbered slats, and a frosted window exactly three slats wide that you slide along it. At each stop you read the three visible numbers through the glass, but the glass weights them: the left pane counts triple, the middle single, the right double. You write down one blended number per stop, then slide one slat over.

Two details carry the mathematics. First, the glass is installed backwards: the triple pane sits over the rightmost visible slat, because convolution flips the kernel before blending. Second, the reported number at each stop depends only on the slats in view, never on distant ones, which is why convolution sees local structure and ignores position.

Where it stops: a real window blurs light, while convolution multiplies and adds exact numbers, and a real window never learns. In a CNN the pane weights are trained, and because they are learned the flip stops mattering, which is why libraries quietly implement cross-correlation instead.

How It Actually Works

The 1D formula

Discrete convolution of signal $f$ with kernel $g$ is:

(f * g)[n] = \sum_k f[k]\, g[n - k]

In plain words: at output position $n$ , overlap the kernel with the signal, multiply each overlapping pair, and add. The index $n - k$ runs backwards through $g$ as $k$ runs forwards through $f$ , which is the flip: the kernel's first weight meets the signal's last overlapping sample.

Cross-correlation drops the flip:

(f \star g)[n] = \sum_k f[k]\, g[n + k]

It slides the kernel as written. Deep learning layers compute this, not convolution, calling it convolution anyway. Since the weights are learned, some flipped twin of every kernel is equally learnable, so nothing is lost, but a hand-designed kernel (a Sobel edge finder, say) must be flipped before pasting into a framework, or its orientation reverses.

Worked example in 1D

Take signal $f = [1, 4, 1, 2, 0]$ and kernel $g = [3, 1, 2]$ . Flip $g$ to $[2, 1, 3]$ , then slide without padding (valid mode), giving $5 - 3 + 1 = 3$ outputs:

Position 0, overlap $[1, 4, 1]$ : $1(2) + 4(1) + 1(3) = 2 + 4 + 3 = 9$ .
Position 1, overlap $[4, 1, 2]$ : $4(2) + 1(1) + 2(3) = 8 + 1 + 6 = 15$ .
Position 2, overlap $[1, 2, 0]$ : $1(2) + 2(1) + 0(3) = 2 + 2 + 0 = 4$ .

Output $[9, 15, 4]$ , exactly the three cells the diagram shows with the middle one highlighted. Cross-correlation without the flip would give $[1(3) + 4(1) + 1(2), \ldots] = [9, 17, 5]$ : same first value by coincidence, different rest.

Worked example in 2D

Take a $3 \times 3$ input with rows $[1, 2, 3]$ , $[4, 5, 6]$ , $[7, 8, 9]$ and kernel rows $[1, 2]$ , $[3, 4]$ . Top-left placement overlaps input rows $[1, 2]$ , $[4, 5]$ .

Cross-correlation, kernel as written: $1(1) + 2(2) + 4(3) + 5(4) = 1 + 4 + 12 + 20 = 37$ .

True convolution flips the kernel both ways to rows $[4, 3]$ , $[2, 1]$ : $1(4) + 2(3) + 4(2) + 5(1) = 4 + 6 + 8 + 5 = 23$ .

Same window, $37$ versus $23$ : the flip is not bookkeeping, it changes the number. Sliding the $2 \times 2$ kernel over all four placements fills a $2 \times 2$ output, and padding or stride then decides the final size, which is the shape arithmetic the Fourier convolution theorem page builds on.

Code

1import numpy as np2
3
4def conv_valid_1d(f: np.ndarray, g: np.ndarray) -> np.ndarray:5    """True convolution: flip g, then slide (valid mode, no padding)."""6    return np.convolve(f, g, mode="valid")7
8
9def xcorr_valid_1d(f: np.ndarray, g: np.ndarray) -> np.ndarray:10    """Cross-correlation: slide g as written (what CNN layers compute)."""11    return np.correlate(f, g, mode="valid")12
13
14f = np.array([1.0, 4.0, 1.0, 2.0, 0.0])15g = np.array([3.0, 1.0, 2.0])16print(conv_valid_1d(f, g))    # -> [ 9. 15.  4.]17print(xcorr_valid_1d(f, g))   # -> [ 9. 17.  5.]18
19kernel = np.array([[1.0, 2.0], [3.0, 4.0]])20patch = np.array([[1.0, 2.0], [4.0, 5.0]])21print(float(np.sum(patch * kernel)))              # -> 37.0  cross-correlation22print(float(np.sum(patch * kernel[::-1, ::-1])))  # -> 23.0  true convolution

The 1D outputs $[9, 15, 4]$ and $[9, 17, 5]$ match the hand computation, and the 2D pair $37.0$ versus $23.0$ reproduces the flip difference.

Watch Out For

Pasting a hand-designed kernel into a CNN without flipping

Textbook kernels assume true convolution. Frameworks implement cross-correlation. Paste a Sobel vertical-edge kernel into a CNN layer and the detector fires on the mirror-image edge: gradients that should be positive come out negative, and the feature map is flipped relative to every diagram in the textbook.

The fix is one line: flip the kernel both ways before assigning it (kernel[::-1, ::-1]), or equivalently negate the expected orientation. Learned kernels need no fix, only transplanted designed ones. Whenever a hand-built filter behaves like its mirror image, the missing flip is the first suspect.

Off-by-one output sizes from padding and stride arithmetic

Valid convolution on length $n$ with kernel $k$ gives $n - k + 1$ outputs: 5 and 3 gave 3 above. Forgetting the $- k + 1$ when stacking layers shrinks feature maps faster than planned, and a network designed for $7 \times 7$ maps silently receives $5 \times 5$ , crashing the first dense layer or, worse, running with misaligned skip connections.

Write the size formula before the code: output length $\lfloor (n + 2p - k)/s \rfloor + 1$ for padding $p$ and stride $s$ . Same padding ( $p$ chosen so stride 1 preserves $n$ ) is the usual way to keep sizes predictable while the network deepens.

The Quick Version

Convolution flips the kernel, then slides it: $(f * g)[n] = \sum_k f[k]\, g[n-k]$ , one dot product per position.
Cross-correlation slides without flipping and is what CNN layers actually compute under the name convolution.
On $[1, 4, 1, 2, 0]$ with kernel $[3, 1, 2]$ , valid outputs are $[9, 15, 4]$ ; cross-correlation gives $[9, 17, 5]$ .
The 2D flip changes real answers: $37$ versus $23$ on the same window, so transplant designed kernels flipped.
Valid mode shrinks length $n$ with kernel $k$ to $n - k + 1$ ; account for it with the padding and stride formula.
Learned kernels absorb the flip, which is why frameworks skip it, but orientation-sensitive analysis must not.