Pooling Layers

Shrink a feature map by keeping only the strongest signal in each small patch, trading exact position for tolerance to roughly where a pattern sat.

A 2 by 2 window reports max 9 and average 3.0 from values 1, 1, 1 and 9, and moving the 9 anywhere inside the window leaves the max unchanged

Why Does This Exist?

A cat photographed slightly left of center fires a whisker detector at one pixel column; the same cat one inch right fires it at another. A stack of pure convolutions inherits every such shift downstream, so nothing treats the two photos as the same fact.

Two ideas make the rest click. A feature map is the grid of detector responses across positions. Stride is how far a window jumps between placements. The detector itself lives in convolution operation, and the full stack in convolutional neural networks.

This page covers the summarizer: one number per patch, shrinking the map and hiding sub-window shifts. How stacked shrinks widen each neuron's view lives in receptive fields. The step-by-step visual lives in pooling layers visualized.

Think of It Like This

A show of hands instead of a headcount

A packed room is asked whether anyone knows the answer, and one bit suffices: did any hand go up. Three hands near the front and three near the back give the identical room-level answer: yes.

Max pooling asks that of each patch: not where the loudest signal sat, only that it was there and how strong. Move the strongest hand-raiser one seat over within the same section and the section's answer does not change. Cross into the next section and a different section reports it.

Where it stops: a room remembers who raised hands if asked again, but pooling discards the three quieter values permanently. No later layer can recover which corner held the 9.

How It Actually Works

Max pooling, the default

Slide a $2 \times 2$ window with stride 2, meaning placements never overlap, and keep the largest of the four values. No weights, no learning: a fixed take-the-maximum. Each side halves, so a $4 \times 4$ map becomes $2 \times 2$ and one quarter of the values survive, chosen as the strongest rather than arbitrary.

Worked example: one window, both poolings

Take the window $[1, 1, 1, 9]$ . Max reports $9$ untouched. Average reports:

(1 + 1 + 1 + 9) / 4 = 12 / 4 = 3.0

The same strong signal survives whole under max and dilutes to a third under average. Now move the 9 to any of the four cells: the max stays 9 regardless. That is the whole tolerance mechanism, and its exact boundary: shifts inside one window vanish, shifts crossing into the next window move the report to a different output cell. Stacked pooling compounds this tolerance while compounding the position loss, which is also how receptive fields grow with depth.

The full 4 by 4 check

On the map with rows $[1,1,0,0]$ , $[1,9,0,0]$ , $[0,0,1,1]$ , $[0,0,1,1]$ , stride-2 max pooling gives $[[9, 0],[0, 1]]$ and average gives $[[3, 0],[0, 1]]$ . The code below recomputes exactly this.

Average pooling and its one big job

Averaging smooths instead of selecting, so a sharp edge activation drowns among quiet neighbors. Max therefore rules classification stacks. Average earns global average pooling: collapse a whole map to one number per channel just before the classifier, so any input size yields a fixed-length vector.

The learned alternative

Strided convolution downsamples with learned weights instead of a fixed rule, extracting and shrinking in one step. Pooling stays common, especially global average pooling at the head, but stride deserves recognition as the same job done differently.

Code

The point is the check: max and average pooling over the same 4 by 4 input with visible window loops.

1import numpy as np2
3
4def pool2x2(x: np.ndarray, mode: str = "max") -> np.ndarray:5    oh, ow = x.shape[0] // 2, x.shape[1] // 2  # 2x2 windows, stride 2 halves each dim6    out = np.zeros((oh, ow))7    for i in range(oh):8        for j in range(ow):9            window = x[i * 2 : i * 2 + 2, j * 2 : j * 2 + 2]10            out[i, j] = window.max() if mode == "max" else window.mean()11    return out12
13
14feature_map = np.array([[1, 1, 0, 0], [1, 9, 0, 0],15                         [0, 0, 1, 1], [0, 0, 1, 1]], dtype=float)16
17print(pool2x2(feature_map, "max"))18# -> [[9. 0.]19#     [0. 1.]]20print(pool2x2(feature_map, "average"))21# -> [[3.   0.  ]22#     [0.   1.  ]]

Max reports the 9 whole; average reports 3.0 for the same window, the dilution computed by hand above. Swap the 9 to the window's other corner and rerun: the max output is unchanged, the tolerance claim executable.

Watch Out For

Assuming shifts anywhere vanish

Translation tolerance gets quoted loosely, inviting the belief that any shift leaves the output unchanged. A one-cell shift crossing a window boundary moves the max to a neighboring output cell and drops the old one to its leftovers.

Symptom: detection boxes jitter or segmentation masks blur under single-pixel input shifts. Fix: for position-sensitive tasks pool late, pool less, or carry skip connections past pooling stages as in residual connections.

Pooling away a small signal forever

Stride-2 max pooling discards three quarters of the values. A moderately strong small feature losing a close contest to a louder neighbor vanishes with no path back.

Symptom: tiny objects detected early never reach the classifier despite strong early activations. Fix: pool less aggressively on small-object tasks or add skip links so detail bypasses the downsampling.

The Quick Version

Pooling summarizes each small window to one number, shrinking the map and discarding the rest.
On $[1, 1, 1, 9]$ max keeps 9 whole while average softens it to 3.0, a third of the signal.
Shift tolerance ends exactly at the window edge: inside vanishes, across moves the report.
Stride 2 with a 2 by 2 window turns 4 by 4 into 2 by 2 at one quarter the values.
Global average pooling collapses a whole map per channel, admitting variable input sizes.
Strided convolution downsamples the same way with learned weights instead of a fixed rule.