Padding and Stride

Padding adds a border so the filter can center on edge pixels. Stride sets how far it jumps. Together they decide the output size.

A 3 by 3 kernel at stride 2 visits 4 positions on a 5 by 5 input, producing a 2 by 2 output.

Why Does This Exist?

A $3 \times 3$ filter cannot center on a border pixel without hanging off the image. Without help, every convolution shrinks the map and starves edges, and after several layers the border signal is gone. Meanwhile every layer needs a cheap way to shrink maps on purpose, or compute explodes.

Padding and stride are those two controls. Padding $P$ adds a zeros border so edges get full windows. Stride $S$ sets the step between windows, trading resolution for compute. They sit beneath convolutional layers and beside pooling, and the full arithmetic reference is CNN operations.

You need one idea first. Output size is counting placements, not learning. The formula tells you where a kernel can legally sit.

Think of It Like This

Tiles, grout and footsteps

Laying $3 \times 3$ tiles on a $5$ wide floor with steps of $2$ covers positions starting at 0 and 2 only, so two tiles span it. Grout around the edge lets the first tile center on the wall line. Wider footsteps mean fewer tiles and a coarser floor.

Padding is the grout, stride is the footstep, and the output count is how many tiles fit. Where the analogy stops: grout is neutral zeros, and the network learns no weights for it.

How It Actually Works

The placement formula

For input size $H_{\text{in}}$ , kernel $K$ , padding $P$ and stride $S$ :

H_{\text{out}} = \left\lfloor \frac{H_{\text{in}} - K + 2P}{S} \right\rfloor + 1

Same formula holds for width. $P = (K - 1) / 2$ with $S = 1$ preserves size, called same padding. $P = 0$ is valid padding and shrinks. Larger $S$ downsamples.

A worked 5 by 5

Input $5 \times 5$ , kernel $3 \times 3$ , $P = 0$ , $S = 2$ :

H_{\text{out}} = \left\lfloor \frac{5 - 3 + 0}{2} \right\rfloor + 1 = \left\lfloor 1 \right\rfloor + 1 = 2

The kernel visits top left covering rows $0$ to $2$ , then jumps $2$ to cover rows $2$ to $4$ . Same across columns, so $2 \times 2 = 4$ placements. With $P = 1$ , $S = 1$ the same input gives $(5 - 3 + 2) + 1 = 5$ , preserving $5 \times 5$ .

Choosing values

Use odd kernels with same padding to preserve size through feature blocks. Use stride $2$ convolution instead of pooling when you want learned downsampling. Keep padding symmetric, or imported weights misalign by a pixel.

Code

1def conv_out(size: int, kernel: int, pad: int, stride: int) -> int:2    """Output size from the placement formula. All args are ints."""3    return (size - kernel + 2 * pad) // stride + 14
5
6print(conv_out(5, 3, 0, 2))  # -> 27print(conv_out(32, 3, 1, 1))  # -> 328print(conv_out(32, 3, 1, 2))  # -> 16

Watch Out For

Even kernels breaking symmetry

Even sizes have no center pixel, so same padding needs uneven left and right borders. Frameworks split them differently, and a PyTorch to ONNX move shifts receptive fields by one. Prefer odd kernels and verify round trips on a $5 \times 5$ probe.

Valid padding eating small objects

Repeated valid convolutions shave two pixels per $3 \times 3$ layer, so border digits vanish after a few blocks. The symptom is edge accuracy far below center accuracy. Use same padding through the backbone and downsample explicitly with stride.

The Quick Version

Padding borders the input so edge pixels get full kernel windows.
Stride sets the jump between windows and downsamples the output.
A 5 by 5 input with a 3 by 3 kernel at stride 2 gives a 2 by 2 output.
Prefer odd kernels with same padding, and downsample with stride on purpose.