Receptive Fields

Why Does This Exist?

A single unit from a 3x3 convolution sees exactly nine input pixels, the patch directly under its kernel, nothing more. On its own, a 3x3 kernel could never notice a shape spanning forty pixels or tell a face from one eyebrow, no matter how it is trained. Its whole world is a 3x3 window.

Yet networks built almost entirely from 3x3 kernels reliably recognize things far larger than nine pixels. This page resolves that gap: how does a network built from tiny local pieces end up with any sense of the whole picture.

The answer is depth. A second layer unit does not see raw pixels. It sees first layer outputs, and each of those already summarized its own 3x3 patch. Stack enough layers and one deep unit depends, indirectly, on a much larger patch of the original input. That indirect footprint is the receptive field. Downsampling in pooling layers grows it faster, which is why real nets interleave the two.

Think of It Like This

How It Actually Works

Tracing one unit backward

The diagram traces this by hand on a one dimensional strip of nine input units through three stacked 3x3 convolutions with stride 1 and no padding. Start at the single traced unit in layer three. It was computed from three units below, which is what a width 3 kernel does. Each of those three was computed from three units below, and because the three windows overlap, their union covers five units, not nine. Repeat once more and the union across the input reaches seven of the nine original positions.

Three layers, and the reachable footprint grew 3 to 5 to 7: a jump of two units per layer, matching kernel width minus one. That is the rule, not a coincidence.

The formula, and why it adds

For $L$ stacked layers with kernel size $k$ and stride 1, the receptive field is:

Plug in the diagram numbers, three layers of size 3, and $\text{RF} = 1 + 3 \times 2 = 7$, exactly the traced span. Growth is additive with depth for stride 1 layers: each extra layer adds a fixed $(k - 1)$, not a multiple. That rate is slow. It is why early nets needed a dozen or more layers before a unit field covered a typical input image.

Two things that grow the field faster

Pooling and strided convolutions grow the field multiplicatively, not additively. Downsampling shrinks the map, so every kernel afterward reads from a coarser grid where each cell already stands for a larger patch. A stride 2 pool roughly doubles the field of everything after it, which is why architectures interleave downsampling with convolutions rather than stacking stride 1 layers alone. See pooling layers for that half of the design.

Dilation grows the field without adding parameters or depth. A dilated 3x3 kernel with rate $r$ spreads three taps across $1 + 2r$ positions, so its effective size is $k_{\text{eff}} = k + (k-1)(r-1)$. Swap that into the same additive formula and one dilated layer covers the ground of several ordinary ones, at the cost of skipping positions in between.

Effective against theoretical

The formula gives the theoretical field: every input position a unit connects to. In practice, center positions reached through many overlapping paths matter far more than edge positions reached through one. The effective field is smaller than the theoretical one and, in trained nets, looks roughly Gaussian, strong in the middle and fading outward. The formula marks the outer boundary. It does not say how much each pixel inside counts.

Worked check: 7 against 12

Three stride 1 3x3 layers give $1 + 3 \times 2 = 7$. Add a stride 2 pool after the first layer and the code below gives 12. The pool adds only 1 itself but doubles the jump to 2, so each later 3x3 adds $(3-1) \times 2 = 4$ instead of 2. The last two layers contribute 8 of the total 12. That doubling is the multiplicative effect, computed step by step.

Code

The point is the check: rebuild the 7 from the diagram and the 12 with downsampling mixed in.

1def receptive_field(layers: list[tuple[int, int]]) -> int:2    """Each layer is (kernel_size, stride). Returns the theoretical RF at the top."""3    rf: int = 14    jump: int = 1  # how many input pixels one output step represents5    for kernel_size, stride in layers:6        rf += (kernel_size - 1) * jump7        jump *= stride8    return rf9
10
11three_stride1: list[tuple[int, int]] = [(3, 1), (3, 1), (3, 1)]12print(receptive_field(three_stride1))  # -> 713
14with_downsampling: list[tuple[int, int]] = [(3, 1), (2, 2), (3, 1), (3, 1)]15print(receptive_field(with_downsampling))  # -> 12

The printout matches the hand trace: 7 for three plain layers, 12 once a stride 2 stage doubles what every later layer is worth. Change the pool to stride 1 and the total falls back toward the additive path.

Watch Out For

The Quick Version

• A single 3x3 kernel sees nine pixels, and the receptive field is how much input one deep unit depends on after tracing back through every layer.
• For stacked stride 1 layers the field grows additively by the rule RF equals 1 plus L times k minus 1.
• Three stride 1 3x3 layers give 7, traced in the diagram as spans of 3 then 5 then 7.
• Pooling and strided convolutions grow the field multiplicatively because later layers read from an already shrunk map.
• Dilation grows the field without depth or parameters by spreading kernel taps apart.
• The effective field is smaller than the theoretical field and concentrates toward the center.

What to Read Next

• Convolution Operation
• Pooling Layers
• Vision Transformers