Channel Attention Recalibrates Maps

Channel attention scores each feature map for usefulness on this image, then turns up the telling channels and down the noise.

Global pooling squeezes two channels to 0.82 and 0.23 gates, so the fur map counts nearly four times the grass map

Why Does This Exist?

Convolutions add all channels equally once trained. On a dog photo, fur channels matter while sky channels distract, but fixed sums cannot adapt per image.

Gates fix that. This page focuses the channel half of convolutional layers recalibration, lifts image classification on clutter, and pairs with location cues for object detection.

Think of It Like This

A mixing desk riding vocal faders

An engineer pushes the vocal fader to 0.82 and pulls guitar bleed to 0.23 for this song only.

It fails when the meter misreads. Riding the wrong fader buries the lead, just as wrong gates mute the target channel.

How It Actually Works

Squeeze channel $c$ by global average $z_c = (1/HW)\sum_{i,j} u_c(i,j)$ , where $u_c$ is that feature map. Excite with $s = \sigma(W_2 \delta(W_1 z))$ , where $\delta$ is ReLU, $\sigma$ is sigmoid, and $s_c$ is the gate. Rescale $\tilde{u}_c = s_c u_c$ .

Worked example

Two channels pool to $z = [0.9, 0.2]$ . Gates come out $s = [0.82, 0.23]$ . The fur map scales by 0.82 while grass scales by 0.23, a ratio of $0.82/0.23 \approx 3.57$ . Accuracy moves 0.71 to 0.80. Cost is two tiny dense layers per block.

Watch Out For

Squeezing away location before gating

Global pooling erases where, keeping only how much. The symptom is better classification with worse small boxes. Keep channel gates for classification strength and add spatial cues when location matters.

The Quick Version

Squeeze pools each map to one number $z_c$ , excite gates it to $s_c$ .
Fixture gates 0.82 and 0.23 give a 3.57 to 1 emphasis.
Gain 0.71 to 0.80 costs two small dense layers.
Gates adapt per image, unlike fixed channel sums.
Pooling loses location, so pair with spatial cues for detection.