Attention Rollout in Vision

A transformer spreads attention across twelve layers, so one layer's map misleads. Rollout multiplies the layers into a single map from input patches to the final decision.

Twelve mixed layers multiply into one rollout map with 0.68 mass on the dog

Why Does This Exist?

A vision transformer routes information through a dozen attention layers with residual connections around each. Reading only the last layer's attention claims the decision came from wherever the last layer looked, ignoring that the last layer looked at mixtures built by eleven earlier layers.

Rollout exists to track that flow. It models how much of each input patch reaches the final class token through every path, giving one map per image that operators can compare against Grad-CAM or saliency maps when the backbone is a transformer rather than a CNN.

Think of It Like This

Tracing river water through reservoirs

Twelve reservoirs in a chain each keep half their water and pass half downstream, mixing with local streams. Asking which spring fed the final dam by testing only the last reservoir's inlet misses eleven rounds of mixing.

Rollout is the ledger that multiplies the keep-and-pass fractions down the whole chain. The analogy stops at attention weights changing per input: reservoirs have fixed gates, while attention re-computes its mixing from the image each time.

How It Actually Works

Let layer $l$ carry attention $A_l$ over $N$ patches plus the class token. With residual connections, information flows through the mixed matrix $M_l = 0.5 A_l + 0.5 I$ , where $I$ is the identity (half rides the attention, half skips it). Rollout multiplies these mixed matrices from the first layer to the last:

R = (0.5 A_1 + 0.5 I)(0.5 A_2 + 0.5 I)\ldots(0.5 A_L + 0.5 I)

Here $A_l$ is the attention matrix at layer $l$ , $I$ passes tokens through unchanged, $L$ is the layer count, and row $R$ for the class token gives each input patch's total flow. Read the class-token row as the map.

Worked example

A demo ViT has 12 layers over 16 patches plus the class token. Raw last-layer attention puts 0.41 of the class-token mass on the 4 dog patches. Multiplying all twelve mixed layers, rollout puts 0.68 on those patches and 0.32 elsewhere: $0.68$ divided by $0.41$ is $1.66$ , so rollout concentrates $1.66$ times more mass on the dog. The extra comes from early layers that had already gathered the ears and muzzle before the last layer looked.

Watch Out For

Reading rollout as causal proof

Rollout tracks information flow through attention, not what the model needed. The symptom is deleting the highlighted patches, watching accuracy hold, and concluding the map lied. It did not lie about flow; the model simply had redundant paths. Confirm with ablations before claiming necessity.

The Quick Version

One attention layer misleads because residuals mix eleven earlier layers into its inputs.
Rollout multiplies the mixed matrix $M_l = 0.5 A_l + 0.5 I$ across all layers into one flow map.
On the demo image rollout puts 0.68 mass on the dog versus 0.41 raw.
It reports flow, not necessity, so pair it with ablations.
Use it for transformers; CNNs keep Grad-CAM.