Vision Mamba Scans Images Linearly

Vision Mamba runs selective scans bidirectionally over patches with position codes, matching transformers at linear cost.

Forward plus backward selective scans cover 196 patches linearly

Why Does This Exist?

Transformers slow quadratically on long patch lists. High-res needs linear scaling without losing order.

Vim selects per input. It builds on state spaces, rivals image classification transformers, and pairs with VMamba 2D scans.

Think of It Like This

Reading a line both ways with notes

You read a sentence forward then backward, noting key words each pass. Two linear reads catch both sides.

Forward plus backward scans are reads, selection is notes. Where it stops: notes still compress, just as states forget rare detail.

How It Actually Works

Patchify to $196$ plus class token plus positions. Run selective scan forward and backward with $B,C,\Delta$ from $x$ . Gate and sum both directions, add MLP. Cost $O(N)$ .

Worked example

$N = 196$ , state $64$ . Forward $196 \times 64 \times 2 = 25{,}088$ mults, double to $50{,}176$ bidirectional. Attention $196^2 \times 64 \approx 2.46$ M, about $49$ times more. Selection raises $\Delta$ $0.1$ to $0.8$ on edges versus $0.1$ on flat, keeping detail.

Watch Out For

Dropping positions because scans have order

One-directional order is not 2D place. Symptom: rotation sensitivity spikes. Fix: keep position embeddings as Vim does.

The Quick Version

Vision Mamba scans patches bidirectionally with selection.
Input steps focus on edges over flat areas.
Fixture $50$ K mults replace $2.46$ M attention, 49 times cheaper.
Keep positions for 2D place.
Scales linearly past 1K tokens.