MobileViT Folds Transformers Into Mobiles

MobileViT unfolds maps to tokens for light transformers then folds back, giving global range inside mobile conv blocks.

Unfold to 49 groups plus fold back lifts 73 to 78 percent at 1.2 GFLOPs

Why Does This Exist?

MobileNet lacks range and ViT lacks lightness. Phones need both in $6$ M weights.

MobileViT nests transformers in convs. It extends mobilenet inverted blocks, fits efficient architectures, and lifts image classification mobiles.

Think of It Like This

Folding laundry to check all pockets at once

You unfold stacked shirts into a row to check pockets globally, then refold to stacks. Local folds keep shape while the row check adds context.

Unfold is the row, transformers are checks. Where it stops: huge piles overwhelm rows, just as large maps strain mobile attention.

How It Actually Works

Conv $3 \times 3$ then $1 \times 1$ to $d$ . Unfold $H \times W$ into $P = hw$ patches per group with $N$ groups. Run $2$ -layer transformers per group, fold back, fuse with input via $1 \times 1$ plus $3 \times 3$ .

Worked example

$56 \times 56$ with $2 \times 2$ patches gives $28 \times 28 = 784$ groups? MobileViT-S uses $P = 4$ ( $2 \times 2$ ) and $N = 784$ . Attention per group $4^2 = 16$ pairs times $784 = 12{,}544$ versus full $3136^2 = 9.8$ M, about $784$ times cheaper. Accuracy $73$ to $78$ percent at $1.2$ GFLOPs.

Watch Out For

Unfolding too large maps on phones

$112$ maps explode groups and memory. Symptom: phone OOM despite low FLOPs. Fix: apply MobileViT blocks at $28$ and below as in the paper.

The Quick Version

MobileViT unfolds to tokens, transformers lightly, then folds back.
Per-group attention keeps global sense at mobile cost.
Fixture $12$ K pairs replace $9.8$ M, 784 times cheaper.
Gains $73$ to $78$ percent at $1.2$ GFLOPs.
Place blocks at 28 maps and below on phones.