Hierarchical Vision Transformers
Restricting self-attention to local windows and merging patches across stages, rebuilding a CNN-like multi-resolution pyramid at linear rather than quadratic cost.
A plain Vision Transformer pays quadratic attention cost and produces one flat resolution. Hierarchical designs, like the Swin Transformer, fix both: attention runs only inside small local windows of patches, making cost linear in patch count, and windows shift between layers so information still crosses window boundaries.
Patches periodically merge into fewer, coarser tokens across stages, rebuilding the multi-scale feature pyramid detection and segmentation tasks rely on — the same shape a CNN's pooling produces, built here from attention instead of convolution.