Swin Transformer
The 2021 paper that brought hierarchical structure and local windows to Vision Transformers, making them practical for high-resolution tasks like object detection and segmentation.
Paper: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Authors: Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo · 2021
Read the paper
The Problem
The original Vision Transformer (ViT) had two major flaws for dense prediction tasks (like finding bounding boxes or pixel-level segmentation). First, it produced a single, low-resolution feature map, missing the hierarchical scales (fine to coarse) that CNNs naturally build. Second, its self-attention was global. Processing high-resolution images meant thousands of patches, causing the memory and compute requirements to explode quadratically.
The Idea
Microsoft Research introduced the Swin (Shifted Window) Transformer. It reintroduced the best ideas from CNNs (hierarchical feature maps and local processing) back into the Transformer. Instead of global attention, Swin computes self-attention only within local windows of patches (linear complexity). To allow information to flow across the image, it "shifts" the window boundaries in alternating layers.
How It Works
Swin modifies the ViT architecture in two key ways:
- Hierarchical Feature Maps: The network starts with small patches. After processing them, a "patch merging" layer combines adjacent patches into a single token, reducing the resolution but doubling the channel depth. This happens multiple times, creating a feature pyramid.
- Shifted Window Attention: In one layer, the image is partitioned into a regular grid of non-overlapping windows (e.g., 7x7 patches per window). Self-attention is computed only inside each window. In the next layer, the grid is shifted down and to the right by half a window size. The new windows now overlap the boundaries of the previous layer's windows, allowing features to communicate globally across the image over multiple layers.
Why It Mattered
Swin Transformer proved that Transformers could serve as general-purpose backbones for all computer vision tasks, not just image classification. It dominated benchmarks in object detection (COCO) and semantic segmentation (ADE20K), permanently replacing ResNet variants as the default backbone for high-performance dense vision systems.
What Came After
Swin prompted a massive wave of research attempting to optimize vision architectures. It directly inspired ConvNeXt (which argued CNNs could match Swin if modernized) and remains a staple architecture in visual-language models and medical imaging systems.