SwinIR

SwinIR restores images with shifted-window transformers that compare distant patches cheaply, beating convolutional restorers on sharp repeating detail.

Shifted attention windows let distant patches compare notes while keeping computation local

Why Does This Exist?

Convolutional restorers see locally. A brick pattern repeating across a wall is relearned per neighbourhood because kernels never compare far-apart patches, wasting the image's best evidence about itself. Global attention would compare everything but costs quadratically, impossible on megapixel images.

SwinIR windows the attention: compare densely inside $8 \times 8$ patches, then shift the grid so information crosses boundaries in alternating blocks. Long-range repetition becomes usable evidence at near-convolutional cost. The convolutional fidelity baseline is SRCNN, and the GAN-texture alternative is ESRGAN.

Think of It Like This

A quilting bee with rotating tables

Quilters sit at square tables, comparing stitches closely within each table. Every hour everyone shifts one seat over, so new neighbours compare notes and patterns propagate across the hall. Nobody ever addresses the whole hall at once, yet within an afternoon every quilter has learned from every other.

Windows are the tables and shifting is the rotation. Fixed seating would trap each quilter with four neighbours forever; rotation spreads the craft. The bee fails when the hall needs one global decision nobody can see whole, rare, and handled by the convolutional tail.

How It Actually Works

Shallow convolutions extract early features, then residual Swin transformer blocks process them: each block alternates window attention (W-MSA) on a fixed $8 \times 8$ grid with shifted-window attention (SW-MSA) offset by $4$ pixels. A final convolution plus pixel-shuffle upsampler reconstructs. Residuals wrap every block so attention refines rather than replaces:

\text{windows} = (H / 8) \times (W / 8), \qquad \text{shift} = 4 \text{ px alternating}

On a $64 \times 64$ feature map, attention splits into $8 \times 8 = 64$ windows of $64$ pixels each: $64 \times 64^2 \approx 262{,}000$ pairwise comparisons per head versus $4{,}096^2 \approx 16.8$ million globally, a $64$ -fold saving that grows with image size.

Worked example

Brick wall, $64 \times 64$ crop. Window attention matches mortar lines within each $8$ -pixel cell; after the $4$ -pixel shift, cells straddle old boundaries and propagate the confirmed pattern across them. Two blocks give every pixel indirect access to the whole crop. The model restores mortar contrast using evidence from thirty cells away, where a $3 \times 3$ convolution would need fifteen stacked layers for the same reach.

Code

Window arithmetic that prices attention against the global alternative.

1def window_cost(px: int = 64, window: int = 8) -> tuple[int, int]:2    """Local windowed versus global attention comparisons per head."""3    n = px * px4    wins = (px // window) ** 25    local = wins * (window * window) ** 26    return local, n * n7
8
9print(window_cost())  # -> (262144, 16777216) a 64 fold saving

Watch Out For

Window grid artefacts on flat skies

Fixed windows can imprint faint grid lines on smooth gradients where neighbouring cells converge slightly differently. The artefact hides in textured regions and surfaces in skies and walls.

Check flat regions at full zoom during validation. Overlapping post-processing or slightly larger windows suppress the grid where it matters.

Expecting GAN textures from an L1 model

Classic SwinIR trains on pixel loss, so it restores faithfully but not flamboyantly: sharp edges, honest textures, no invented detail. Users expecting ESRGAN-style pop call it blurry by comparison.

Match model to goal: SwinIR for truth, GAN variants for beauty. The architecture supports both losses; the checkpoint decides.

The Quick Version

SwinIR alternates fixed and shifted $8 \times 8$ attention windows inside residual blocks.
Shifting propagates pattern evidence across boundaries without global attention.
A $64$ grid splits into $64$ windows: $262{,}144$ versus $16{,}777{,}216$ comparisons.
Repeating textures restore from far-apart evidence convolutions cannot reach cheaply.
Watch flat regions for grid lines, and pair the model with the right loss for the goal.