Attention Is All You Need — the transformer, explained

The problem it solved

Recurrent models process tokens one at a time, which blocks parallelism and makes long-range dependencies hard to learn. The paper asks: what if attention alone could relate any two positions directly?

The key idea

Replace recurrence with self-attention. Every token computes a weighted sum over every other token, so information flows across the whole sequence in a single step. Multiple attention "heads" learn different relationships in parallel.

What made it work

• Scaled dot-product attention keeps gradients stable as dimensionality grows.
• Positional encodings reintroduce order that attention alone ignores.
• Residual connections + layer norm make deep stacks trainable.

Why it matters

Removing recurrence unlocked massive parallelism, which made training on huge corpora practical. Every modern LLM is a descendant of this architecture.