Vision Transformers
Splitting an image into fixed-size patches, embedding each as a token, and running ordinary self-attention over the resulting sequence instead of convolution.
A Vision Transformer cuts an image into a grid of fixed patches, flattens and projects each one into a token, adds a position embedding, then runs the same self-attention blocks a language transformer uses. No convolution, no built-in locality — every patch can attend to every other patch from the first layer.
That flexibility comes at a cost: without convolution's locality and translation-equivariance assumptions, a Vision Transformer needs substantially more training data to match a CNN's accuracy, though it tends to overtake CNNs once that data threshold is cleared.