Layer Normalization
Standardizing every feature of a single example to mean zero and variance one, computed independently per example rather than across a batch of examples.
Think of It Like This
Like leveling the audio volumes of each track within a single song before mixing, without caring how loud a completely different album is playing.
Take one example's feature vector, subtract its own mean, divide by its own standard deviation, then apply a learned scale and shift. No other example in the batch is involved, which is the entire difference from batch normalization — that per-example independence is why layer normalization tolerates a batch size of one and variable-length sequences without changing behavior.
That's exactly the situation a transformer sits in: sequences arrive at different lengths and training frequently runs small batches per device, so a normalization scheme that never reads across the batch dimension became the default rather than an alternative.