Gradient Clipping
Capping the size of a gradient before the optimizer step, so one unusually large batch can't throw the parameters somewhere training can't recover from.
Think of It Like This
Like installing a governor on a race car engine that caps peak acceleration, preventing one wild slip of the pedal from sending the car into a wall.
Two versions exist. Clip by value caps every individual gradient entry at a fixed threshold, which is cheap but changes the direction of the update. Clip by global norm rescales the whole gradient vector when its length exceeds a threshold, preserving direction and only shrinking magnitude — the version almost every modern training recipe means by default.
It's a safety net, not a cure: a model that needs clipping on every single step still has an underlying instability, usually the learning rate or the initialization, that clipping is only masking rather than fixing.