Numerical Overflow
When a computed value exceeds the largest number its float type can hold and becomes infinity.
Every float type has a ceiling: about for float32, and only 65,504 for float16. Exceed it and the result is inf rather than an error. The mirror case is underflow, where a value too small to represent becomes exactly zero.
Neither is fatal on its own. What makes them dangerous is what comes next: inf - inf and inf / inf are nan, log(0) is -inf, and a single nan in a gradient converts every parameter it reaches to nan on the backward pass. A model that was training normally produces nan loss from one step to the next, several operations downstream of the actual cause.
The three usual sources are worth memorising. Exponentiating a large logit, which log-sum-exp exists to prevent. Squaring large values before a square root, which is why sqrt(sum(x**2)) is not how you compute a norm. And a long chain-rule product in float16, which is why mixed-precision training keeps a float32 master copy of the weights and scales the loss before the backward pass.