L2 Norm

The L2 norm is $\lVert v \rVert_2 = \sqrt{\sum_i v_i^2}$, and when a formula writes $\lVert v \rVert$ with no subscript this is what it means. For $v = [3, -4]$ it is 5.

It is privileged among norms because it is the only one that comes from a dot product, since $\lVert v \rVert_2 = \sqrt{v \cdot v}$. That connection is what gives it angles, projections, and a smooth derivative everywhere except the origin — none of which L1 has. It is also what gradient clipping measures, and what weight decay penalises.

As a penalty it shrinks weights toward zero without reaching it, because its unit ball is a smooth circle with no corners on the axes. One practical warning: never compute it as sqrt(sum(x**2)). Squaring overflows for large components and underflows for small ones, so a vector whose true norm is perfectly representable can return inf or 0. Library implementations factor out the largest component first to avoid exactly that.