L1 Norm

The L1 norm is $\lVert v \rVert_1 = \sum_i |v_i|$. For $v = [3, -4]$ it is 7, against an L2 norm of 5.

Its defining behaviour as a penalty is that it produces exact zeros, not just small values. The reason is geometric: the set of vectors with L1 norm 1 is a diamond whose corners sit precisely on the axes, and a corner is where one coordinate is exactly zero. A constrained optimum is far more likely to land on a spike than on a flat face, so L1 regularisation performs feature selection as a side effect. That is what lasso regression is.

The cost is that those same corners are points where the derivative does not exist, so L1 penalties need optimisers tolerant of a subgradient and are less smooth to train than L2. It is also more robust to outliers than L2, because it grows linearly with error rather than quadratically.