Lp Spaces

Families of functions with finite p-th power integrals, the home of norms, errors and convergence that ML optimizes.

Why Does This Exist?

Errors need sizes: mean absolute error, root mean square error, worst-case error. Each is an $L^p$ norm with different spike tolerance. Theory needs the same sizes to state convergence, generalization and stability without redefining distance each time.

$L^p$ spaces collect measurable functions with $\int |f|^p < \infty$ , normed by $\|f\|_p = (\int |f|^p)^{1/p}$ . For $p \ge 1$ these are Banach spaces; $L^2$ is Hilbert with an inner product.

This page defines the family and its two workhorse inequalities. Vector norms are in norms and probabilistic moments in expectation, variance and covariance.

The one idea you need first

A norm measures size with triangle inequality: $\|f + g\| \le \|f\| + \|g\|$ . $L^p$ norms apply this to functions via integrals instead of sums.

Think of It Like This

Fines for speeding

$L^1$ fines every mile-per-hour over the limit equally. $L^2$ fines the square, so a brief $30$ -over spike costs far more than long $5$ -over cruising. $L^\infty$ fines only the peak speed.

Error norms choose the fine schedule. Robust losses pick $L^1$ to forgive spikes; least squares picks $L^2$ to punish them. Where the analogy stops: fines are one-sided, while norms treat positive and negative deviations symmetrically.

How It Actually Works

$\|f\|_p = (\int |f|^p \, dm)^{1/p}$ for $1 \le p < \infty$ , and $\|f\|_\infty = \text{ess sup } |f|$ . Holder: $\int |fg| \le \|f\|_p \|g\|_q$ with $1/p + 1/q = 1$ . Minkowski: $\|f + g\|_p \le \|f\|_p + \|g\|_p$ . Both need $p \ge 1$ .

Worked example

On $[0,1]$ , $f(x) = x^{-0.4}$ . Then $|f|^2 = x^{-0.8}$ integrates to $[x^{0.2}/0.2]_0^1 = 5$ , so $\|f\|_2 = \sqrt{5} \approx 2.236$ . For $p = 3$ , $|f|^3 = x^{-1.2}$ diverges at $0$ , so $f \notin L^3$ . Same function lives in $L^1$ and $L^2$ but not $L^3$ : higher $p$ is stricter about spikes.

Watch Out For

Using triangle inequality below p equals 1

Symptom: $\|f + g\|_{0.5} \le \|f\|_{0.5} + \|g\|_{0.5}$ assumed, then sparsity penalties violate it and proofs collapse. Fix: for $p < 1$ only a quasi-triangle holds with a constant; state it or stay at $p \ge 1$ .

Assuming norm equivalence in infinite dimensions

Symptom: "all norms are equivalent" quoted for function spaces, so $L^1$ convergence is read as $L^\infty$ convergence. Fix: equivalence holds only in finite dimensions; in function spaces exhibit the mode of convergence explicitly.

The Quick Version

$L^p$ collects functions with finite $\int |f|^p$ ; $p = 1, 2, \infty$ give MAE, RMSE and worst-case sizes.
Holder controls products and Minkowski controls sums for $p \ge 1$ .
Larger $p$ punishes spikes harder: $x^{-0.4}$ is in $L^2$ with norm $2.236$ but outside $L^3$ .
$L^2$ alone has an inner product; all $L^p$ with $p \ge 1$ are complete.
Name $p$ whenever an error or risk bound is stated.