Bounded Linear Operators

A bounded operator stretches vectors by at most a fixed factor, so small input wobbles stay small at the output. The smallest such factor is its norm.

The unit circle maps through A into an ellipse reaching radius 2, which is exactly the operator norm

Why Does This Exist?

Neural layers, attention maps and integral transforms are all linear operators, and training perturbs their inputs constantly. If an operator amplifies small wobbles without bound, gradients explode, solvers diverge and no stability claim survives. Boundedness is the certificate that amplification stays capped, and the operator norm is the cap.

This page defines boundedness, equates it with continuity, and computes one norm exactly. Size measurement is norms; the rooms operators act in are Banach spaces.

The one idea you need first

A linear map satisfies $T(\alpha x + \beta y) = \alpha Tx + \beta Ty$ . A norm $\|x\|$ measures length. Boundedness asks for one constant $C$ with $\|Tx\| \le C\|x\|$ for every $x$ : relative distortion capped everywhere at once.

Think of It Like This

A photocopier with a max zoom

An office copier enlarges at most 200 percent no matter the original: a thumbnail and a poster both scale by at most that factor. The 200 percent sticker is the operator norm. Pages never come out infinitely large, and halving the input halves the output.

An unbounded copier has no sticker: most pages copy fine, but one cursed original feeds back and grows forever. Where it stops: the sticker is global. Local good behavior on nice inputs never implies boundedness; one bad direction voids it.

How It Actually Works

A linear $T: X \to Y$ between normed spaces is bounded when its operator norm is finite:

\|T\| = \sup_{\|x\| \le 1} \|Tx\| = \sup_{x \ne 0} \frac{\|Tx\|}{\|x\|}

In plain words: feed every input of length at most $1$ , measure the longest output, that maximum stretch is $\|T\|$ . For linear maps, bounded equals continuous equals continuous at zero: one condition buys the other two. Bounded operators compose with $\|ST\| \le \|S\|\|T\|$ , forming a Banach algebra when $X = Y$ is complete.

Worked example

$A = \mathrm{diag}(2, 0.5)$ on $\mathbb{R}^2$ with Euclidean norm. For $x = (x_1, x_2)$ : $\|Ax\|^2 = 4x_1^2 + 0.25x_2^2 \le 4(x_1^2 + x_2^2) = 4\|x\|^2$ , so $\|A\| \le 2$ . The bound is sharp: $x = (1, 0)$ gives $\|Ax\| = 2 = 2\|x\|$ . Hence $\|A\| = 2$ .

Spot check: $x = (0, 1)$ maps to $(0, 0.5)$ with length $0.5 \le 2 \times 1$ . The squashed direction stays comfortably inside the cap the stretched direction sets.

Code

Power iteration estimates the norm (largest singular value) without forming any decomposition.

1import numpy as np2
3
4def operator_norm(A: np.ndarray, steps: int = 50) -> float:5    x = np.ones(A.shape[1])  # starting probe direction6    for _ in range(steps):  # repeatedly stretch and renormalize7        x = A.T @ (A @ x)8        x /= np.linalg.norm(x)9    return float(np.sqrt(x @ (A.T @ (A @ x))))10
11
12A = np.diag([2.0, 0.5])13print(round(operator_norm(A), 3))  # -> 2.0

Fifty stretches land on $2.0$ : the hand-computed norm, recovered numerically.

Watch Out For

Assuming all linear maps are bounded

Symptom: differentiation on polynomials with sup norm looks linear and safe, yet $d/dx$ sends $x^n$ (unit-ish) to outputs of size $n$ : unbounded, discontinuous, solver-breaking. Fix: check the sup ratio on a sequence escaping to high frequency before trusting any infinite-dimensional map.

Confusing matrix norm with entry size

Symptom: the largest entry $2$ happens to equal the norm here, so readers generalize and quote max-entry as the norm elsewhere. Fix: the norm is the top singular value; bound entries by norms, never the reverse.

The Quick Version

Bounded means one global stretch cap: $\|Tx\| \le C\|x\|$ for all inputs at once.
For linear maps, bounded, continuous and continuous-at-zero coincide.
$\mathrm{diag}(2, 0.5)$ has norm exactly $2$ , attained at $(1, 0)$ .
Norms multiply across composition: $\|ST\| \le \|S\|\|T\|$ .
Differentiation on naive function norms is the canonical unbounded trap.