Compact Operators

Why Does This Exist?

Kernel integral equations, covariance operators and attention matrices are infinite-dimensional yet practically low-rank: a few modes carry the action. Compactness is the property that makes this rigorous. It guarantees discrete spectra, approximability by finite matrices, and the Fredholm alternative (finite-dimensional-flavored solvability) for equations that otherwise resist all matrix intuition.

This page defines compactness and truncates one operator with exact error. Matrix factors behind the truncation are in singular value decomposition; the eigenvalue picture is eigenvalues and eigenvectors.

The one idea you need first

A finite-rank operator has finite-dimensional range: $Tx = \sum_{k=1}^r \langle x, u_k \rangle v_k$. Operator norm $\|T\|$ is the top stretch factor. Compactness means finite-rank maps approximate $T$ in that norm.

Think of It Like This

How It Actually Works

Bounded $T: X \to Y$ is compact when the image of the unit ball has compact closure: bounded inputs map to sets small enough that finite nets approximate them. Equivalently, $T$ is a norm limit of finite-rank operators:

In plain words: truncate and the worst-case error vanishes as rank grows. On Hilbert space, compact $T$ has a singular expansion $Tx = \sum s_k \langle x, u_k \rangle v_k$ with $s_k \to 0$; truncation error is $s_{r+1}$. Hilbert-Schmidt operators ($\sum s_k^2 < \infty$) are the tractable subclass. Fredholm: $I - T$ is injective iff surjective, with finite-dimensional kernel.

Worked example

Diagonal $D$ on $\ell^2$ with entries $1, 1/2, 1/3, 1/4, 1/5, \ldots$. Singular values are the entries themselves, tending to $0$: compact. Rank-3 truncation $D_3$ keeps the first three; error $\|D - D_3\| = 1/4 = 0.25$ (largest dropped entry).

Hilbert-Schmidt check on the first three modes: $1^2 + (1/2)^2 + (1/3)^2 = 1 + 0.25 + 0.111 = 1.361$, so partial HS norm $\sqrt{1.361} \approx 1.167$. Full $\sum 1/k^2 = \pi^2/6 \approx 1.645$, HS norm $\approx 1.283$: finite, so Hilbert-Schmidt, hence compact twice over.

Code

Truncate the diagonal and read off the exact tail error.

1import numpy as np2
3
4def truncation_error(s: np.ndarray, rank: int) -> tuple[float, float]:5    kept_hs = float(np.sqrt((s[:rank] ** 2).sum()))  # HS norm of kept part6    tail = float(s[rank]) if rank < s.size else 0.0  # operator error7    return kept_hs, tail8
9
10s = 1.0 / np.arange(1, 11)11print(truncation_error(s, 3))  # -> (1.167, 0.25)

Kept HS norm $1.167$, tail error $0.25$: the hand numbers, reproduced.

Watch Out For

The Quick Version

• Compact operators map bounded sets to nearly finite sets and are norm limits of finite-rank maps.
• Singular values tend to zero; rank-$r$ truncation error is the first dropped value.
• Diagonal $(1/k)$ has rank-3 error $0.25$ and HS norm about $1.28$.
• Hilbert-Schmidt ($\sum s_k^2 < \infty$) implies compact with a computable norm.
• The identity on infinite dimensions is never compact.

What to Read Next

• Singular Value Decomposition
• Eigenvalues and Eigenvectors
• Hilbert Spaces
• Bounded Linear Operators