Matrix Perturbation Theory

How far can eigenvalues and eigenvectors move when the matrix wobbles? Eigenvalues shift gently; eigenvectors depend on how lonely they are.

A 0.1 off diagonal nudge moves eigenvalues from 4 and 2 to 4.005 and 1.995, staying inside the Weyl allowance while the eigenvectors tilt toward each other

Why Does This Exist?

Real matrices wobble: rounding error, noisy covariance estimates, weights that keep training. Perturbation theory answers the stability question before you trust a decomposition: if $A$ becomes $A + E$ , how far do the eigenvalues and eigenvectors move? The answers split in a way that surprises beginners. Eigenvalues of symmetric matrices are sturdy (they move at most $\lVert E \rVert$ ). Eigenvectors are fragile exactly when eigenvalues crowd (small eigengap). Non-symmetric matrices can betray both.

These bounds run stability analysis everywhere: PCA directions under sampling noise, spectral clustering under graph changes, convergence rates under Hessian drift. Eigenvalues and eigenvectors names the objects; this page prices their stability.

The one idea you need first

A matrix norm $\lVert E \rVert_2$ measures the wobble's worst stretch, from norms. The eigengap is the distance between neighbouring eigenvalues: how lonely each one is. Bounds below pair wobble size against loneliness.

Think of It Like This

Two radio stations, one strong signal apart

Two radio stations broadcast on nearby frequencies. A little static (the perturbation) barely moves where each station comes through clearest (the eigenvalues): you retune by a hair.

But separating the stations (the eigenvectors) depends on their spacing. Far apart on the dial, each tunes cleanly despite static. Nearly overlapping, a whisper of noise blends them and no tuner isolates either. The eigengap is the dial spacing, and Davis-Kahan is the statement that separation quality scales with it.

Non-normal matrices are broken radios where the dial markings themselves shift under static. That regime needs pseudospectra, a different instrument.

How It Actually Works

Three bounds, each with its job:

Weyl: for symmetric $A$ and symmetric $E$ , $|\lambda_i(A+E) - \lambda_i(A)| \le \lVert E \rVert_2$ for every $i$ (suitably ordered). Eigenvalues cannot outrun the wobble.
Davis-Kahan: the sine of the angle between true and perturbed eigenvectors is bounded by $\lVert E \rVert_2 / \delta$ , where $\delta$ is the eigengap. Lonely eigenvectors hold still; crowded ones swing.
Bauer-Fike: for diagonalisable $A = V\Lambda V^{-1}$ , eigenvalues move at most $\kappa(V)\lVert E \rVert$ . The eigenvector conditioning $\kappa(V)$ prices non-normality: orthogonal $V$ costs 1, skewed $V$ multiplies the damage.

Worked example

Perturb $A = \mathrm{diag}(4, 2)$ with $E = \begin{bmatrix} 0 & 0.1 \\ 0.1 & 0 \end{bmatrix}$ (size $\lVert E \rVert_2 = 0.1$ ). The perturbed matrix $\begin{bmatrix} 4 & 0.1 \\ 0.1 & 2 \end{bmatrix}$ has trace 6 and determinant $7.99$ , so:

\lambda = \frac{6 \pm \sqrt{36 - 31.96}}{2} = \frac{6 \pm \sqrt{4.04}}{2} \approx 4.005,\ 1.995

Eigenvalues moved $0.005$ , twenty times inside the Weyl allowance of $0.1$ . The eigengap here is $\delta = 2$ , so Davis-Kahan caps eigenvector tilt at $\sin\theta \le 0.1/2 = 0.05$ : barely a tremor. Shrink the gap to $0.01$ and the same wobble allows total eigenvector scrambling.

Code

The point is the check: verify the Weyl bound holds on the worked example.

1import numpy as np2
3
4A = np.array([[4.0, 0.0], [0.0, 2.0]])5E = np.array([[0.0, 0.1], [0.1, 0.0]])6before = np.linalg.eigvalsh(A)7after = np.linalg.eigvalsh(A + E)8print([round(float(v), 3) for v in after])  # -> [1.995, 4.005]9print(float(np.max(np.abs(after - before))) <= float(np.linalg.norm(E, 2)))  # -> True

Watch Out For

Trusting eigenvectors across a tiny eigengap

PCA directions with nearly tied variance (eigenvalues 4.001 vs 4.000) swing wildly between data samples: the subspace is stable but the individual axes are not. Downstream code that names "component 3" as a fixed meaning breaks on refit. Compare subspaces (projection distances) or merge tied directions instead of tracking shaky axes.

Applying symmetric bounds to non-symmetric matrices

Weyl and Davis-Kahan need symmetry (or normality). A non-normal matrix can move eigenvalues far beyond $\lVert E \rVert$ : the bound carries $\kappa(V)$ , which reaches $10^8$ in real examples. Check normality ( $AA^T = A^T A$ ) before quoting the gentle bounds; otherwise budget for pseudospectral behaviour.

The Quick Version

Weyl: symmetric eigenvalues drift at most the wobble size; the example moves 0.005 of 0.1.
Davis-Kahan: eigenvector tilt is bounded by wobble over eigengap, so lonely axes hold still.
Bauer-Fike prices non-normality through the eigenvector conditioning $\kappa(V)$ .
Tied eigenvalues mean stable subspaces with unstable individual axes: compare subspaces.
Symmetric-only bounds on non-symmetric matrices understate the damage badly.