Convergence Theorems

The two licenses that let limits move inside integrals: climbing sequences and dominated sequences.

Monotone functions climb to their limit from below while dominated functions stay under one integrable roof

Why Does This Exist?

Training moves limits inside expectations everywhere: empirical averages approaching true risk, truncated losses approaching the full loss, gradients of limits versus limits of gradients. Without a license the swap is a guess, and guesses fail on spikes that run away.

Two theorems cover almost every ML case. Monotone convergence handles rising non-negative sequences. Dominated convergence handles sequences trapped under one integrable roof, no matter how they oscillate.

This page states both with the failure that each prevents. The integral itself is in Lebesgue integration, and expectation practice is in integrals in probability.

The one idea you need first

Swapping means $\lim \int f_n = \int \lim f_n$ . Pointwise convergence alone does not imply it. The moving bump with area $1$ sliding to infinity converges pointwise to $0$ but keeps integral $1$ .

Think of It Like This

Water filling a tank, with and without a lid

Monotone: water pours in and never drains. The level only rises, so the final volume is the limit of partial volumes. No surprise is possible.

Dominated: water sloshes but a fixed lid caps every wave. Since nothing escapes above the lid and the tank floor is fixed, the average settles with the surface. Where the analogy stops: the lid must itself have finite volume; an infinite lid licenses nothing.

How It Actually Works

Monotone convergence: if $0 \le f_1 \le f_2 \le \cdots$ and $f_n \to f$ pointwise, then $\int f_n \to \int f$ , allowing $\infty$ . Dominated convergence: if $f_n \to f$ pointwise and $|f_n| \le g$ with $\int g < \infty$ , then $\int f_n \to \int f$ and $\int |f_n - f| \to 0$ . Fatou: $\int \liminf f_n \le \liminf \int f_n$ for non-negative $f_n$ .

In probability: if $X_n \uparrow X$ then $\mathbb{E}[X_n] \uparrow \mathbb{E}[X]$ ; if $|X_n| \le Y$ with $\mathbb{E}[Y] < \infty$ then $\mathbb{E}[X_n] \to \mathbb{E}[X]$ .

Worked example

Monotone: $f_n = 1 - (0.5)^n$ constants rising: $0.5, 0.75, 0.875, 0.9375$ on $[0,1]$ . Integrals equal the constants, limit $1$ , and integrals converge to $1$ . Dominated: $f_n(x) = \sin(nx)/n$ on $[0,1]$ , bounded by $g = 1/n \le 1$ then by $1$ . Values shrink: sup $|f_n| = 1/n$ giving $1, 0.1, 0.01$ for $n = 1, 10, 100$ . Integrals tend to $0$ , the integral of the pointwise limit $0$ .

Watch Out For

Swapping without a dominating function

Symptom: $f_n = n$ on $(0, 1/n)$ has area $1$ but pointwise limit $0$ ; the swap reports $0$ . Fix: exhibit an integrable $g$ above all $|f_n|$ ; here none exists since $\sup_n f_n$ is not integrable.

Applying monotone to decreasing sequences

Symptom: $1/n$ decreasing to $0$ is treated as monotone convergence from below, hiding the non-negativity direction. Fix: flip sign or use dominated with $g = 1$ ; monotone needs rising from below, not falling.

The Quick Version

Monotone convergence licenses limits for rising non-negative sequences, even to infinity.
Dominated convergence licenses them under one integrable roof, covering oscillation.
Fatou gives the one-sided bound when no roof exists.
Moving bumps with constant area show why pointwise alone is never enough.
In ML both justify passing sample or iteration limits inside risk and expectation.