Real Analysis

The rules for limits, continuity and proof that make calculus honest, so loss curves and convergence claims mean what they say.

Why Does This Exist?

Training is full of sentences like "the loss converges" or "the gradient goes to zero". Without real analysis those are wishes. With it they are checkable claims: a sequence converges when its tail stays arbitrarily close to a limit, a function is continuous when small input changes give small output changes, and proofs use the completeness of the reals.

It matters because ML reuses the same few analytic moves everywhere. A learning-rate schedule assumes a series converges. A universal-approximation claim assumes continuous functions can be approached uniformly on compacts. Early stopping assumes a monitored sequence actually settles.

This page covers the core objects: sequences, limits, suprema, continuity and completeness. Neighbours teach the pieces that build on them, such as derivatives and partial derivatives and integration and area.

The one idea you need first

A limit is a promise about tails. To say $a_n \to L$ means: pick any tolerance $\epsilon > 0$ , and there is a point $N$ after which every $|a_n - L| < \epsilon$ . The number $N$ may depend on $\epsilon$ . Smaller tolerance usually needs larger $N$ .

Think of It Like This

A fence that closes in on a sheep

A sheep stands at position $L$ in a field. You build a fence with two rails, $L - \epsilon$ and $L + \epsilon$ . A sequence of sheepdogs runs in from far away, one after another. Convergence means: no matter how narrow you make the fence, all but finitely many dogs end up inside it.

The fence is the epsilon band in the diagram. The dogs are $a_1, a_2, \ldots$ . The guarantee fails if even one straggler keeps stepping outside infinitely often. Where the analogy stops: real proofs also need completeness, the fact that the field has no missing holes where a Cauchy herd could aim at nothing.

How It Actually Works

A sequence is an ordered list $a_1, a_2, \ldots$ of real numbers. It converges to $L$ when the epsilon condition above holds. A series $\sum a_n$ converges when its partial sums $S_N = a_1 + \cdots + a_N$ converge as a sequence.

The supremum $\sup S$ is the least upper bound of a set $S$ . For $S = \{1 - 1/n\}$ , values are $0, 0.5, 0.667, 0.75, \ldots$ , bounded above by $1$ , and $\sup S = 1$ even though $1$ is never attained. Completeness says every nonempty bounded-above set of reals has a supremum. Rationals lack this: $\{q : q^2 < 2\}$ has no rational supremum.

Continuity at $x_0$ means $\lim_{x \to x_0} f(x) = f(x_0)$ : the epsilon-delta version says for every $\epsilon > 0$ there is $\delta > 0$ with $|x - x_0| < \delta$ implying $|f(x) - f(x_0)| < \epsilon$ .

Worked example

Take $a_n = 1/n$ : $1, 0.5, 0.333, 0.25, 0.2, 0.167$ . Claim $L = 0$ . Fix $\epsilon = 0.1$ . Need $1/n < 0.1$ , so $n > 10$ . Take $N = 10$ . Every $n > 10$ satisfies $|a_n - 0| < 0.1$ . For $\epsilon = 0.01$ , take $N = 100$ . Since such $N$ exists for every $\epsilon$ , $a_n \to 0$ . The series $\sum 1/2^n = 1 + 0.5 + 0.25 + \cdots$ has partial sums $1, 1.5, 1.75, 1.875$ , converging to $2$ .

Watch Out For

Confusing bounded with convergent

Bounded means the sequence stays in some interval. Convergent means it settles on one number. The symptom: treating oscillation as convergence because values look stable in range. The sequence $(-1)^n$ is bounded in $[-1, 1]$ but never settles. Fix: check the tail against a candidate $L$ with shrinking $\epsilon$ , not just the range.

Assuming the limit can be moved inside

Writing $\lim f(a_n) = f(\lim a_n)$ is only legal when $f$ is continuous at the limit. The symptom: passing a limit through a step or indicator and getting 0 versus 1 wrong. Fix: verify continuity at that point first, or keep the limit outside.

The Quick Version

Real analysis turns "converges" into a check: tails stay inside every epsilon band.
Supremum is the least upper bound; completeness guarantees it exists for bounded real sets.
Continuity lets limits pass through functions, and only then.
Series converge when partial sums converge, as with $1 + 0.5 + 0.25 + \cdots = 2$ .
No convergence claim in training means anything without the epsilon- $N$ test behind it.