Limits and Continuity

A limit is the value a function homes in on, and continuity means the graph arrives there with no jump, hole, or break.

The curve x squared minus 4 over x minus 2 runs straight except for a hole at x equals 2: arrows from both sides converge on 4, and filling the hole with the value 4 makes it continuous

Why Does This Exist?

Derivatives are limits, integrals are limits, and training stability is continuity. Gradient descent assumes small steps change the loss a small amount, which is exactly continuity of the loss. Step functions and hard thresholds break that promise, which is why they get soft replacements like sigmoids. And every 0/0 form in ML, from L'Hôpital's rule cases to normalized attention weights, resolves through limit thinking.

Single-variable calculus places this idea on the map. This page teaches the idea itself: approach from both sides, name the value, and check the graph actually arrives.

The one idea you need first

Plugging $x = a$ into $f(x)$ evaluates the function at the point. A limit asks what happens near the point, which can differ: a hole at $x = 2$ changes nothing about the approach from 1.999 and 2.001. Evaluation and approach agree only for continuous functions.

Think of It Like This

A bridge with one plank missing

Walk toward a footbridge. From the left bank you approach plank 4, and from the right bank you also approach plank 4, but plank 4 itself was never installed: one hole in an otherwise fine bridge. The limit is where both approaches agree (plank 4's position); continuity is whether the plank is actually there to stand on.

Laying the missing plank, defining the value to equal the limit, repairs the bridge. That repair has a technical name, removable discontinuity, and it is the friendliest kind of break: the approaches already agree, so one well-chosen value fixes everything.

How It Actually Works

The limit $\lim_{x \to a} f(x) = L$ means: $f(x)$ gets arbitrarily close to $L$ whenever $x$ is close to (but not equal to) $a$ , from both sides. One-sided limits $\lim_{x \to a^-}$ and $\lim_{x \to a^+}$ check each side; the two-sided limit exists only if both agree.

Continuity at $a$ needs three things: $f(a)$ is defined, the limit exists, and they are equal:

\lim_{x \to a} f(x) = f(a)

In plain words: approach the point and arrive at the value. Breaks come in three flavors: holes (removable, fixable by one value), jumps (left and right disagree, as in step functions), and blowups (values run to infinity, as in $1/x$ at 0).

Worked example

f(x) = \frac{x^2 - 4}{x - 2}, \qquad x \neq 2

Plugging $x = 2$ gives $0/0$ : undefined, a hole. Factor and cancel for $x \neq 2$ :

f(x) = \frac{(x-2)(x+2)}{x-2} = x + 2

So near 2, the function behaves exactly like the line $x + 2$ . From the left: $f(1.999) = 3.999$ . From the right: $f(2.001) = 4.001$ . Both home in on 4, hence $\lim_{x \to 2} f(x) = 4$ . Defining $f(2) = 4$ fills the hole and makes the function continuous there: approach meets arrival.

Watch Out For

Plugging into a 0 over 0 form and declaring the answer

The symptom is "undefined, so no limit" for expressions whose limit plainly exists, like the worked example's 4. A 0/0 form is not an answer; it is a signal to do algebra first: factor, cancel, rationalize, or reach for L'Hôpital's rule. Only after simplification may you evaluate, and one-sided disagreement still vetoes the two-sided limit.

Assuming your loss is continuous where it is not

Hard thresholds, argmax selections, and quantization steps all jump, so gradients through them are zero or undefined and descent stalls. The symptom is training that flatlines exactly at a discretization boundary. The fix is a continuous relaxation during training (softmax instead of argmax, straight-through estimators where needed), keeping the jumps for inference only. Continuity is an assumption to verify, not a default.

The Quick Version

A limit is the approached value from both sides; evaluation is the value at the point.
Continuity means the two agree: $\lim_{x \to a} f(x) = f(a)$ .
Breaks are holes (fixable), jumps (sides disagree), or blowups (infinite).
$(x^2-4)/(x-2)$ has limit 4 at $x = 2$ after cancelling, despite the 0/0 hole.
A 0/0 form means simplify first, never "undefined, no limit."
Gradient methods assume continuous losses, so jumps need soft relaxations.