Number Systems: Naturals to Reals

Counting numbers sit inside integers, which sit inside rationals, which sit inside reals, and each bigger set fixes something the smaller one cannot say. ML lives in the reals but computes in finite slices of them.

Integers land exactly on their marks while one half lands between them and the square root of 2 lands near 1.414, already rounded because no fraction reaches it

Why Does This Exist?

Programs keep several kinds of number because each kind promises something different. Label counts are never negative and never fractional, so they want the naturals. Token offsets go below zero, so they want integers. Money-style exact thirds want rationals. Model weights want the reals, the continuous line with no gaps.

The catch is that no computer holds the reals. It holds a finite slice of them, the floats, and the gap between the math and the machine is where $0.1 + 0.2 \neq 0.3$ comes from. Knowing which set each value belongs to tells you which operations are safe, which comparisons need a tolerance, and which counts must stay integers. How the finite slice works is the job of floating-point formats; this page is about the sets themselves.

The one idea you need first. A fraction $p/q$ means $p$ parts out of $q$ equal parts, with $q$ never zero. Everything below is asking which numbers can be written that way and what lives outside.

Think of It Like This

Russian nesting dolls

Four dolls, each inside the next. The smallest doll is the naturals: $1, 2, 3$ , and onward for counting things that exist. Open it and you are in the integers, which add zero and the negatives for debts, offsets and below-sea-level. Inside that sit the rationals, every fraction, filling the gaps between integers with exact ratios. The outermost doll is the reals: everything so far plus the numbers no fraction can name.

Each doll fixes a failure of the one inside. Subtraction can leave the naturals, so integers exist. Division can leave the integers, so rationals exist. And some lengths, like the diagonal of a unit square, escape every fraction, so the reals close the gaps.

Where it stops: dolls are finite and wooden. The sets are infinite, and the outermost one is uncountably larger than the rest, a difference no stack of dolls can show.

How It Actually Works

The four sets, smallest first. Naturals $\mathbb{N} = \{1, 2, 3, \ldots\}$ count things; some conventions include $0$ , which is worth checking once per codebase and then forgetting. Integers $\mathbb{Z} = \{\ldots, -2, -1, 0, 1, 2, \ldots\}$ add zero and negatives, so any subtraction stays inside. Rationals $\mathbb{Q}$ are all ratios $p/q$ with $q \neq 0$ : $1/2$ , $-3/4$ , and also $2/1 = 2$ , so every integer is already rational. Reals $\mathbb{R}$ are the whole continuous line, rationals plus irrationals such as $\sqrt{2}$ and $\pi$ that no fraction equals exactly.

Worked example: sorting five values into their sets

Take $-3$ , $0$ , $1/2$ , $\sqrt{2}$ and $\pi$ .

$-3$ is an integer, hence also rational and real, but not natural. $0$ is an integer (and rational, and real); whether your code calls it natural depends on convention. $1/2 = 0.5$ is rational but not an integer. $\sqrt{2} \approx 1.41421$ is real but irrational: no pair of integers $p, q$ satisfies $p/q = \sqrt{2}$ , a fact the ancient Greeks proved by contradiction. $\pi \approx 3.14159$ is likewise irrational.

The ML sting: $1/3$ cannot be stored exactly in decimal or binary, so 1 / 3 prints $0.3333333333333333$ , a rounded imposter sixteen digits deep. And $0.1 + 0.2$ prints $0.30000000000000004$ , because neither tenth lands exactly on a binary fraction. Both answers are correct arithmetic on rounded inputs, which is exactly why equality tests on floats fail.

Code

1print(0.1 + 0.2)  # -> 0.300000000000000042print(1 / 3)      # -> 0.33333333333333333print(0.1 + 0.2 == 0.3)  # -> False

Three lines that have ended careers' worth of debugging sessions. Neither tenth is exact in binary, so the sum carries both roundings. The third line is the lesson: never test floats for equality, test closeness, which floating-point formats makes precise.

Watch Out For

Testing two floats for equality

After ten thousand updates, a weight that mathematically equals $0.3$ will be $0.29999999999999993$ or similar, and w == 0.3 is False. Convergence checks written as equalities never trigger, and tests comparing full-precision outputs fail on machines the author never touched.

Compare with a tolerance instead: math.isclose(w, 0.3, rel_tol=1e-9). Reserve == for integers, counts and tokens, the values that are exact.

Python floor division on negative indices

// rounds toward negative infinity, not toward zero, so -3 // 2 is $-2$ , not $-1$ . Index arithmetic that wanders below zero lands one slot further left than C-trained intuition expects, and the bug only shows on the negative half of the data.

The fix is to avoid // near zero-crossings, or to use int(-3 / 2) when truncation toward zero is what you mean. Say which rounding you want before you write the operator.

The Quick Version

Naturals count, integers add zero and negatives, rationals add every fraction, reals add the irrationals.
Each set contains the previous one; each exists because an operation escapes the smaller set.
$\sqrt{2}$ and $\pi$ are real but not rational, so every stored value of them is rounded.
Floats are a finite slice of the reals: compare them with tolerance, never with ==.
Integer division floors toward negative infinity in Python, which bites below zero.