Exponents and Roots for ML

An exponent counts repeated multiplication and a root undoes it. Together they set the scale of everything from parameter counts to learning-rate schedules.

Bars doubling from 1 to 2 to 4 to 8 to 16 show each step multiplying the last, which is why ten doublings reach 1024

Why Does This Exist?

Exponents are how ML talks about scale. A million parameters is $10^6$ . A batch dimension of 1024 is $2^{10}$ . Halving a learning rate every ten epochs is exponential decay. And roots run the film backwards: standard deviation is the square root of variance, RMS normalisation divides by a root, and every distance is a root of summed squares.

Three combining rules plus the root-as-fractional-power idea cover nearly every computation you will meet. The deep end of powers of two, why big-O expressions are full of them, belongs to computational complexity for ML. The partner operation that asks "which power?" instead of "what power gives?" is the logarithm.

The one idea you need first. Multiplication is repeated addition: $3 \times 4$ is three fours. Exponentiation repeats multiplication instead: $2^4$ is four twos multiplied, $2 \times 2 \times 2 \times 2 = 16$ .

Think of It Like This

Folding a strip of paper

Fold a paper strip once: 2 layers. Twice: 4. Three times: 8. Each fold doubles the last count, so ten folds give $2^{10} = 1024$ layers, a stack thicker than your fist from a single sheet.

The root is unfolding. Handed a stack of 144 sheets and told each fold doubled it, you ask how many doublings: $\sqrt{144} = 12$ is the wrong question phrased backwards, it asks the side of a square, but unfolding asks $2^n = 144$ , which has no whole answer. Roots and logarithms split the backwards questions between them: roots undo the squaring, logarithms undo the repeated doubling.

Where it stops: paper refuses after about seven folds, while numbers keep doubling forever. The analogy shows the growth; only the arithmetic shows how violent it gets by fold 30, past a billion.

How It Actually Works

The definition: $a^n$ multiplies $n$ copies of $a$ , so $2^5 = 32$ . Three laws do the combining. Multiplying powers of the same base adds exponents: $a^m \times a^n = a^{m+n}$ , so $2^3 \times 2^4 = 2^7 = 128$ . Raising a power multiplies exponents: $(a^m)^n = a^{mn}$ , so $(10^2)^3 = 10^6$ , one million. Dividing subtracts: $a^m / a^n = a^{m-n}$ . Two edge cases finish the system: $a^0 = 1$ for any nonzero $a$ , and $a^{-n} = 1/a^n$ , so $2^{-1} = 0.5$ .

A root is a fractional power. The square root $\sqrt{a} = a^{1/2}$ asks which number squared gives $a$ : $\sqrt{144} = 12$ because $12^2 = 144$ . The cube root is $a^{1/3}$ . Roots inherit every power law, which is why $\sqrt{a \times b} = \sqrt{a} \times \sqrt{b}$ .

Worked example: powers of two and a memory budget

Build $2^{10}$ from $2^5 = 32$ : $2^{10} = (2^5)^2 = 32^2 = 1024$ . That is why 1024, not 1000, sits under every batch size and embedding dimension.

Now a budget: one million parameters at 4 bytes each. $10^6 \times 4 = 4 \times 10^6$ bytes, about $4$ MB. At 2 bytes (half precision) it halves to $2$ MB. Exponent notation turns the budget into one multiplication instead of counting zeros.

Code

1print(2**10)              # -> 10242print(2**3 * 2**4)        # -> 1283print(144**0.5)           # -> 12.04
5params: int = 10**66print(params * 4)         # -> 4000000

The third line uses the fractional-power form of the root, the one worth internalising. The last line counts bytes for a million-parameter model in full precision: four million, or 4 MB.

Watch Out For

Distributing a power over a sum

$(a + b)^2$ is not $a^2 + b^2$ . With $a = 1, b = 2$ : the left side is $3^2 = 9$ , the right side is $1 + 4 = 5$ . The missing $2ab = 4$ is the cross term, and it is exactly what variance expansions and quadratic features are made of.

Expand by hand on small numbers whenever a derivation squares a sum. If a loss decomposition drops the cross term without justification, the math is wrong.

The square root of a square is the absolute value

$\sqrt{x^2}$ is $|x|$ , not $x$ . For $x = -4$ : $(-4)^2 = 16$ and $\sqrt{16} = 4$ , the sign is gone. Code that "simplifies" a root of a square introduces a sign flip on every negative input, and norms rely on this behaviour deliberately.

Keep the absolute value in any hand-derived simplification. When a distance comes out negative, this is the first line to check.

The Quick Version

$a^n$ multiplies $n$ copies; same-base powers combine by adding, multiplying or subtracting exponents.
$a^0 = 1$ and $a^{-n} = 1/a^n$ ; $2^{10} = 1024$ is why computing loves powers of two.
Roots are fractional powers: $\sqrt{a} = a^{1/2}$ , and they obey the same laws.
$(a+b)^2$ keeps a $2ab$ cross term; dropping it silently breaks derivations.
$\sqrt{x^2} = |x|$ ; the sign does not survive the round trip.