Summation and Product Notation

Sigma adds a whole list without writing every plus, Pi multiplies one. Means, losses and likelihoods are all Sigma in disguise, and logs turn Pi into Sigma.

Sigma from 1 to 4 of i squared expands to 1 plus 4 plus 9 plus 16 and totals 30, the compact loop every loss function is written in

Why Does This Exist?

ML sums constantly: over examples, over classes, over sequence positions, over pixels. Writing every term is impossible at a million examples and unreadable at ten. Summation notation compresses the whole loop into one line: $\sum_{i=1}^{n} a_i$ means add $a_1$ through $a_n$ . Product notation $\prod$ does the same for multiplication, and it appears wherever independent probabilities multiply into likelihoods.

Reading this notation is non-negotiable: every loss, every expectation and every gradient is written in it. Writing it keeps derivations honest about what varies with the index and what is constant. Expectations build directly on it in expectation, variance and covariance; the other great consumer, matrix products as sums, is matrix multiplication.

The one idea you need first. The letter under Sigma is a counter that walks from the bottom number to the top one, and the rule beside it says what to add per stop. $\sum_{i=1}^{4} i^2$ walks $i$ through 1, 2, 3, 4 and adds the squares.

Think of It Like This

A cashier's tally tape

A receipt tape lists each item and the register adds them: same operation per line, running total at the end. Sigma is the tape header ("add the price column, all lines") and the total in one line, without reprinting every item.

The index is the line number. The bounds say which lines count. The rule names the column. Constants factor out exactly like a store-wide discount applies once to the total, not per line.

Where it stops: tapes are finite and ordered, and Sigma usually is too, but analysis sums to infinity with convergence questions no register ever faces. Finite ML sums stay firmly in cashier territory.

How It Actually Works

Anatomy of $\sum_{i=1}^{n} a_i$ : $\sum$ says add, $i$ is the index, $1$ and $n$ are the lower and upper bounds, $a_i$ is the rule. Expand by walking: $\sum_{i=1}^{4} i^2 = 1^2 + 2^2 + 3^2 + 4^2 = 1 + 4 + 9 + 16 = 30$ .

Three manipulations cover most derivations. Constants factor out: $\sum_i c\,a_i = c\sum_i a_i$ . Sums split: $\sum_i (a_i + b_i) = \sum_i a_i + \sum_i b_i$ . Index shifts relabel freely: summing $i = 0$ to $n - 1$ of $a_{i+1}$ is the same total. The mean is the signature application: $\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$ .

Products mirror all of it: $\prod_{i=1}^{4} i = 1 \times 2 \times 3 \times 4 = 24$ . And the bridge between them: $\log\prod_i p_i = \sum_i \log p_i$ , the log that rescues likelihoods from underflow, detailed in logarithms.

Worked example: a sum of squares, a product, and a mean

Sum: $\sum_{i=1}^{4} i^2 = 1 + 4 + 9 + 16 = 30$ . Each term computed, then one addition chain.

Product: $\prod_{i=1}^{4} i = 24$ . Note $4! = 24$ : the factorial is Pi notation in disguise.

Mean of $[2, 4, 6, 8]$ : $\sum = 2 + 4 + 6 + 8 = 20$ , count $n = 4$ , mean $20/4 = 5$ . The diagram above expands the sum-of-squares version of this same routine.

Code

1import math2
3terms: list[int] = [1, 4, 9, 16]4print(sum(v**2 for v in [1, 2, 3, 4]))  # -> 305print(math.prod([1, 2, 3, 4]))          # -> 246print(sum(terms) / len(terms))          # -> 7.5

sum over a generator is Sigma transcribed: rule on the left, walk on the right. math.prod is Pi. The third line warns against skimming: the mean of the squares is $30/4 = 7.5$ , not the square of the mean.

Watch Out For

Translating 1-based math to 0-based code

Math writes $\sum_{i=1}^{n} a_i$ ; Python writes sum(a[i] for i in range(n)) with a[0] first. The translation shifts every index by one, and hand-rolled loops that start at 1 skip the first element and read one past the last. The symptom is a loss averaged over $n - 1$ examples with an out-of-bounds crash waiting at the edge.

Write the generator form and let range own the counting. When indices must match a paper, assert the first and last index explicitly.

Splitting a sum of products into products of sums

$\sum_i a_i b_i$ is not $(\sum_i a_i)(\sum_i b_i)$ . With $a = b = [1, 2]$ : the left side is $1 + 4 = 5$ , the right side is $3 \times 3 = 9$ . Splitting is legal across addition, never across multiplication inside the rule. Dot products and covariances are exactly this shape, so the slip corrupts the most-used sums in ML.

Expand on two-element vectors whenever a derivation factors inside a Sigma. If the factors both vary with the index, they stay inside together.

The Quick Version

$\sum_{i=1}^{n} a_i$ walks the index and adds the rule; $\prod$ walks and multiplies.
Constants factor out, sums split over addition, indices relabel freely.
$\sum_{i=1}^{4} i^2 = 30$ ; $\prod_{i=1}^{4} i = 24$ ; mean is one $n$ th of a sum.
$\log\prod p_i = \sum \log p_i$ : the underflow rescue for multiplied probabilities.
Math counts from 1, Python from 0: translate the bounds, never the habit.