Product Measures

How to build joint distributions from marginals, so pairs, sequences and independence have rigorous mass.

Product measure multiplies rectangles: the mass of a pair is the iterated mass of its slices

Why Does This Exist?

Pairs need probabilities: $(x, y)$ from data and label, $(z_1, z_2)$ from two latents, sequences $x_{1:T}$ . Declaring marginals is not enough; the joint must give every rectangle $A \times B$ a mass consistent with iterated integration.

Product measures provide the joint for independent components and the base for kernels in dependent ones. Fubini's theorem then computes its integrals in either order.

This page builds the independent case. Dependent joints via kernels extend it, and integration order is in Fubini's theorem.

The one idea you need first

A rectangle $A \times B$ pairs an $X$ -event with a $Y$ -event. The product rule sets $(\mu \times \nu)(A \times B) = \mu(A)\nu(B)$ , then extends to the generated sigma-algebra.

Think of It Like This

Area from side lengths

A tiled floor has width distribution $\mu$ and length distribution $\nu$ . The area of one tile is width times length. Whole-floor area adds tile areas.

Rectangles are tiles. Countable additivity extends tiles to oddly shaped rooms. Where the analogy stops: floors are finite, while product measures handle infinite sequences by extension theorems like Kolmogorov.

How It Actually Works

For $\sigma$ -finite $(\Omega_1, \mathcal{F}_1, \mu)$ and $(\Omega_2, \mathcal{F}_2, \nu)$ , a unique product measure $\mu \times \nu$ on $\mathcal{F}_1 \otimes \mathcal{F}_2$ satisfies $(\mu \times \nu)(A \times B) = \mu(A)\nu(B)$ . Sections of measurable sets are measurable, and $\int f \, d(\mu \times \nu)$ equals either iterated integral for non-negative or integrable $f$ .

Independence is product structure: $X, Y$ independent exactly when $P_{(X,Y)} = P_X \times P_Y$ .

Worked example

Two fair coins: $\mu = \nu = (0.5, 0.5)$ on $\{H, T\}$ . Rectangles: $P(H,H) = 0.25$ , $P(H,T) = 0.25$ , each product of halves. With biased $\mu = (0.3, 0.7)$ and $\nu = (0.4, 0.6)$ : masses $0.12, 0.18, 0.28, 0.42$ summing to $1.0$ . Marginals recover by summing rows or columns.

Watch Out For

Multiplying marginals for dependent pairs

Symptom: joint built as product while data has correlation $0.8$ , so modeled $P(X, Y)$ misses the diagonal by $0.3$ . Fix: product is independence; dependent joints need a copula or kernel, not side multiplication.

Assuming products exist without sigma-finiteness

Symptom: uncountable products with counting measure give non-unique extensions and contradictory integrals. Fix: check $\sigma$ -finiteness; for arbitrary processes use Kolmogorov with consistent finite-dimensional laws.

The Quick Version

Product measures set rectangle mass to side-mass product, then extend uniquely.
They are the definition of independence for pairs and sequences.
Biased coins $(0.3, 0.7)$ by $(0.4, 0.6)$ give joint $(0.12, 0.18, 0.28, 0.42)$ .
Iterated integrals against products agree by Tonelli and Fubini.
Dependent structure needs kernels; products alone assert independence.