Characteristic Functions

A characteristic function spins a distribution into waves. The wave pattern encodes every moment and identifies the distribution completely.

The characteristic function of a standard normal is a decaying wave with phi(1) equal to e to the minus one half, about 0.607

Why Does This Exist?

Sums of independent variables are painful in distribution space: densities convolve, an integral per summand. In Fourier space they multiply, one factor per summand. The characteristic function $\\varphi(t) = \\mathbb{E}[e^{itX}]$ is that Fourier view, and unlike its cousin the moment-generating function, it always exists because $|e^{itX}| = 1$ .

It earns its keep three ways: moments fall out by differentiating at zero, independence becomes multiplication, and the central limit theorem becomes a two-line limit of $(1 - t^2/2n)^n \\to e^{-t^2/2}$ . It builds on probability distributions by giving every one of them a uniform handle.

The one idea you need first

$e^{itX} = \\cos(tX) + i\\sin(tX)$ wraps the variable around the unit circle at speed $t$ . Averaging that position gives a complex number inside the unit disc. Fast oscillation averages toward zero, which is why $\\varphi(t)$ decays for spread-out laws.

Think of It Like This

Chords instead of notes

Identifying a chord by hearing each note separately is slow. Hearing the combined shimmer is instant, and a trained ear recovers the notes from it. The distribution is the separate notes; the characteristic function is the shimmer.

Addition of independent variables is like stacking chords: the shimmers multiply while the note lists convolve. The analogy stops at inversion: recovering notes from a shimmer needs an integral, and two very different mixtures can shimmer alike over short ranges.

How It Actually Works

Definition: $\\varphi_X(t) = \\mathbb{E}[e^{itX}]$ for real $t$ . Always $\\varphi(0) = 1$ and $|\\varphi(t)| \\leq 1$ . Key rules, each with $X$ and $Y$ independent:

Independence: $\\varphi_{X+Y}(t) = \\varphi_X(t)\\,\\varphi_Y(t)$ .
Moments: $\\mathbb{E}[X^k] = (-i)^k \\varphi^{(k)}(0)$ when the moment exists.
Uniqueness: $\\varphi$ determines the law; inversion recovers the CDF.
Continuity: $X_n \\to X$ in distribution exactly when $\\varphi_n(t) \\to \\varphi(t)$ pointwise.

Worked example

Standard normal $X \\sim N(0, 1)$ : $\\varphi(t) = e^{-t^2/2}$ . At $t = 1$ , $\\varphi(1) = e^{-0.5} \\approx 0.6065$ . Bernoulli check with $P(X=1) = 0.3$ : $\\varphi(t) = 0.7 + 0.3e^{it}$ , so $\\varphi(0) = 1$ and $|\\varphi(\\pi)| = |0.7 - 0.3| = 0.4$ . Differentiate the Bernoulli case: $\\varphi'(0) = 0.3i$ , and $(-i)(0.3i) = 0.3$ , the mean. The machinery returns the number you started with.

Code

1import numpy as np2
3rng = np.random.default_rng(3)4x = rng.standard_normal(200000)5for t in [0.0, 1.0, 2.0]:6    emp = np.exp(1j * t * x).mean()  # empirical phi(t)7    print(t, round(float(emp.real), 3))  # -> 1.0, 0.607, 0.135

Real parts land at 1.0, 0.607, and 0.135 against exact $e^{-t^2/2}$ values 1, 0.6065, and 0.1353.

Watch Out For

Swapping in the MGF where it does not exist

The moment-generating function $M(t) = \\mathbb{E}[e^{tX}]$ blows up for heavy tails like Cauchy or log-normal sums, and code that evaluates it returns inf. People then conclude the distribution has no handle.

Use $\\varphi(t)$ instead: it exists for every law. Reach for the MGF only when you have verified exponential tails.

Reading pointwise closeness as uniform closeness

$\\varphi_n \\to \\varphi$ pointwise gives convergence in distribution, but densities may still spike: pointwise control at each $t$ is not uniform control. Smoothing claims need more, like an integrable majorant or a local limit theorem.

If the goal is density approximation, verify the extra integrability instead of quoting the continuity theorem alone.

The Quick Version

$\\varphi(t) = \\mathbb{E}[e^{itX}]$ is the Fourier handle on a law; it always exists with $\\varphi(0) = 1$ and $|\\varphi| \\leq 1$ .
Independence turns convolution into multiplication: $\\varphi_{X+Y} = \\varphi_X \\varphi_Y$ .
Moments come from derivatives at zero; the law is pinned down by uniqueness and inversion.
Standard normal: $\\varphi(1) = e^{-0.5} \\approx 0.607$ , matched by simulation.
Pointwise convergence of $\\varphi_n$ is exactly convergence in distribution.
Prefer it over the MGF whenever tails might be heavy.