Order Statistics

Sort a sample and each position becomes a statistic: the smallest, the median, the largest. Each has its own exact distribution you can write down.

Five unsorted draws of 1.2 to 8.9 sort into positions where the median 4.1 and the maximum 8.9 follow their own laws

Why Does This Exist?

Models ship on extremes and quantiles: worst-case latency, median income, top-1 retrieval score. Sample means have the CLT; extremes do not, and their behavior surprises people who only know averages. The maximum of 100 draws sits far above the mean, and its distribution has a closed form.

Order statistics $X_{(1)} \\leq \\cdots \\leq X_{(n)}$ give that form. For i.i.d. draws with CDF $F$ : $P(X_{(n)} \\leq x) = F(x)^n$ and $P(X_{(1)} > x) = (1-F(x))^n$ . They ground descriptive statistics in sampling theory and feed tolerance intervals in confidence intervals.

The one idea you need first

Sorting does not create information; it renames it by rank. The median of $[5.0, 1.2, 8.9, 3.3, 4.1]$ is $4.1$ , the third of five sorted values. Ranks turn "how big" into "how many fall below".

Think of It Like This

Race finish order

Five runners finish in some order. Nobody records times, only places: first, middle, last. The middle finisher's time estimates the typical pace; the winner's time estimates the frontier.

Knowing the pack's time distribution predicts each place's time distribution. Where it stops: runners interact and pace each other, while the clean formulas need i.i.d. draws with no jostling.

How It Actually Works

For i.i.d. continuous draws with pdf $f$ and CDF $F$ , the $k$ -th order statistic has density $f_{(k)}(x) = \\frac{n!}{(k-1)!(n-k)!}\\,F(x)^{k-1}(1-F(x))^{n-k}f(x)$ . In words: $k-1$ draws below, one near $x$ , $n-k$ above, times arrangements. The joint law of all $n$ is $n!\\prod f(x_{(i)})$ on the sorted simplex.

Worked example

Sample $[1.2, 3.3, 4.1, 5.0, 8.9]$ sorted is $[1.2, 3.3, 4.1, 5.0, 8.9]$ : median $X_{(3)} = 4.1$ , max $X_{(5)} = 8.9$ , range $8.9 - 1.2 = 7.7$ . If draws are Uniform $[0,10]$ , then $F(x) = x/10$ and $P(\\text{max} \\leq 8.9) = 0.89^5 \\approx 0.558$ . Expected max is $\\frac{n}{n+1} \\times 10 = 50/6 \\approx 8.333$ , so 8.9 runs slightly hot but well within range.

Code

1import numpy as np2
3rng = np.random.default_rng(6)4draws = rng.uniform(0, 10, size=(50000, 5))5print(round(float(draws.max(axis=1).mean()), 3))  # -> 8.333

Fifty thousand groups of five average a max of 8.333, matching $50/6$ . Medians average near 5.0 by symmetry.

Watch Out For

Treating the max like a mean

The max has no CLT at the usual rate and skews hard: it piles near the top with a long left tail. Averaging maxima across folds and reporting symmetric error bars understates downside risk.

Model extremes with $F^n$ or extreme-value theory, not with $\\sigma/\\sqrt{n}$ . Plot the max's histogram before quoting it.

Using iid formulas on dependent data

Time series, retrieval lists, and beam outputs correlate, so $F(x)^n$ overstates how fast the max concentrates. The effective sample size is smaller than $n$ .

Estimate the max's law by block bootstrap or by modeling dependence. If shuffling the data moves the max's histogram, independence fails.

The Quick Version

Sorted draws $X_{(1)}$ to $X_{(n)}$ name the min, median, max, and range.
I.i.d. maximum: $P(X_{(n)} \\leq x) = F(x)^n$ ; minimum mirrors with survival powers.
Five Uniform $[0,10]$ draws average a max of about 8.333, not 5.
The $k$ -th density counts arrangements of below, near, and above.
Extremes skew and need their own theory, never mean-style error bars.
Dependence breaks $F^n$ ; verify with shuffles or block resampling.