Posterior Consistency

Posterior consistency means the Bayesian update eventually piles nearly all belief on the truth as data grows, which separates methods that learn from ones that stall.

A wide prior narrows onto the true value as ten, one hundred, and one thousand samples arrive

Why Does This Exist?

Bayesian methods promise learning from data, but a bad prior or a misspecified model can refuse to learn: the posterior camps on the wrong value forever while data piles up. Users need to know which setups guarantee eventual truth and how fast belief tightens. Early theory gave conflicting answers because conditions hid in prior tails.

Consistency theorems name the price: prior mass near truth plus tests that separate it from impostors. Contraction rates then quantify speed. This page states Schwartz's conditions and computes the classic coin example end to end.

The one idea you need first

The posterior is proportional to prior times likelihood. Concentration means posterior mass on every fixed neighborhood of truth tends to one as $n \to \infty$ . A rate says how fast neighborhoods may shrink while still capturing mass, like $1/\sqrt{n}$ . Bayesian inference covers the update itself.

Think of It Like This

A searchlight tightening on a lost hiker

Night rescuers sweep a wide beam across a valley (the prior). Each shouted reply (data point) narrows the beam toward the voice. With a working radio and open terrain, the beam ends on the hiker. With a dead battery (prior gap at truth) or echoes off cliffs (no separating tests), it locks onto a reflection forever.

Prior support is a charged battery, tests are open terrain without echoes, and the beam width is posterior spread. Where the analogy stops: real posteriors can concentrate on predictive twins of truth rather than truth itself under misspecification.

How It Actually Works

Schwartz's theorem needs two ingredients at true density $p_0$ : the prior charges every Kullback-Leibler neighborhood of $p_0$ (prior support), and uniformly consistent tests separate $p_0$ from distant alternatives (testability). Then the posterior of any fixed weak neighborhood of $p_0$ tends to one almost surely.

Rates refine this: in smooth parametric models with $n$ samples, the posterior contracts at $1/\sqrt{n}$ around truth, matching maximum likelihood. In nonparametric density estimation with Dirichlet priors, rates slow to powers like $n^{-1/3}$ depending on smoothness. Misspecification redirects concentration to the KL-closest model point instead.

Worked example

Coin with true heads chance $0.6$ , uniform Beta(1,1) prior, 100 flips with 62 heads. Posterior is Beta( $1+62$ , $1+38$ ) = Beta(63, 39). Mean $63/102 \approx 0.6176$ . Variance $63 \times 39/(102^2 \times 103) \approx 2457/1071612 \approx 0.002293$ , so sd $\approx 0.0479$ .

Mass on $[0.5, 0.7]$ : $z$ -scores $(0.5-0.6176)/0.0479 \approx -2.46$ and $(0.7-0.6176)/0.0479 \approx 1.72$ , giving about 0.951 of mass. With 10 flips (6 heads) the same interval held only about 0.62. Tenfold data visibly tightened belief onto truth.

Code

1from math import erf, sqrt2
3
4def normal_cdf(z: float) -> float:5    return 0.5 * (1 + erf(z / sqrt(2)))  # standard normal tail6
7
8mean, sd = 63 / 102, 0.04799mass = normal_cdf((0.7 - mean) / sd) - normal_cdf((0.5 - mean) / sd)10print(round(mass, 3))  # -> 0.951

The normal approximation gives 0.951 on the interval, matching the hand $z$ -scores. Exact Beta integration agrees to two decimals.

Watch Out For

Zero prior mass at truth

A prior that rules out the true value (a point mass elsewhere, or a support gap) never recovers no matter the data, because Bayes multiplies by zero. The symptom is a posterior stuck far from the maximum likelihood estimate with growing confidence.

Check the prior charges neighborhoods of plausible truths before trusting asymptotics.

Quoting consistency under misspecification

With a wrong model family, the posterior concentrates on the KL projection, which can sit far from any substantive truth. Predictions may still score well while parameters mislead. Teams that read consistent parameter recovery off a misspecified run overclaim.

Separate predictive consistency from parameter consistency, and test expansion of the model family.

The Quick Version

Consistency means posterior mass on truth tends to one as $n$ grows.
Schwartz needs prior KL support plus separating tests.
Smooth models contract at $1/\sqrt{n}$ ; nonparametric rates are slower powers.
The coin run concentrates 0.95 mass on $[0.5,0.7]$ after 100 flips versus 0.62 after 10.
Zero prior mass or misspecification breaks the guarantee, so check both first.