Approximate Bayesian Computation

When the likelihood has no formula you can evaluate, keep the simulated parameters whose fake data land close to the real data.

Simulated datasets scatter around the observed data, and only the parameters whose summaries fall inside the tolerance circle are kept

Why Does This Exist?

Many science simulators (epidemics, galaxy formation, agent economies) generate data freely but offer no evaluable likelihood: you can run the model forward, never score $p(\text{data} \mid \theta)$ . Standard Bayes needs that score, so it stalls. Approximate Bayesian Computation replaces evaluation with comparison: simulate from a proposed $\theta$ , reduce both real and fake data to summaries, and keep $\theta$ when the distance falls under a tolerance $\epsilon$ .

The price is approximation in three places: the summaries discard information, $\epsilon > 0$ admits near-misses, and tiny acceptance rates demand many simulations. This page gives the rejection loop, the tolerance trade-off with numbers, and the two standard upgrades. Likelihood-based posterior sampling is under Bayesian inference; plain Monte Carlo covers sample averages without any likelihood.

The one idea you need first

A summary statistic $s(x)$ compresses data to a few numbers (mean, variance, a count). ABC compares summaries, not raw datasets, because raw high-dimensional distances are always large. Good summaries are near-sufficient: they keep almost everything the data says about $\theta$ .

Think of It Like This

Finding a jam recipe by taste tests

You lost the recipe card but remember the jam's taste. You cannot analyze its chemistry (no likelihood), but you can cook trial batches. Each attempt varies sugar and pectin, a friend tastes blind, and you keep the recipes rated within a point of your memory, discarding the rest.

The friend's score is the distance between summaries; your tolerance is how close counts as a match. Demand an exact match and you cook forever. Accept anything and the kept pile includes chutney. The kept recipes cluster around the lost original exactly when the tasting notes capture what matters (sweetness, set) and the tolerance is tight but reachable.

How It Actually Works

The rejection loop:

Draw $\theta$ from the prior.
Simulate fake data $x_{\text{sim}}$ from the model at $\theta$ .
Compute summaries $s_{\text{sim}}, s_{\text{obs}}$ and distance $d = \lVert s_{\text{sim}} - s_{\text{obs}} \rVert$ .
Keep $\theta$ when $d \le \epsilon$ .

Kept draws approximate the posterior conditioned on $\{d \le \epsilon\}$ . As $\epsilon \to 0$ with sufficient summaries, this tends to the true posterior; as $\epsilon \to \infty$ , it degrades to the prior.

Worked example

Infer a coin bias $\theta$ with prior uniform on $[0,1]$ , observing $7$ heads in $10$ flips. Summary: head count. Tolerance $\epsilon = 1$ keeps simulations with $6$ to $8$ heads. Draw $\theta = 0.5$ : $P(6\text{--}8 \text{ heads}) \approx 0.61$ , usually kept. Draw $\theta = 0.2$ : that probability is about $0.006$ , almost never kept. Over $10{,}000$ draws, roughly $1{,}200$ survive, and their mean sits near $0.65$ with standard deviation about $0.13$ : close to the exact posterior mean $8/12 \approx 0.667$ (Beta(8,4)) while slightly wider from the $\epsilon$ slack. Tighten to $\epsilon = 0$ (exactly $7$ ): about $400$ survive with mean $0.66$ , tighter but three times the simulation cost.

Upgrades

ABC-MCMC proposes local moves instead of fresh prior draws, raising acceptance. ABC-SMC runs a chain of shrinking tolerances, resampling survivors as the next round's proposals. Both keep the same accept rule; they just waste fewer simulations reaching it.

Code

1import random2
3
4def simulate(theta: float, n: int = 10) -> int:5    return sum(1 for _ in range(n) if random.random() < theta)  # head count6
7
8def abc_reject(obs: int, eps: int, draws: int) -> list[float]:9    kept = []10    for _ in range(draws):11        theta = random.random()  # uniform prior on [0, 1]12        if abs(simulate(theta) - obs) <= eps:13            kept.append(theta)14    return kept15
16
17random.seed(7)18kept = abc_reject(7, 1, 10000)19print(len(kept), round(sum(kept) / len(kept), 2))  # -> about 1200 kept, mean about 0.65

About twelve percent survive at $\epsilon = 1$ with mean near the exact $0.667$ ; exact matching keeps roughly four percent.

Watch Out For

Summaries that silently drop the signal

ABC conditions on summaries, so a summary blind to some parameter returns the prior for it no matter how many simulations run. Symptoms: one marginal never moves from the prior while others sharpen. Test summaries on fake data with known $\theta$ first: if the distance cannot tell $\theta = 0.3$ from $0.7$ on simulations, it cannot do it on real data either.

Setting tolerance by impatience

Pushing $\epsilon$ up to raise acceptance trades compute for bias directly: at $\epsilon = 3$ on the coin example the kept mean drifts toward $0.5$ (the prior mean) and the interval covers the truth by luck, not information. Fix a simulation budget first, then pick the smallest $\epsilon$ that yields a few hundred survivors, never the reverse.

The Quick Version

ABC keeps prior draws whose simulated summaries land within $\epsilon$ of observed summaries.
It needs only a simulator, never a likelihood formula.
Three approximations stack: summary choice, tolerance slack, and Monte Carlo error.
On $7/10$ heads with $\epsilon = 1$ , about $1{,}200$ of $10{,}000$ draws survive with mean near $0.65$ against truth $0.667$ .
ABC-MCMC and ABC-SMC cut simulation waste with local moves and shrinking tolerances.
Validate summaries on fake data with known parameters before trusting real-data results.