Diffusion Models

What you are looking at

A diffusion model on a 2D scatter plot. The training distribution is a spiral — 400 points on a one-dimensional curve embedded in a plane, which is small enough to compute exactly and structured enough that "did the model get it right" is a distance rather than an opinion.

Image generators work on exactly this principle at a larger scale. If you can slowly destroy a structure by adding noise, and train a network to predict the noise that was added, you can start from pure static and carve new samples out of it. The mathematics is identical whether you are diffusing a million pixels or two coordinates.

The forward process

Drag the timestep and you are running the forward process, $q(x_t|x_0) = \mathcal{N}(\sqrt{\bar\alpha_t}\,x_0,\ (1-\bar\alpha_t)I)$.

It is one formula. Not a loop — the point at timestep 37 is computed directly from the clean point and a single noise draw, in one multiplication. Nothing is learned, and nothing accumulates. That is why the shape snaps back perfectly when you drag the scrubber down again: the formula is trivially invertible when you already have $x_0$ to plug into it.

Watch the Signal share readout as you go. It is $\bar\alpha_t$, the fraction of the variance still carried by the data rather than by noise, and it falls from 1.000 to 0.005 over 50 steps. By the end, noise outweighs signal about 215 to 1.

The beta schedule

The Variance schedule panel shows two curves on one axis, and the gap between them is the whole reason schedules are designed rather than picked.

$\beta$ is the variance added at each individual step. It rises linearly and never gets past 0.2 — every single step is a small perturbation. $\bar\alpha_t = \prod_{i \le t}(1-\beta_i)$ is the running product, and it collapses: already below half at step 19, less than a hundredth of the way to 1 by step 50. Fifty gentle nudges compose into total destruction.

That asymmetry is exactly what makes the reverse direction learnable. Each step is small enough that undoing it is nearly a Gaussian problem, while the whole chain is long enough to reach pure noise.

The denoiser's guess

Here the lab stops describing and starts computing.

At the Denoiser's Guess stage, every arrow runs from a noisy point to a cross marking where the denoiser thinks that point started. The denoiser is not a trained network — it is the Bayes-optimal one for this dataset: treating the 400 training points as a uniform mixture, the exact posterior mean is a softmax over them weighted by $-\lVert x_t - \sqrt{\bar\alpha_t}s_i\rVert^2 / 2(1-\bar\alpha_t)$. That closed form is precisely the function a perfectly trained noise predictor converges to here, so whatever it can and cannot do is a property of the method, not of a training budget.

Two readouts grade it. Recovery error is how far the reconstruction sits from the true $x_0$, and it climbs steadily as noise takes over. Guess alignment is how well the noise it infers matches the noise actually added, and it does something less obvious: it is high at both ends and dips to about 0.42 near step 11.

Both extremes have a cheap answer. Near step 50 the input is almost pure noise, so echoing the input back is nearly correct. Near step 0 there is barely any noise to get wrong. The hard regime is the middle, where signal and noise are comparable — which is why real diffusion training weights its loss towards exactly that band instead of sampling timesteps uniformly.

Generating from noise

At the Generating From Noise stage the lab throws away $x_0$ entirely. 220 fresh Gaussian draws appear, with no clean point behind any of them, and Denoise walks them down to $t=0$ using nothing but the denoiser and the schedule.

Each step samples from the exact posterior $q(x_s|x_t, \hat x_0)$, which mixes the current noisy point with the predicted clean one and then adds fresh noise back in:

$$\mu = \frac{\sqrt{\alpha_{t|s}}(1-\bar\alpha_s)}{1-\bar\alpha_t}x_t + \frac{\sqrt{\bar\alpha_s}\,\beta_{t|s}}{1-\bar\alpha_t}\hat x_0, \qquad \sigma^2 = \frac{\beta_{t|s}(1-\bar\alpha_s)}{1-\bar\alpha_t}$$

That re-injected $\sigma$ looks like a strange thing to do — you are undoing part of your own work — and it is the single most important term in the lab.

Output coverage and Off-shape distance grade the finished result against the spiral. At 50 steps: coverage 1.00, mean distance 0.003. The generator lands on the manifold and covers all of it.

Why the step count is not just a speed knob

Take Reverse steps down and watch both numbers.

Five steps still works — coverage around 0.85, the spiral clearly recognisable. A tenfold saving for very little. Two steps loses half the shape. One step destroys it completely: every sample lands in a blob a fraction of a unit across, and coverage falls below 0.09.

The reason is exact rather than approximate, and you can read it off the formula above. Set $s=0$ and $\bar\alpha_s = 1$: the weight on $x_t$ becomes zero, $\sigma$ becomes zero, and the weight on $\hat x_0$ becomes one. A single reverse step returns the denoiser's prediction and nothing else — and at $t=50$, with 0.5% of the signal left, the posterior over $x_0$ is nearly uniform, so that prediction is close to the average of the training set.

An average is not a sample. The mean of a spiral is not on the spiral. This is the classic "one-shot generation gives you a blurry mess", and here it is not a metaphor for a network's limitations: the denoiser is optimal, and it still cannot do it, because a conditional mean is the wrong object.

Many small steps work because each one keeps $\sigma > 0$. Every step commits to a slightly more specific answer and then re-adds enough uncertainty to keep the other possibilities open — which is what makes the walk a draw from the distribution rather than a slide towards its centre.