Diffusion Models

Why Does This Exist?

Type "a cup of coffee on a wooden table at sunrise" into a text-to-image model and you get that exact scene back: steam, warm light, grain in the wood. No photo of that cup exists anywhere, and nobody drew it. The pixels are new.

Earlier generators each paid a price you can see. A generative adversarial network trains a generator against a discriminator, which samples in one pass but trains unstably, and the generator can collapse onto a handful of outputs. A variational autoencoder trains calmly but hedges, so its samples look blurry next to diffusion output. Diffusion chose the opposite trade: stable supervised training and sharp images, paid for with slow sampling, because generation needs many repeated network passes instead of one.

This page covers that trade at the level you need first: the fixed forward noising, the learned reverse cleanup, and sampling from static. The schedule math lives in forward and reverse processes, the speed fix in diffusion samplers, the compressed-space version in latent diffusion, and prompt steering in classifier-free guidance. The one idea you need first: Gaussian noise, the static this whole method adds and removes. Every pixel is independent randomness with mean zero, and the full picture is in normal distribution.

Think of It Like This

How It Actually Works

Diffusion has two directions and only one of them learns. The forward process destroys an image with fixed math. The reverse process learns to undo a single step of that destruction. Generation runs the learned undo from pure noise.

The forward pass is fixed arithmetic

Write $x_0$ for the clean image and $x_t$ for its noisy version after $t$ steps, where $t$ counts upward to $T$, the total step count. At each step you mix in fresh Gaussian noise $\epsilon$, which is independent static with mean zero and variance one, resampled every step. How much noise enters at step $t$ comes from a fixed plan called the noise schedule: $\beta_t$ is the noise variance at step $t$, a small positive number chosen before training and never learned.

Set $\alpha_t = 1 - \beta_t$, the signal kept at step $t$, and $\bar{\alpha}_t$, the product of $\alpha_1$ through $\alpha_t$, which is the fraction of the original image still alive at step $t$. Then any noisy version follows in one jump:

The left term is surviving image, the right term is accumulated static. Push $t$ all the way to $T$ and the first term dies out, so $x_T$ is pure static. No network touches this direction. That's the point: destroying information on a fixed plan needs no learning.

The network learns one reverse step

The trained object is a denoiser $\epsilon_\theta(x_t, t)$: a neural network with weights $\theta$ that takes a noisy image $x_t$ plus its step number $t$ and guesses the noise $\epsilon$ hiding inside. Training is plain supervised regression with free labels. Take a real photo, pick a random $t$, sample an $\epsilon$, build $x_t$ with the formula above, and the correct answer is the $\epsilon$ you just added. The loss is the squared miss:

Here $\mathbb{E}$ means the average over photos, steps, and noise draws, and $\|\cdot\|^2$ is the squared length of the miss.

Sampling runs the skill backward. Draw $x_T$ from pure static, ask the network for its noise guess, subtract a scaled version of that guess to get $x_{T-1}$, one notch cleaner, and repeat down to $x_0$. The prompt steers every subtraction, which is its own mechanism in classifier-free guidance. Each step moves only a little, so coherence belongs to the whole chain, never to one pass. That's also why sampling is slow, the price named above. When one pass must do, that job belongs to a single-forward generator or to a sampler built for few steps, covered in diffusion samplers.

This rests on one assumption: the schedule must destroy signal gradually enough that each reverse step has structure to grip. A plan tuned for small thumbnails can shred a large detailed photo too fast, and samples come back blurry or broken. Schedules get tuned per dataset and resolution, as forward and reverse processes shows.

Worked example: one noisy pixel

Shrink the image to a single pixel with value $x_0 = 0.80$. Suppose at step $2$ the surviving fraction is $\bar{\alpha}_2 = 0.81$. Then $\sqrt{0.81} = 0.90$ keeps the signal, and $\sqrt{1 - 0.81} = \sqrt{0.19} \approx 0.44$, rounded to two decimals, mixes the noise. Draw $\epsilon = 0.50$:

The pixel drifted from $0.80$ to $0.94$ on $0.22$ of added static. Say the network predicts $\epsilon_\theta = 0.45$. This example's loss is:

A small loss, and note what it measures: noise error, not photo quality. The diagram above carries these same numbers, and the snippet below recomputes them.

Code

The whole forward jump plus the loss, on the worked numbers. No training loop, no library model, only the two formulas above.

1import math2
3
4def forward_closed_form(x0: float, abar: float, eps: float) -> float:5    """One jump to step t: surviving signal plus accumulated noise."""6    return math.sqrt(abar) * x0 + math.sqrt(1.0 - abar) * eps7
8
9def noise_loss(eps: float, eps_pred: float) -> float:10    """Squared miss between the true noise and the network guess."""11    return (eps - eps_pred) ** 212
13
14x0 = 0.8015abar = 0.8116eps = 0.5017xt = forward_closed_form(x0, abar, eps)18print(round(xt, 2))  # -> 0.9419print(round(noise_loss(eps, 0.45), 4))  # -> 0.0025

Watch Out For

The Quick Version

• A diffusion model trains one skill: guessing the noise inside a noisy image at a known step.
• The forward pass is fixed arithmetic with no learned weights, and scheduled noise buries the image until only static remains.
• Training labels are free, because each example's correct answer is the noise the forward pass just added.
• Generation starts from pure static and applies the learned cleanup step by step, with the prompt steering each pass.
• Few-step sampling needs its own sampler, and the noise loss measures denoising error rather than picture quality.

What to Read Next

• Forward and Reverse Diffusion
• Diffusion Samplers
• Latent Diffusion
• Classifier-Free Guidance
• Diffusion Language Models