Skip to content
AI360XpertBeta

Diffusion Samplers

A trained diffusion model knows which way is cleaner from any noisy image. The sampler is the walking plan: how big each stride is, and how many you take before you call the image done.

One trained denoiser reaches the image by two routes: a thousand tiny DDPM steps or a few long DDIM strides
One trained denoiser reaches the image by two routes: a thousand tiny DDPM steps or a few long DDIM strides

Why Does This Exist?

The original DDPM recipe has a painful bill attached. Training corrupts images across 1000 noise levels, and generation walks the whole chain backwards: one full network call per level, each step waiting on the last. That is a thousand sequential calls for a single image, where a GAN needs one pass. The quality was worth it. The queue was not.

Samplers exist because that bill is optional. Look at what the trained network really is: a function that takes a noisy image plus a noise level and returns predicted noise. Nothing in that function says you must visit every level you trained on. A sampler is the inference walking plan laid over the same network: which levels to actually visit, how far to stride between them, and what arithmetic turns each prediction into the next image.

Scope, so you read the right page: the DDPM page owns the original thousand-step chain, and forward and reverse diffusion owns the noise schedule plus what the network predicts. This page owns only the walking plan.

The one idea you need first: at any level the network predicts the noise in the image, not the clean image itself. Every sampler below spends its network calls asking that one question, then differs in how far it trusts each answer. The full treatment of the question is in forward and reverse diffusion.

Think of It Like This

Crossing a stream in fog, with a friend calling directions

You must cross a wide stream in thick fog, hopping between stones you can barely see. On the bank stands a friend who has memorised the streambed. You shout your position, they shout back which way is shallower, you hop.

Hopping to every pebble is DDPM: a thousand safe, tiny, slow hops. Then you notice the middle stretch runs straight, so you take long confident strides and ask for directions far less often. That is DDIM. Then you get cleverer still: before each stride you take half a step, ask how the current pulls there, and aim your stride along the bend instead of straight. That is a higher order solver.

Here is where the analogy stops. Real stones stay where they are. Here every hop rewrites the stream, and your friend only practised on stones like the training ones. Stride clean past everything familiar and the directions go bad, which is exactly what oversized sampler steps do.

How It Actually Works

The move that unlocked everything was reading the trained network as a continuous equation instead of a chain. Fix the network weights, let the noise level vary smoothly, and the denoising path becomes a differential equation: at each point the network says which direction is cleaner, and generating means walking that direction field from pure noise to an image. Deterministic samplers walk the ordinary version of that equation. Samplers that keep randomness walk the stochastic version. Once you see it that way, a century of numerical solver research applies directly, and skipping levels is just taking longer integration steps.

Three symbols carry the whole section. The noisy image at level tt is xtx_t, plain pixels plus noise. The signal fraction αt\alpha_t (alpha) is how much of the original image survives at level tt: near 1 means almost clean, near 0 means almost pure noise. The predicted noise ϵ\epsilon (epsilon) is what the network outputs when shown xtx_t. Every update below spends one network call to get ϵ\epsilon, then strides.

DDIM strides past the levels it skips

DDIM showed the training chain did not have to be a strict chain at inference time. Training stays Markovian, level by level, but generation is allowed to jump: visit levels 1000, 980, 960 and so on down to 0, spending 50 calls where DDPM spent 1000. Each stride does two things with its single prediction. First it estimates the clean image the prediction implies. Then it re-noises that estimate to exactly the next visited level and lands there.

The deterministic DDIM stride, with αt\alpha_t the signal fraction at the current level and αt−1\alpha_{t-1} at the next visited one, is:

x^0=xt−1−αt ϵαt,xt−1=αt−1 x^0+1−αt−1 ϵ\hat{x}_0 = \frac{x_t - \sqrt{1 - \alpha_t}\,\epsilon}{\sqrt{\alpha_t}}, \qquad x_{t-1} = \sqrt{\alpha_{t-1}}\,\hat{x}_0 + \sqrt{1 - \alpha_{t-1}}\,\epsilon

In words: x^0\hat{x}_0 is the clean image the current prediction points at, and the landing point mixes that aim with the predicted noise in exactly the proportion the next level requires. Same seed, same image, every time: there is no randomness left except your starting noise. A knob called η\eta (eta) controls how much fresh noise each stride re-adds. At η=0\eta = 0 you get the deterministic stride above. Turn it up and the sampler behaves more like the original stochastic chain, which brings back variety at the price of reproducibility.

Euler walks straight, Heun looks before it leaps

Euler's method is the simplest stride: step straight along the network's predicted direction. On gently curving stretches that is plenty. Where the true path bends hard, straight strides overshoot, and the symptom is unmistakable: blurred detail and waxy textures that sharpen as you add steps.

Heun's method fixes the bend by spending two calls per stride. It predicts the direction, takes a trial half step, asks the network again at the trial point, and averages the two directions into one curved stride. The DPM-Solver family is the same idea tuned specifically for the diffusion equation rather than equations in general, which is how it gets recognisable images out of roughly ten to twenty calls.

Count network calls, not steps. A 25 stride Heun run makes 50 network calls, and a 50 stride Euler run makes 50 too. They cost the same. Whenever two samplers are compared by step count alone, someone is being flattered. The honest unit is evaluations of the network, and wall clock follows that, not the stride count.

Deterministic or ancestral

Sampler names ending in "a", like Euler a, are ancestral: every stride mixes in fresh random noise before denoising again. Deterministic samplers converge: same seed, same pixels. Ancestral samplers wander: the injected noise washes out banding and plastic smoothness and leaves more texture, but the same seed never twice gives the same image.

Pick by what you want from the run. You want the deterministic one when you are comparing prompts, measuring quality, or regenerating a keeper. You want the ancestral one when one more varied batch matters more than exact repeatability. Neither is sharper in general. They fail differently, which the pitfalls below make concrete.

Worked example: one DDIM stride, by hand

Take a single number standing in for one pixel channel. Current value xt=1.00x_t = 1.00. The signal fractions are αt=0.64\alpha_t = 0.64 at this level and αt−1=0.81\alpha_{t-1} = 0.81 at the next visited level, so αt=0.80\sqrt{\alpha_t} = 0.80, 1−αt=0.60\sqrt{1 - \alpha_t} = 0.60, αt−1=0.90\sqrt{\alpha_{t-1}} = 0.90, and 1−αt−1=0.19≈0.4359\sqrt{1 - \alpha_{t-1}} = \sqrt{0.19} \approx 0.4359. The network predicts noise ϵ=−0.30\epsilon = -0.30.

First, the clean image this prediction implies:

x^0=1.00−0.60×(−0.30)0.80=1.180.80=1.475\hat{x}_0 = \frac{1.00 - 0.60 \times (-0.30)}{0.80} = \frac{1.18}{0.80} = 1.475

Then the landing point at the next level:

xt−1=0.90×1.475+0.4359×(−0.30)=1.3275−0.1308≈1.20x_{t-1} = 0.90 \times 1.475 + 0.4359 \times (-0.30) = 1.3275 - 0.1308 \approx 1.20

One stride moved 1.00 to about 1.20, partway toward the aimed 1.475, which is correct: the landing point still carries the next level's noise, so it must sit short of the clean estimate. The diagram above draws exactly these three numbers: aim diamond at 1.475, landing dot at 1.20.

The timetable arithmetic is the other half of the example. Visiting every 20th level of a 1000 level chain, 1000 down to 20, costs 50 calls instead of 1000. That is twenty times fewer evaluations for the same trained weights, and it is the whole economic argument for samplers in one division.

Code

A strided timetable plus one deterministic DDIM stride, matching the worked numbers above. Shorten num_calls and watch the gaps widen: that widening is the entire speed-quality trade-off.

import numpy as np

def ddim_stride(x_t: float, eps: float, alpha_t: float, alpha_prev: float) -> tuple[float, float]:    x0_pred = (x_t - np.sqrt(1.0 - alpha_t) * eps) / np.sqrt(alpha_t)  # clean image this step aims at    x_prev = np.sqrt(alpha_prev) * x0_pred + np.sqrt(1.0 - alpha_prev) * eps  # landing at next level    return float(x0_pred), float(x_prev)

def strided_timetable(train_steps: int, num_calls: int) -> list[int]:    stride = train_steps // num_calls  # even spacing across the trained levels    return list(range(train_steps, 0, -stride))

aim, landed = ddim_stride(1.00, -0.30, 0.64, 0.81)print(round(aim, 4))  # -> 1.475print(round(landed, 4))  # -> 1.1967print(strided_timetable(1000, 50)[:3])  # -> [1000, 980, 960]

The prints agree with the hand computation: aim 1.475, landing about 1.20, timetable opening 1000, 980, 960. Change num_calls to 10 and the timetable jumps by hundreds per stride, which is where curvature error starts eating detail.

Watch Out For

Comparing samplers by steps instead of network calls

A second order sampler looks twice as fast on a step-count chart and arrives at the same wall clock. Heun at 20 strides makes 40 network evaluations, so Euler at 40 strides costs the same and sometimes wins. The symptom is a benchmark that crowns a solver which feels no faster in your own runs.

Fix it by quoting evaluations, not strides, and timing the loop. If a config halves your strides but doubles calls per stride, you changed the stride shape, not the budget.

Strong guidance shattering fast solvers

Classifier-free guidance pushes each prediction harder toward the prompt, and higher order solvers amplify that push. Past middling guidance strengths the combination overshoots: neon saturation, crunchy contrast, warped faces. The symptom looks like a bad model, but the same prompt at the same seed renders fine on Euler or DDIM.

Fix it from either end. Drop the guidance strength until the solver behaves, or keep your strength and fall back to a first order sampler. As a rule, the stronger the guidance, the simpler the solver you can afford.

The Quick Version

  • A sampler is the inference walking plan for a trained diffusion model: which noise levels to visit and what arithmetic turns each noise prediction into the next image.
  • DDIM proved generation can skip levels, so 20 to 50 network calls do the work of the original 1000-step chain with the same weights.
  • Deterministic samplers give the same image from the same seed. Ancestral ones mix in fresh noise each stride for texture and variety you can never exactly repeat.
  • Second-order solvers spend two network calls per stride to follow bends, so compare samplers by evaluations, not strides.
  • Few strides trade fine detail for speed, and strong prompt guidance can shatter fast solvers first. When quality matters most, add strides before blaming the model.
  • Distilling the whole walk into one or two passes is a different technique: consistency models.