Classifier Guidance

Classifier guidance steers a diffusion sampler with gradients from a noisy image classifier, pushing each step toward the requested class.

A noisy classifier's gradient nudges each denoising step toward the target class, with the scale knob setting the push strength

Why Does This Exist?

Unconditional diffusion draws from the whole data distribution at once. Asking it for a golden retriever means hoping one appears, and retraining a conditional model per control is prohibitively expensive. A steering method that bolts onto a frozen model would separate the concerns: one big unconditional model, many cheap controls.

Classifier guidance is that bolt-on. Train a small classifier on noisy images at every noise level, then at each reverse step push the sample uphill on the classifier's log-probability for the target class. No generator retraining, one reusable sampler. The guidance-free successor that avoids the extra classifier lives in the classifier-free guidance owner page.

Think of It Like This

A bloodhound leading a painter

A painter works half-blind, sketching whatever emerges. A bloodhound beside them sniffs each stroke and tugs the brush toward the scent of retriever. A gentle leash lets the painter compose freely with a nudge; a yanked leash drags every stroke onto the same canonical dog pose.

The guidance scale is the leash tension. Slack guidance wanders off-class, and a death grip prints the same prize dog forever. Both failure ends come from one knob, which is why tuning it per use matters more than any architecture choice.

How It Actually Works

At reverse step $t$ with current sample $x_t$ , the unconditional model predicts mean $\mu(x_t)$ . A noise-aware classifier $p_\phi(y \mid x_t)$ scores the target class $y$ . The guided step shifts the mean along the classifier gradient:

\hat{\mu} = \mu(x_t) + s \cdot \Sigma \, \nabla_{x_t} \log p_\phi(y \mid x_t)

$s$ is the guidance scale, $\Sigma$ the step covariance. At $s = 0$ sampling is unconditional; $s = 1$ follows the classifier's natural slope; production values often land near $s = 5$ – $10$ for crisp class features at visible diversity cost. The classifier must train on the same noise schedule, because a clean-image classifier's gradients point meaninglessly on heavy noise.

Worked example

Mid-sampling, the classifier gives target probability $0.7$ with gradient norm $0.4$ per pixel direction, and $\Sigma$ scales steps by $0.02$ . At $s = 1$ the nudge is $0.008$ per step, gentle compositional pressure across fifty steps. At $s = 8$ the nudge is $0.064$ per step, enough to drag marginal samples onto canonical class features within a dozen steps, and enough to collapse variety if every sample takes the same ride.

Code

Guided mean update, the arithmetic inside one reverse step.

1def guided_mean(mu: float, grad: float, sigma: float = 0.02, s: float = 1.0) -> float:2    """Shift the predicted mean along the classifier gradient."""3    return mu + s * sigma * grad4
5
6print(guided_mean(0.5, 0.4, s=1.0))  # -> 0.508 gentle pressure7print(guided_mean(0.5, 0.4, s=8.0))  # -> 0.564 firm steering

Watch Out For

Guiding with a clean-image classifier

A classifier trained on pristine photos has never seen noise, so its gradients on $x_t$ at high noise are arbitrary directions that shred composition. Samples degenerate into adversarial textures that maximise classifier score while looking like static.

Always train or fine-tune the classifier across the diffusion noise schedule. Check gradient sanity:denoise the gradient direction slightly and confirm recognisable class features emerge.

Cranking scale until diversity dies

High $s$ prints the classifier's favourite exemplar repeatedly. Benchmarks scored at $s = 10$ flatter fidelity metrics while coverage collapses, the same single-knob illusion as truncation in BigGAN.

Sweep $s$ and plot fidelity against diversity. Ship the knee of the curve, not the fidelity maximum.

The Quick Version

Classifier guidance adds $s \cdot \Sigma \nabla \log p(y \mid x_t)$ to each denoising mean.
The classifier must train on the same noise schedule as the diffusion model.
Our step moves $0.508$ at $s = 1$ versus $0.564$ at $s = 8$ .
Scale trades class crispness against sample variety on one knob.
Clean-image classifiers and maxed-out scales are the two standard failures.