Image Generation

Image generation is the process where models learn the underlying distribution of visual data to synthesize entirely new images from scratch, often guided by prompts.

Generation starts from pure noise at 93.79 and denoises it step by step to 100.0, with a text prompt steering each cleanup pass toward the finished image.

Why Does This Exist?

Editing tools only rearrange pixels a camera already captured: cropping, recoloring, or filtering a photograph. They cannot invent a street that was never photographed or a product shot from a angle that was never staged. Image generation exists to produce new pixels that form coherent images nobody captured, from a sentence, a sketch, or nothing at all.

Studios need endless variations of scenes, robotics teams need rare edge cases for training, and designers need a first draft in seconds instead of days. A generative model meets this by learning the distribution of real images during training, then sampling fresh points from it at request time. The dominant modern recipe is diffusion: destroy images with noise until only static remains, train a network to reverse each destruction step, and run that reversal to grow new images from static.

Think of It Like This

A restorer cleaning a soot-covered fresco

A restorer faces a wall blackened by centuries of soot, with a written description of the scene underneath. She does not repaint the wall freehand: she wipes away one thin layer, checks the emerging shapes against the description, then wipes again, each pass revealing more detail while the description keeps her from inventing the wrong saint. Early passes recover only vague composition; late passes recover eyelashes.

The noisy image is the soot-covered wall, the denoising network is the restorer, and the text prompt is the written description consulted at every pass. Where it stops: the restorer uncovers paint that is really there, while the network invents every pixel, so nothing it produces is evidence that anything was ever real.

How It Actually Works

Training defines a forward process that adds a little Gaussian noise per step, $x_t = \sqrt{1 - \beta_t} \, x_{t-1} + \sqrt{\beta_t} \, \epsilon_t$ , with schedule values like $\beta_1 = 0.04$ and $\beta_2 = 0.08$ . After enough steps the image is pure static. A network, usually a U-Net, is trained on these pairs to predict the noise $\epsilon_t$ that was added at each step, conditioned on the step index and on text embeddings fused through cross-attention.

Sampling runs the reverse: start from pure noise and repeatedly subtract the predicted noise, $x_{t-1} = (x_t - \sqrt{\beta_t} \, \hat{\epsilon}_t) / \sqrt{1 - \beta_t}$ . Early steps settle layout and composition; late steps paint texture and edges. Production models such as Stable Diffusion run this loop in a compressed latent space rather than on raw pixels, the trick taught in latent diffusion. Output quality is scored by distribution metrics covered in FID IS and KID.

Worked example

Take a single-pixel image with clean value $x_0 = 100.0$ and fixed noise draws $\epsilon_1 = 0.5$ , $\epsilon_2 = -1.0$ . Step one uses $\sqrt{0.96} = 0.9798$ and $\sqrt{0.04} = 0.2$ : $x_1 = 0.9798 \cdot 100.0 + 0.2 \cdot 0.5 = 97.98 + 0.10 = 98.08$ . Step two uses $\sqrt{0.92} = 0.9592$ and $\sqrt{0.08} = 0.2828$ : $x_2 = 0.9592 \cdot 98.08 + 0.2828 \cdot (-1.0) = 94.07 - 0.28 = 93.79$ . Two steps drag 100.0 down to 93.79.

Now reverse step two with an imperfect prediction $\hat{\epsilon}_2 = -0.9$ instead of the true $-1.0$ : $\hat{x}_1 = (93.79 - 0.2828 \cdot (-0.9)) / 0.9592 = (93.79 + 0.25) / 0.9592 = 94.05 / 0.9592 = 98.05$ . The recovered 98.05 misses the true 98.08 by 0.03, exactly the residue of the 0.1 prediction error scaled through the division. Every sampling step pays this tax, which is why generation runs dozens of small steps instead of a few large ones.

Code

The point of this snippet is the two-step fixture above, recomputed with only the standard library: forward noise with fixed draws, then one reverse step with a slightly wrong noise prediction.

1import math2
3x0 = 100.04betas = [0.04, 0.08]5eps = [0.5, -1.0]6
7x = x08trace = [x]9for beta, e in zip(betas, eps):10    x = math.sqrt(1 - beta) * x + math.sqrt(beta) * e11    trace.append(x)12print([round(v, 2) for v in trace])13# -> [100.0, 98.08, 93.79]14
15x1_hat = (trace[2] - math.sqrt(betas[1]) * -0.9) / math.sqrt(1 - betas[1])16print(round(x1_hat, 2))17# -> 98.05

The reverse recovers 98.05 against the true 98.08, off by 0.03 from the imperfect noise guess. Real samplers repeat this predict-and-subtract loop with a learned network standing in for the hardcoded -0.9.

Watch Out For

Prompt words landing on the wrong object

Asking for a red cube on a blue sphere can return a blue cube on a red sphere. The symptom is all prompt words present but bound to the wrong regions. Fix: keep prompts compositionally simple, name one relation per clause, and verify binding on a small grid of seeds before scaling up.

Confident anatomy that was never learned

Hands arrive with six fingers and railings melt into staircases. The symptom is photorealistic texture over impossible structure, because the model stores pixel statistics, not a body plan. Fix: zoom into hands, text, and repeating geometry on every candidate, and regenerate or inpaint the failures instead of trusting the first sample.

The Quick Version

Image generation samples new pixels from a learned distribution instead of editing captured ones.
Diffusion trains a network to predict per-step noise, then subtracts predicted noise iteratively from static.
The fixture drags 100.0 to 93.79 forward, and an imperfect reversal recovers 98.05 against the true 98.08.
Text prompts steer every step through cross-attention; early steps set layout, late steps paint detail.
Production models diffuse in latent space, and binding plus anatomy errors demand manual inspection.