Parti

Parti scales autoregressive text-to-image to twenty billion parameters, showing that bigger token predictors keep getting better at photorealism.

Why Does This Exist?

Diffusion dominated text-to-image just as autoregressive models conquered language, an uncomfortable split. If next-token prediction scales everywhere else, its weak image showing might reflect tokeniser quality and model size rather than a deep limitation. Somebody had to run the scaling experiment properly.

Parti ran it: up to $20$ billion parameters on ViT-VQGAN tokens with a strong text encoder. Quality climbed steadily with size across photorealism, composition, and text alignment, matching diffusion on many prompts. The tokeniser it depends on descends from VQ-VAE, and the small-scale proof of concept was ImageGPT.

Think of It Like This

A muralist who only paints larger walls

A muralist paints the same scene on a postcard, a door, and a warehouse wall. The bigger walls get more detail, better proportions, fewer mistakes, not because technique changed but because capacity did. Critics who judged postcards declared the muralist hopeless; the warehouse proved them wrong.

Parameters are wall size. Parti's lesson is that token prediction deserved a warehouse before dismissal. It fails where even warehouses cannot reach: tokenisers that quantise away the very detail scale cannot restore.

How It Actually Works

Text passes through a frozen encoder into prefix embeddings. Images become ViT-VQGAN discrete tokens, $1{,}024$ per image, with a friendlier reconstruction tradeoff than early VQ-VAEs. An encoder-decoder transformer predicts image tokens autoregressively conditioned on the text prefix with standard cross-entropy:

L = -\sum_{i=1}^{1024} \log p_\theta(t_i \mid t_{<i}, c_{\text{text}})

Models span $350$ M to $20$ B parameters, trained on billions of pairs. At inference, tokens sample left to right, then the ViT-VQGAN decoder renders pixels. Larger models follow complex multi-object prompts more faithfully and texture surfaces more finely, with gains visible at every scale step.

Worked example

Four model sizes, roughly $350$ M, $750$ M, $3$ B, $20$ B: each step multiplies parameters about $4$ – $6\times$ . On compositional prompts, human preference for the larger model runs consistently above $60$ percent at every jump, compounding across three jumps. A $20$ B run predicting $1{,}024$ tokens at tens of milliseconds per token needs the better part of a minute per image, the autoregressive speed tax that diffusion avoids.

Code

Scale ladder arithmetic, showing the parameter jumps behind the quality curve.

1scales = [0.35, 0.75, 3.0, 20.0]  # billions of parameters2
3
4def jumps(scales: list[float]) -> list[float]:5    """Multiplicative growth between consecutive model sizes."""6    return [round(b / a, 1) for a, b in zip(scales, scales[1:])]7
8
9print(jumps(scales))  # -> [2.1, 4.0, 6.7] each jump buys visible quality

Watch Out For

Crediting scale for tokeniser improvements

Parti upgraded tokens (ViT-VQGAN) and scale together. Ablations matter: some quality gains belong to the kinder tokeniser, and a diffusion model on the same tokens would share them.

Separate the claims when citing Parti: scaling curves for the predictor, reconstruction wins for the tokeniser. Conflating them misleads architecture choice.

Ignoring the inference speed tax

Minutes per image confines huge autoregressive models to offline use. Demos that hide latency mislead product planning about interactivity.

Budget tokens per second before committing: $1{,}024$ sequential steps lose to fifty parallel diffusion steps at any reasonable latency target.

The Quick Version

Parti scales autoregressive text-to-image from $350$ M to $20$ B parameters.
ViT-VQGAN tokens and frozen text encoders feed encoder-decoder transformers.
Size jumps of $2.1\times$ , $4.0\times$ , $6.7\times$ each buy visible quality gains.
Token prediction rivals diffusion on many prompts given enough scale.
Tokeniser and scale contributions differ, and sequential sampling stays slow.