Vector-Quantized Models
A model that replaces an encoder's continuous output with the nearest entry in a fixed, learned codebook, turning a continuous latent space into a sequence of discrete tokens.
Think of It Like This
Like painting a sunset by matching every blended color to the closest shade in a standard 64-color box of crayons.
A codebook of learned vectors replaces each encoder output with its nearest match, producing a discrete grid of codes instead of continuous values. Since the nearest-match lookup isn't differentiable, a straight-through estimator lets gradients reach the encoder as if the lookup were the identity function.
The resulting discrete grid can be modeled autoregressively, the same way a language model predicts the next word, which a continuous latent space can't directly support.