Nucleus Sampling
A decoding strategy for text generation that truncates the vocabulary to the smallest set of tokens whose cumulative probability exceeds a threshold 'p'.
Think of It Like This
Like letting a chef choose from the top 90% most likely ingredients for a dish, completely ignoring the bizarre 10% outliers.
Also known as top-p sampling, this method dynamically adjusts the number of candidate tokens based on the model's confidence. If the model is highly certain, the 'nucleus' might contain only one or two tokens; if uncertain, it might contain dozens. This strikes an optimal balance between coherence and creativity, preventing bizarre hallucinations.