Skip to content
AI360XpertBeta

Min-P Sampling

You keep only the tokens that score at least a fixed fraction of the best token. When the leader is strong the bar is high, and when the race is close the bar drops.

Min-p 0.10 on a 0.50 leader sets a 0.05 cutoff that keeps 3 tokens and cuts 3
Min-p 0.10 on a 0.50 leader sets a 0.05 cutoff that keeps 3 tokens and cuts 3

Why Does This Exist?

Every time you ask a model for the next word, it scores its whole vocabulary and turns those scores into probabilities that sum to one. You then have to draw one token, but drawing from the raw list is reckless. Thousands of weak long-shot tokens each hold a sliver of mass, and one of them will eventually come up. So you truncate first and sample second. The full setup lives in next-token prediction.

Top-P sampling is the usual truncation. It sorts tokens from best to worst, adds up probabilities until the running total hits a target such as 0.90, and keeps that head of the list. Top-K sampling does the same job with a fixed head count, and temperature shapes the list before either rule runs.

Top-P has one weak spot, and it matters. A crowd of weak tokens can pool their mass to reach the target. Picture a leader at 0.40 with sixty also-rans at 0.01 each. The total is exactly 1.001.00. Top-P with P=0.90P = 0.90 keeps the leader plus fifty of those 0.01 tokens, because 0.40+50×0.01=0.900.40 + 50 \times 0.01 = 0.90. The kept mass is 0.900.90, and the mediocre share is 0.50/0.90≈0.560.50 / 0.90 \approx 0.56. You now face about a 56 percent chance of drawing a token you never wanted, not because any one of them looked good, but because together they filled the quota.

min-p sampling answers with a relative threshold. It keeps a token only if that token stands close enough to the leader, no matter how many weak tokens wait behind it. This page covers that rule and its worked arithmetic. For the cumulative-mass rule it replaces, read Top-P first. For the shaping steps around it, read temperature and repetition penalties.

Think of It Like This

A sieve that sizes its holes from the biggest stone

You tip a bucket of stones onto a sieve. First you find the biggest stone, then you set the hole size to one-tenth of it. If the biggest stone is a melon-sized rock, the holes are large and only a few big stones stay on top. If the biggest stone is a pebble and everything else is nearly the same size, the holes shrink and almost everything stays.

Min-p works the same way. The biggest stone is the top probability, the hole size is the cutoff, and staying on the sieve means surviving to the final draw.

Where it stops: stones keep their size, but token probabilities are fresh guesses at every step. And sieving finishes the job, while min-p only filters. A dice roll among the survivors still picks the token.

How It Actually Works

You start with one probability per token after temperature, a list that sums to one. Call the probability of token ii by the name pip_i. Call the largest probability in that list pmax⁡p_{\max}, which means the most likely token. Pick a fraction mm, which is the min_p setting such as 0.100.10. The cutoff is cc, defined as c=m×pmax⁡c = m \times p_{\max}. Nothing is sorted. One pass finds the maximum, one pass filters, and you will renormalise what survives.

The cutoff rule

  1. Find pmax⁡p_{\max}, the single largest value in the distribution.
  2. Compute c=m×pmax⁡c = m \times p_{\max}. With m=0.10m = 0.10 and pmax⁡=0.50p_{\max} = 0.50, you get c=0.05c = 0.05.
  3. Keep every token with pi≥cp_i \ge c. Set the rest to zero.
  4. Divide each survivor by the sum of the survivors so the kept probabilities sum to one again, then sample.

Because cc is a fraction of the leader, the bar follows the shape. A dominant leader pushes cc up and the crowd is cut. A flat field pulls cc down and the options survive. You trade diversity for coherence with mm: raise it and answers get safer and duller, lower it and answers get wider and stranger.

Worked example

Take six tokens with probabilities [0.50,0.30,0.12,0.04,0.02,0.02][0.50, 0.30, 0.12, 0.04, 0.02, 0.02]. Check the sum first: 0.50+0.30=0.800.50 + 0.30 = 0.80, plus 0.120.12 is 0.920.92, plus 0.040.04 is 0.960.96, plus 0.020.02 plus 0.020.02 is 1.001.00. Good, this is a real distribution.

Set m=0.10m = 0.10. The leader is pmax⁡=0.50p_{\max} = 0.50, so c=0.10×0.50=0.05c = 0.10 \times 0.50 = 0.05. Keep pi≥0.05p_i \ge 0.05: that is 0.500.50, 0.300.30 and 0.120.12. Cut 0.040.04, 0.020.02 and 0.020.02. Three stay, three go, exactly as the diagram shows.

The survivor mass is 0.50+0.30+0.12=0.920.50 + 0.30 + 0.12 = 0.92. Renormalise by dividing each survivor by 0.920.92:

0.50/0.92≈0.5435,0.30/0.92≈0.3261,0.12/0.92≈0.13040.50 / 0.92 \approx 0.5435, \quad 0.30 / 0.92 \approx 0.3261, \quad 0.12 / 0.92 \approx 0.1304

Those three sum to 1.001.00 up to rounding. Sampling now draws only from the three survivors with those odds.

Why the bar moves with confidence

Now take a flat field where the model is unsure: [0.15,0.15,0.14,0.14,0.14,0.14,0.14][0.15, 0.15, 0.14, 0.14, 0.14, 0.14, 0.14]. The sum is 0.15+0.15=0.300.15 + 0.15 = 0.30 plus five times 0.140.14, which is 0.700.70, for 1.001.00 total. The leader is pmax⁡=0.15p_{\max} = 0.15, so c=0.10×0.15=0.015c = 0.10 \times 0.15 = 0.015. The smallest token is 0.140.14, far above 0.0150.015. All seven survive.

That is the behaviour you want. Don't use min-p when you need wide-open brainstorming and the field is already flat, because even a small mm can still bite when one token edges ahead. And don't trust the leader blindly: min-p assumes the top probability signals real confidence. When the model is confidently wrong, a high leader sets a high bar and locks out the valid alternatives. That is the failure to watch for, and it is why factual answers earn a higher mm while open-ended drafts earn a lower one or none at all.

Code

Use this snippet to check the worked example. It filters the same six probabilities, counts the survivors, and prints the renormalised list.

def min_p_filter(probs: list[float], min_p: float) -> list[float]:    cutoff: float = max(probs) * min_p    kept: list[float] = [p if p >= cutoff else 0.0 for p in probs]    total: float = sum(kept)    return [p / total for p in kept]

probs: list[float] = [0.50, 0.30, 0.12, 0.04, 0.02, 0.02]filtered: list[float] = min_p_filter(probs, 0.10)print(max(probs) * 0.10)  # -> 0.05print(sum(1 for p in filtered if p > 0))  # -> 3print([round(p, 4) for p in filtered])  # -> [0.5435, 0.3261, 0.1304, 0.0, 0.0, 0.0]

The three prints confirm the hand computation: the cutoff is 0.050.05, three tokens survive, and the renormalised survivors match [0.5435,0.3261,0.1304][0.5435, 0.3261, 0.1304] with zeros where the cut tokens were.

Watch Out For

Cranking min-p until creativity dies

You raise min_p to clean up rambling answers, and suddenly every story starts the same way. Raising temperature does nothing. The symptom is a survivor count stuck at one or two even on open-ended prompts: the filter turns sampling into greedy decoding.

The cause is a bar set for factual answers applied to creative ones. With a leader at 0.500.50 and m=0.30m = 0.30, the cutoff is 0.150.15 and only two tokens survive. Lower mm for open work, keep it higher only for short factual answers, and log how many tokens survive per step while you tune.

Filtering and forgetting to renormalise

You zero out the cut tokens in a hand-rolled sampler and the draws look wrong. Cut tokens still appear, or the kept odds do not match your arithmetic. The symptom is a distribution that sums to the survivor mass, such as 0.920.92 instead of 1.001.00.

Never skip the renormalisation. Divide each survivor by the sum of the survivors and assert the result sums to one before you sample. Library samplers do this for you, but your own code must do it explicitly.

The Quick Version

  • Min-p sampling keeps a token only when its probability is at least a fraction mm of the top probability pmax⁡p_{\max}.
  • The cutoff is c=m×pmax⁡c = m \times p_{\max}, so a 0.500.50 leader with m=0.10m = 0.10 sets the bar at 0.050.05.
  • A strong leader raises the bar and cuts the mediocre crowd, while a flat field lowers the bar and keeps the options.
  • Survivors must be renormalised to sum to one before the final draw happens.
  • Min-p assumes the leader signals real confidence, so it locks in mistakes when the model is confidently wrong.