Gen AI

Repetition Penalties

LLMs can easily get trapped in infinite loops of repeating the same phrase. Repetition penalties mathematically tax the probability of words that have already appeared, forcing the model to generate new vocabulary.

Why Does This Exist?

By their nature, autoregressive transformers are prone to getting stuck in infinite loops. If an LLM generates the phrase "the quick brown fox", and then accidentally generates "the quick brown" again a few sentences later, the self-attention mechanism sees a massive, highly-correlated pattern in the context window. The model thinks, "Ah, I know exactly what comes next!" and generates "fox". The pattern reinforces itself, becoming mathematically stronger with every iteration, until the model is generating "the quick brown fox" 5,000 times in a row.

To break these degenerate loops, researchers introduced repetition penalties (often split in modern APIs into Frequency Penalty and Presence Penalty). These are manual, mathematical interventions applied to the raw logits before the softmax function, intentionally suppressing the probability of words the model has already used.

Think of It Like This

A radio DJ's playlist rules

Imagine a radio DJ who naturally wants to play the most popular song right now (the highest probability token). If left to their own devices, they might play the exact same Taylor Swift song every 10 minutes because it's always the mathematically "best" choice for ratings.

A Presence Penalty is a strict station rule: "If you have played a song at all today, you lose 5 points of desire to play it again." It guarantees you will eventually hear different artists. A Frequency Penalty is an escalating tax: "Every single time you play a song, you lose 1 point of desire to play it." The first replay hurts a little. The 10th replay is impossible. Eventually, the DJ is forced to dig into their back catalog to find fresh music.

How It Actually Works

Repetition penalties are applied to the raw logits ( $z$ ) immediately after the forward pass, before temperature or the softmax function.

Presence Penalty

This is a flat tax. If a token appears at least once in the generated text (or the prompt, depending on the implementation), a constant value is subtracted from its logit. $z_i = z_i - \text{presence\_penalty}$ This is fantastic for forcing the model to cover new topics and introduce novel vocabulary, as it makes reusing any word slightly harder.

Frequency Penalty

This is an escalating tax. The engine counts exactly how many times token $i$ has appeared in the sequence ( $C_i$ ). The penalty scales linearly with that count. $z_i = z_i - (\text{frequency\_penalty} \times C_i)$ This is a hard counter to infinite loops. If the model says "fox" once, it pays a small tax. If it gets stuck in a loop and says "fox" 20 times, the tax becomes massive, crushing the logit into a negative abyss and forcing the model to break the loop.

The legacy multiplicative penalty

Early papers (like CTRL) used a multiplicative penalty ( $z_i / 1.2$ ) rather than subtraction. This is largely abandoned today because logits can be negative. Dividing a negative logit by 1.2 actually makes the number larger (closer to zero), inadvertently rewarding the model for repeating terrible words. Subtraction is mathematically safer.

Show Me the Code

You can simulate how an escalating frequency penalty dynamically alters the model's top choice as a word gets repeated.

1import numpy as np2
3def apply_frequency_penalty(logits: np.ndarray, token_counts: np.ndarray, penalty: float) -> np.ndarray:4    # Subtract (count * penalty) from the raw logits5    return logits - (token_counts * penalty)6
7# Vocabulary: ["cat", "dog", "fox"]8raw_logits = np.array([5.0, 4.5, 4.0])9
10# Scenario 1: Nothing has been generated yet11counts_start = np.array([0, 0, 0])12penalized_start = apply_frequency_penalty(raw_logits, counts_start, penalty=1.0)13print(f"Start  - Top choice index: {np.argmax(penalized_start)} (cat)")14
15# Scenario 2: The model has generated "cat" twice16counts_loop = np.array([2, 0, 0])17penalized_loop = apply_frequency_penalty(raw_logits, counts_loop, penalty=1.0)18print(f"Loop 1 - Top choice index: {np.argmax(penalized_loop)} (dog)")19
20# Notice what happened to the math:21# "cat" logit: 5.0 - (2 * 1.0) = 3.022# "dog" logit: 4.5 - (0 * 1.0) = 4.5  <-- Dog takes the lead!

By manually subtracting from the logit, the frequency penalty artificially dethroned "cat", forcing the model to generate "dog" instead and breaking the repetitive cycle.

Watch Out For

Destroying code and formatting

Applying high repetition penalties is disastrous for code generation or structured data (like JSON or Markdown). Code requires repetition. If you apply a presence penalty to a JSON generation task, the model will use the " and { characters once, get penalized, and then hallucinate bizarre unicode symbols to avoid reusing the quotation marks, completely breaking the JSON syntax. Always turn penalties off for structured data.

The Quick Version

Autoregressive models are naturally prone to getting stuck in infinite, degenerate loops of repeating text.
Repetition penalties intervene by artificially lowering the raw logit score of tokens that have already been used.
A Presence Penalty applies a flat, one-time tax to any token that has appeared at least once, encouraging new vocabulary.
A Frequency Penalty applies an escalating tax based on exactly how many times a token has appeared, strictly preventing infinite loops.
Penalties should be kept low for creative writing, and turned off completely for code and structured data where repetition is required.