Mode Collapse

Mode collapse is when a generator discovers one image that fools the critic and then prints variations of it forever, abandoning the rest of the data.

A collapsed generator covers two of ten data modes, so its samples cluster while eight modes go unrepresented

Why Does This Exist?

GAN training rewards fooling, not covering. If one sharp face earns high critic scores, the generator's gradient points at making more of that face, not at exploring unwed faces. Coverage is nobody's objective, so it is the first thing training sacrifices.

The damage is silent because samples look good. A collapsed face generator prints gallery-worthy portraits, all siblings of one another. Only distribution metrics reveal the theft: recall craters while precision stays high. The metric that exposes it is GAN evaluation, and the loss redesign that attacks its cause is covered by the Wasserstein GAN owner page.

Think of It Like This

A student who memorises one exam answer

A student discovers that one particular essay earns full marks from a tired examiner. They submit that essay, lightly reworded, for every question: history, biology, mathematics. Each submission scores well in isolation, and the examiner, seeing only one paper at a time, never notices every answer is the same.

The generator is the student and the critic is the examiner grading papers one by one. The fix is batch inspection: lay fifty answers side by side and the repetition screams. That is exactly what minibatch-aware critics do, and it fails when the repetition hides inside plausible variation.

How It Actually Works

Write the data as $10$ modes, clusters of similar images. A healthy generator places mass on all $10$ ; a collapsed one places mass on $2$ . The critic still sees real samples from all $10$ but fakes from only $2$ , so it learns perfect rejection everywhere except near the covered modes, and the generator, receiving gradient only there, digs in deeper. The loop is self-reinforcing: less coverage, sharper local scores, still less coverage.

Collapse comes in degrees. Partial collapse drops some modes but varies within the rest. Complete collapse emits near-identical images, detectable by near-zero pairwise distances in a sample batch. Training dynamics oscillate too: the generator may hop between single modes across epochs, covering each in turn but never jointly.

Worked example

Ten equally likely face modes, generator covering $2$ . Recall is $2/10 = 0.2$ while precision can sit near $1.0$ , the classic split. A batch of $64$ samples from the collapsed model contains roughly $32$ near-copies of each covered mode; mean pairwise LPIPS distance drops from a healthy $0.45$ to $0.08$ . Minibatch standard deviation, the ProGAN stabiliser, feeds exactly this batch statistic to the critic so repetition becomes directly visible in the loss.

Code

Pairwise batch diversity, the five-line collapse detector.

1def batch_diversity(samples: list[list[float]]) -> float:2    """Mean pairwise distance inside a batch; near zero means collapse."""3    total, pairs = 0.0, 04    for i in range(len(samples)):5        for j in range(len(samples)):6            if i == j:7                continue  # exclude self pairs8            total += sum((a - b) ** 2 for a, b in zip(samples[i], samples[j])) ** 0.59            pairs += 110    return total / pairs11
12
13print(round(batch_diversity([[0.0], [1.0], [2.0], [3.0]]), 2))  # -> 1.6714print(round(batch_diversity([[1.0], [1.01], [0.99], [1.0]]), 2))  # -> 0.01

Watch Out For

Celebrating sharp samples without measuring coverage

Demo grids are selected, not sampled. A collapsed model yields forty stunning portraits that all share one jawline, and viewers conclude training succeeded. Every GAN release needs recall-style numbers beside the gallery.

Sample uncurated batches of at least fifty and compute pairwise distances. If the gallery looks better than random batches, selection is hiding collapse.

Mistaking mode hopping for recovery

Watching samples change across epochs feels like progress: epoch ten prints blond faces, epoch twenty brunettes. But joint coverage never exceeds one mode at a time, and FID stays poor throughout.

Track distribution metrics across epochs, not sample appearance. Hopping is collapse in motion, not recovery.

The Quick Version

Mode collapse means the generator covers a few data modes and ignores the rest.
The critic grading single samples cannot punish missing variety, so the loop self-reinforces.
Covering $2$ of $10$ modes gives recall $0.2$ with precision near $1.0$ .
Minibatch statistics make repetition visible to the critic directly.
Uncurated batch distances expose collapse; hopping between modes is not recovery.