Skip to content
AI360Xpert
Core ML
Visual explainer

Early Stopping

Why you shouldn't train for a fixed number of epochs, and how to reliably stop when the model actually reaches its best performance.

Validation loss bottoms out and climbs afterward, meaning the final weights are overfit and worse than earlier ones.
Validation loss bottoms out and climbs afterward, meaning the final weights are overfit and worse than earlier ones.

Train a network for a fixed 30 epochs and check the validation curve: it falls steadily, bottoms out, then climbs as the model starts memorizing the training set. The weights you have at the end of the loop are the worst on the entire curve for anything the model will see in production. Running fewer epochs doesn't fix this either, unless you know exactly which epoch is the turning point in advance.

Track The Best

Check validation loss every epoch, and save a copy of the weights whenever the score is the best seen so far.
Check validation loss every epoch, and save a copy of the weights whenever the score is the best seen so far.

Early stopping is the mechanical procedure for finding that turning point during training itself. It tracks three things: the best validation score seen so far, the weights that produced it, and how many checks have passed since it last improved. Every epoch, if the validation loss beats the best-so-far, you save those weights to disk.

Confirm the Turn

Training keeps running for several more epochs after an apparent minimum to confirm the upward trend isn't just noise.
Training keeps running for several more epochs after an apparent minimum to confirm the upward trend isn't just noise.

Training doesn't halt the instant validation loss ticks upward once; a single worse epoch is often just batch noise. The patience counter exists to ride out that noise. You keep training for a set number of checks — the patience setting, commonly 5 or 10 — to confirm the trend really has turned before committing to stop.

The Restore Step

Stopping without restoring the best checkpoint throws away the benefit; the loop exits with the worse weights.
Stopping without restoring the best checkpoint throws away the benefit; the loop exits with the worse weights.

This is the single most common implementation mistake: stopping the loop correctly, but keeping whatever weights happen to be loaded at that moment. Because patience forces the loop to run past the minimum, the final weights are strictly worse than the ones that triggered the best score. The mechanism only works if you explicitly restore the saved checkpoint.

Where It Breaks

Setting patience too low causes training to halt on a temporary noise spike before the true minimum is reached.
Setting patience too low causes training to halt on a temporary noise spike before the true minimum is reached.

A validation curve is rarely perfectly smooth. If patience is set to 1 or 2, a random batch with slightly higher loss will trigger a halt well before the model has finished learning. There is no universal correct value — it depends on the noise in your metric — but a stop in the first few epochs of an otherwise improving run is a signal to increase patience.

The Quick Version

  • Track the best validation score and save its weights.
  • Wait for a set number of epochs (patience) without improvement before halting.
  • The halt point is always later than the epoch with the actual best score.
  • You must restore the best checkpoint at the end, not keep the final weights.
  • It's regularization by training duration: cheap and reliable.

What to Read Next