Visual explainer

Early Stopping

Monitor validation loss, count failed improvement checks, stop at the patience limit and restore a saved checkpoint. The validation winner can still lose on new data.

Training loss can keep falling after validation loss reaches its best observed value.

Early stopping ends training after a monitored score fails to improve for a chosen number of checks. It can avoid spending the whole training budget on a deteriorating validation score. Selecting and restoring an earlier checkpoint is a separate step, and the validation winner need not be best on independent test data.

Training data supplies the weight updates; validation data supplies a selection signal. In this synthetic eight-epoch history, validation loss is 0.80, 0.60, 0.50, 0.52, 0.45, 0.46, 0.47, 0.48. Training loss is 0.75, 0.55, 0.42, 0.35, 0.28, 0.23, 0.19, 0.16. These designed values illustrate a rule, not a measured experiment or a universal curve.

The solid validation trace reaches its lowest observed loss at epoch 5. The dashed training trace keeps falling. Both share a zero-based loss scale, with one validation check after each epoch. Lower loss is better. Epoch labels are one-based: epoch 1 means the first completed training pass.

Keep a Best Snapshot

The saved best score changes only when validation reaches a new minimum.

Maintain a best score, its checkpoint and a wait counter. For now, any strictly lower loss qualifies as an improvement. The running best is 0.80, 0.60, 0.50, 0.50, 0.45, 0.45, 0.45, 0.45. Save the current weights whenever that record improves.

The mint staircase is the running best; the dashed trace is the score actually observed. Epoch 4 is worse than the record, so the saved epoch-3 weights survive. Epoch 5 replaces them. The vertical axis is deliberately zoomed to loss 0.4–0.8, at twice the previous height scale, so small deviations from the record are easier to see. The staircase's vertical segments mark updates to the saved record, not intermediate training states.

Count Failed Improvement Checks

Patience three stops after three consecutive checks without a new best.

Choose patience 3 for this example, with no minimum improvement margin and no baseline or delayed monitoring. Reset the wait counter to zero on an improvement; otherwise increment it. Stop when the counter reaches 3. This follows the basic monitored-score rule documented by Keras EarlyStopping.

After the best score 0.45 at epoch 5, losses 0.46, 0.47 and 0.48 give counters 1, 2 and 3. Training stops after epoch 8. The picture magnifies only epochs 5–8 and uses 2,000 SVG units per loss unit, versus 250 in the first scene; the numeric labels disclose the small loss differences. The lower row counts failed checks, not loss magnitude.

Patience is a tolerance for nonimprovement. Three checks do not prove an upward trend is statistically real, and a decrease from the previous epoch still counts as a failure if it does not beat the saved best.

Restore the Selected Weights

Stopping at epoch eight and restoring epoch five are separate actions.

The timeline reaches the stop at epoch 8, but the lower branch loads the saved epoch-5 checkpoint for use. These shapes encode checkpoint identity, not parameter values or loss. Stopping the update loop alone leaves the last weights active.

Enable the framework's restoration option or explicitly reload the saved snapshot. Keras exposes restore_best_weights; its documented default is false. Saving a copy costs memory or storage and copying work. Restoring model weights also need not restore optimizer state; resuming a run reproducibly may require additional state. Keep the checkpoint associated with the chosen monitoring rule.

Compare the Observed Scores

Restoring epoch five reduces this observed validation loss by three hundredths.

On this same validation history, the last checkpoint scores 0.48 and the saved one scores 0.45: $0.48-0.45=0.03$ . Both bars start at zero on an identical scale. The short bracket is intentionally small, rather than exaggerating the difference with a cropped baseline.

Reloading does not train a new model. It chooses an already saved state with a better observed validation score. The 0.03 difference is not a promised test improvement, and validation computation and checkpoint storage are not free. Keep final evaluation separate from the data used to choose the checkpoint.

An Improvement Resets Waiting

A new best resets the wait counter if training has continued long enough to reach it.

Zoom into epochs 3, 4 and 5, from left to right: losses 0.50, 0.52 and 0.45. The loss trace uses 1,000 SVG units per loss unit; its slope is magnified relative to the opening plot. The two rows below show the alternative wait counters.

Patience 1 stops at epoch 4, after the first failed check. Patience 3 tolerates that check, reaches the new best at epoch 5, and resets its counter from 1 to 0. That run later stops at epoch 8 as calculated above.

A larger patience can allow more learning through fluctuations, at the cost of more updates and checks after progress has ended. There is no universal default that distinguishes noise from overfitting. Choose it alongside validation cadence, the loss's variability and the training budget.

The Future Is Unobserved

Patience one can stop before a later improvement that the run never observes.

The solid path ends at epoch 4 under patience 1. The dashed continuation is what the constructed history would contain if training continued to epoch 5; the stopped run never observes it. Restoring the best state available at its stop selects epoch 3, with score 0.50, missing the later illustrative 0.45.

This view keeps the opening loss scale but expands epochs 1–5 across the horizontal axis. It shows the cost of a particular rule on a known teaching history. In a real run, you cannot know from the first failed check whether the next epoch would improve. Neither immediate stopping nor waiting forever resolves that uncertainty.

Require a Meaningful Improvement

A smaller loss can still fail the minimum improvement threshold.

Reset to a saved best score 0.50 and choose an absolute minimum improvement margin, min_delta = 0.02. Under a strict comparison, a candidate qualifies only when $v < 0.50-0.02=0.48$ . Score 0.49 is lower than 0.50 but does not qualify; 0.47 does. They are alternative candidates from the same saved state.

The horizontal ruler is cropped to loss 0.40–0.52, with 4,000 SVG units per loss unit. The upper bracket spans the margin from 0.48 to 0.50. The cross marks a failed candidate; the diamond marks a qualifying candidate. The open ring at 0.48 indicates that equality does not qualify under this exact rule. Keras's monitor comparison implementation uses a strict comparator after applying the margin.

With a positive margin, the stored qualifying best may differ from the smallest raw score ever seen. Specify whether a separate checkpoint saver uses the same rule; inconsistent rules can select different epochs. Floating-point comparisons also matter at an exact boundary.

Count Checks, Name Their Cadence

Checking less often moves a three-check stopping rule to a later epoch.

Consider an explicit loop that increments patience only when a fresh validation result is available. After a saved best at epoch 5, checking every epoch gives failures at 6, 7 and 8. Checking every two epochs from that point gives failures at 7, 9 and 11, so the third failure arrives at 11.

For this constructed continuation, validation losses at epochs 9, 10 and 11 are 0.49, 0.50 and 0.51; none beats 0.45. The two lanes share an epoch axis. Numbers above the markers count failed checks; the terminal ring and diamond mark the respective stops. They do not show loss values.

This is the stated loop's check-based convention, not a promise about every library's cadence semantics. Check the callback contract, when it receives monitored values, its monitoring start and its direction for the chosen metric. For an accuracy metric, improvement normally means larger rather than smaller.

Where It Breaks

The best validation checkpoint need not have the lowest independent test loss.

Keep the observed validation scores for checkpoints 5 and 8: 0.45 and 0.48. Construct a possible independent test outcome of 0.55 and 0.50 respectively. The selected checkpoint 5 is better on validation but worse on test. These additional synthetic scores demonstrate a possibility; they are not measured results or an argument to select checkpoint 8 using test data.

The double ring tracks checkpoint 5 across both rows; diamonds track checkpoint 8. Both loss rulers share a cropped 0.40–0.60 range, at 2,000 SVG units per loss unit. Left is smaller loss. The dashed amber connector follows identity across datasets, not a training update. Its move to the right exposes the failed transfer of the validation ranking.

Repeated validation checks select a model, so their winning score is not an independent assessment. The scikit-learn evaluation guide separates validation for selection from testing for final evaluation. Keep the test set out of checkpoint and patience choices. A small or unrepresentative validation set, selection noise or a changed deployment distribution can defeat the ranking even after careful early stopping. Patience cannot guarantee the global best epoch or future generalization.

The Quick Version

Falling training loss can coexist with worsening validation loss.
Save weights when the monitored score achieves a qualifying best.
Patience counts failed improvement checks under a specified rule.
Stopping and restoring a checkpoint are separate actions.
Validation improvements do not promise equal test improvements.
A new qualifying best resets the wait counter.
Stopped runs cannot observe their possible future improvements.
A minimum margin can reject a smaller raw loss.
Validation cadence changes when a check-based patience limit is reached.
The validation winner can lose on independent test data.