Skip to content
AI360Xpert
Core ML
Visual explainer

Overfitting vs Underfitting

The gap between training and test error, diagnosed from learning curves.

Training error vs test error: why they diverge
Training error vs test error: why they diverge

A model's goal is to perform well on new data, not just the data it was trained on. As complexity increases, the generalisation gap emerges.

Underfitting

Underfitting: model too simple, misses the pattern
Underfitting: model too simple, misses the pattern

When a model is too simple, it has high bias. It fails to capture the underlying pattern of the data entirely.

Overfitting

Overfitting: model memorises noise, fails on new data
Overfitting: model memorises noise, fails on new data

When a model is too complex, it has high variance. It threads every point, memorising noise and failing to generalise.

Learning Curves

Learning curves diagnose which problem you have
Learning curves diagnose which problem you have

Plotting error over time or complexity reveals the diagnosis. High bias shows converging high errors; high variance shows a large gap.

Where It Breaks

The failure: more data fixes variance, not bias
The failure: more data fixes variance, not bias

If your model is underfitting, adding more data will not help. You must increase model complexity first.

The Quick Version

  • Generalisation: Models must work on unseen data.
  • Underfit: Too simple to learn the pattern.
  • Overfit: Too complex, memorises the noise.
  • Diagnosis: Learning curves reveal the issue.
  • Failure: More data only fixes variance, not bias.

What to Read Next