Overfitting
A modeling error that occurs when a neural network memorizes the random noise in its training data, resulting in poor generalization to new, unseen data.
Think of It Like This
Like memorizing the exact answers to a specific practice test, but failing the final exam because the numbers in the questions changed.
Overfitting happens when a model is too complex relative to the amount of training data, or when it is trained for too many epochs. The training loss continues to drop, but the validation loss begins to rise. It is combated using techniques like dropout, weight decay, early stopping, and aggressive data augmentation.