Memorization
A phenomenon where a model learns to perfectly recall exact examples from its training data rather than generalizing abstract patterns to new data.
Think of It Like This
Like a student who memorizes the exact answers to a practice test but fails the real exam because the questions changed slightly.
While neural networks must memorize some facts, excessive memorization indicates severe overfitting. In Large Language Models, memorization poses major legal and privacy risks if the model verbatim regurgitates copyrighted code, books, or sensitive PII. It is mitigated by early stopping, deduplication, and weight decay.