Train-Test Split
Dividing data so a model is fitted on one part and measured on another it has never seen.
Think of It Like This
Like keeping half of the textbook practice problems locked in the teacher's desk so students can't just memorize the answers before the exam.
Three parts rather than two, in practice. Train fits the parameters, validation makes every choice you tune — hyperparameters, features, thresholds, when to stop — and test estimates the chosen thing once, on rows that never informed a decision.
The one-test-set rule is measurement rather than etiquette. Try twenty variants against one 1,000-row test set and the winner carries roughly 1.8 percentage points of pure luck, because you selected the maximum of twenty noisy draws.
A random split assumes any row is as likely to be a future row as any other, and three things break that: time needs a chronological cut, repeated entities need grouped splitting, and rare classes need stratifying. Split before exploring and before fitting any transform, and record the seed.