Visual explainer

Model Evaluation

Why you cannot trust training scores and how train, validation, and test splits keep you honest.

Memorizing scores 100 percent seen and 55 percent unseen.

You fit names instead of patterns and training hits 100 percent. Fresh faces drop you to 55. That gap is not bad luck, it is proof you measured on answers you memorized.

Split Into Three

One thousand rows split 700, 150, and 150.

You cut 1000 rows into 700 for training, 150 for tuning, and 150 locked for the end. Train learns weights, validation picks settings, test reports once. No peeking across lines.

Count The Grid

Forty, ten, twenty, thirty give accuracy 0.70 and F1 0.73.

Your grid reads TP 40, FP 10, FN 20, TN 30. Accuracy is 70 over 100, so 0.70. Precision is 40 over 50, so 0.80, while recall is 40 over 60, so 0.67. F1 lands at 0.73.

Tune Then Lock

Tune on validation and lock test until the end.

You train, check validation, tweak, and repeat. Test stays sealed while you iterate. You open it once for the honest number, then you stop tuning.

Honest Score Wins

A locked test reports 72 percent, not leaked 98.

Run the same model twice on one scale. Reused test claims 98 percent while a locked test reports 72. The lower number is the true one you will meet outside.

Fold When Small

Use five folds when data is scarce and one split when large.

You run five folds when rows are few, so each row gets tested once. You keep one split when rows are plenty since folds cost time. Flip to folds when your validation wobbles.

Where It Breaks

Scaling before you split leaks test clues into training.

You fit the scaler on all rows, then split. Test means leak into training, so scores inflate. Split first, fit only on train, then apply the same shift to test.

The Quick Version

Memorizing hits 100 seen, 55 unseen.
Split 1000 into 700, 150, 150.
Grid 40, 10, 20, 30 gives 0.70.
Precision hits 0.80, recall 0.67.
Lock test until the final check.
Fold five times when data is scarce.
Scale after split or you leak.