Skip to content
AI360XpertBeta
Glossary
Definition

Nested Cross-Validation

Two nested loops: an inner one tunes a hyperparameter using only outer training rows, an outer one scores the chosen setting on rows the tuning never saw.

Think of It Like This

Like taking a practice exam to calibrate your study plan, and then taking a completely separate statewide final exam that you never saw during study prep.

Tuning a hyperparameter and reporting its score against the same fold lets the search overfit the split — the setting that wins is partly the one that flukes that fold's noise, not the one that generalizes best. The gap between a naive and a nested estimate grows with the size of the search and shrinks with the amount of data.

Nested CV costs roughly the outer fold count times an ordinary search, and it buys an honest estimate rather than a model to ship; the deployed model is refit once, tuned by ordinary cross-validation on the full dataset.