Skip to content
AI360XpertBeta
Glossary
Definition

Statistical Model Comparison

A test comparing two models scored on the same rows by using only the disagreements, since agreement carries no information about which model is better.

Think of It Like This

Like a taste test where you only compare the dishes where two judges disagreed, ignoring all the meals they both liked.

Two models scored on the same test set produce paired data, not two independent samples, and treating it as unpaired discards the pairing and estimates the wrong variance. McNemar's test looks only at rows where the models disagree and asks whether that split is far from a coin flip.

The Wilcoxon signed-rank test is the paired equivalent for continuous per-fold scores rather than binary right-or-wrong outcomes.