Skip to content
AI360Xpert
Glossary
Definition

Stacking

Training a meta-model on the out-of-fold predictions of several diverse base models, so the combination learns which model to trust rather than a fixed average.

The meta-model's training rows must be out-of-fold predictions: each base model's guess on rows it never trained on, produced the same way a cross-validation score is. Feed it predictions the base models made on their own training rows instead, and it learns to trust whichever one memorised hardest, which flatters cross-validation and does nothing on real data.

Diversity of model family is what makes the extra layer worth its cost — a tree, a linear model, and a distance-based model cover different failure modes, while three near-identical models leave the meta-model with almost nothing to blend.