Feature Importance
The importance your tree library prints for free answers a narrower question than it looks like. Permuting a column on held-out data answers the one you meant.
Why Does This Exist?
A credit model scores loan applications and a regulator asks which features drive the decision. The number you print becomes part of a filing, so the measure behind it matters as much as the ranking.
The trap is that the free importance inside tree libraries answers where split credit fell during training, not what the model depends on now. A customer ID column with thousands of distinct values offers thousands of candidate cut points, so it collects split credit by opportunity. Duplicate income fields split one real signal into two moderate scores. Print that ranking and an ID can outrank income while the model itself leans on income.
This page covers the two measures side by side and when each applies. The honest aggregate measure lives in permutation importance. The per decision split lives in SHAP. The trees that produce the free numbers live in decision trees.
Think of It Like This
Crediting a relay team two ways
A four person relay team wins and a reporter asks who mattered. Ask who held the baton at the tape and you always name the anchor, because the format forces that answer. That credits position, not contribution.
Ask instead how much slower the team would finish with each runner replaced by an average club runner, one at a time. Now you learn what each runner added in a way that survives a rerun with a different order.
Impurity importance is the first question. Permutation importance is the second.
Where it stops: runners tire and conditions change between races. Shuffling a column keeps all other columns fixed, so it misses joint effects where two features only matter together.
How It Actually Works
Call impurity importance MDI, meaning mean decrease in impurity. Training sums how much each winning split reduced Gini impurity across all trees. It costs nothing extra, which is why it ships as the default. Two biases follow from the counting itself. A high cardinality column gets more chances to win splits, so its total inflates with opportunity. Correlated columns share credit by draw order, so each looks weaker than their joint effect.
Permutation importance asks the relay question. Shuffle one column across rows of held-out data, breaking its link to the target while leaving other columns intact, and record the score drop. A relied upon feature causes a real fall. Noise causes almost none. Held-out data is required, for the same reason cross-validation never scores on training rows: an overfit model appears to depend on memorized noise.
Worked example: 100 held-out applications
A forest scores 100 held-out rows and gets 85 right, so baseline accuracy is 0.85. Shuffle the income column and it gets 62 right, so accuracy is 0.62. Shuffle the row ID column and it gets 84 right, so accuracy is 0.84.
Income drop:
Row ID drop:
Income matters at 0.23 while the ID sits at 0.01, a 23 fold gap. MDI on the same forest still credits the ID for its many training splits, which is the diagram contrast: one ranking from split counts, one from held-out drops, with 0.23 against 0.01 deciding the question. Once the question moves to one applicant, the additive per row split lives in SHAP and LIME.
Reading the two together
Use MDI during development as a cheap smoke signal. Use permutation on a held-out set for any ranking that leaves the notebook. When two columns correlate, report them as a group effect rather than crowning one winner.
Code
The point is the check: rebuild both drops from counts and confirm 0.23 against 0.01.
baseline_correct: int = 85income_correct: int = 62id_correct: int = 84n: int = 100
drop_income: float = (baseline_correct - income_correct) / ndrop_id: float = (baseline_correct - id_correct) / n
print(f"{drop_income:.2f} {drop_id:.2f}") # -> 0.23 0.01The printout matches the hand subtraction: income costs 23 points per 100 rows, the ID costs one. Swap the ID column for a second copy of income and both drops fall together, showing the correlated credit split.
Watch Out For
Reading importance as causation
A readmission model can rank prior admissions first, which predicts well but gives no lever to pull.
Symptom: a top feature that follows the outcome in time, like prior claims predicting future claims, gets treated as a cause to intervene on. Fix: report importance as predictive weight only, then test levers with experiments or causal tools instead of reordering operations around the chart.
Filing MDI rankings as settled fact
An MDI chart can place an ID above income or split a real driver across duplicates, and a reader outside modeling takes the order literally.
Symptom: a compliance draft or executive deck quotes the free chart with an ID column near the top. Fix: rerun with permutation importance on a held-out set, note grouped effects for correlates, and keep the MDI chart in the appendix as a training diagnostic.
The Quick Version
- MDI sums impurity removed by each feature across training splits, which is free but favors high cardinality columns and splits credit between correlates.
- Permutation shuffles one column on held-out data and measures the score fall, answering what the model depends on now.
- Our 100 held-out rows drop 0.23 when income is shuffled and 0.01 when the ID is shuffled, so income drives the model.
- Always run permutation on data the model never trained on, or overfit noise looks important.
- Neither measure shows causation, and single row why questions need SHAP values instead.