Partial Dependence and ICE
Importance says a feature matters. Partial dependence shows how it moves the prediction on average, and ICE shows how it moves each individual underneath that average.
Why Does This Exist?
Permutation importance tells you age matters for a credit model. It never tells you how. Does approval rise with age, fall, rise then flatten? A single importance number cannot draw a shape.
Partial dependence and ICE exist to draw it. Freeze every other feature of a real borrower, sweep age from 20 to 60, record the prediction at each step. Do that per borrower and you get ICE lines. Average those lines at each age and you get the partial dependence curve. The broader explainability map, including when a global shape is enough and when a local reason is needed, lives in model explainability. Additive per-prediction attributions live in SHAP, and one local surrogate around a single decision lives in LIME.
Think of It Like This
Testing one medicine on a thousand hearts
Give a new medicine to a thousand people and record each heart-rate change. Average all thousand and you get one number: down 10 beats per minute on average. That average is the PDP.
But the average can hide harm. Suppose healthy volunteers drop 30 beats while volunteers with diabetes rise 10. The single average still reads down, and the danger vanishes into arithmetic. Plot every volunteer's own line on one chart and the rising group jumps out. That spaghetti of individual lines is the ICE plot.
Each volunteer is one borrower, the dose is age, the heart-rate change is the approval chance, and the average over volunteers is the PDP.
Where it stops: bodies share one chemistry plus measured differences, while model inputs can combine impossibly, like a teenager earning half a million. The analogy covers averaging, not the impossible-row trap below.
How It Actually Works
Both are post-hoc and model-agnostic: no retraining, any model, same recipe.
ICE first, one line per row
Pick borrower A with income 60k and debt 10k at age 25. Fix income and debt. Set age to 20, predict, plot. Set age to 21, predict, plot. Continue to 60 and join the dots. That polyline is A's ICE line: exactly how age moves this borrower with everything else frozen. Repeat for every borrower. A thousand borrowers give a thousand thin lines.
PDP second, the average of those lines
At each age, average all ICE values. That average is the partial dependence at that age. Join across ages and you get one thick readable curve: the global effect of age with other features averaged out. Formally, with the swept feature and the frozen rest:
Here is the trained model, is the row count, is age, and is borrower 's frozen rest.
Worked example: three borrowers, one falling line
Give the model three simple approval rules in approval chance:
At age 20: , , . The PDP average is .
At age 40: , , . The average is .
At age 60: , , . The average is .
The PDP climbs to to and looks reassuring. Borrower C falls to to the whole way. The diagram draws exactly these three lines plus the thick average, which is why you never read the average without the lines behind it. When lines fan in opposite directions like this, the average is a compromise no individual follows. The glossary form of the average lives in partial dependence.
Code
The point is the averaging: rebuild the three ages from the worked example and confirm 0.43, 0.53 and 0.63.
def appr_a(age: float) -> float: return 0.10 + 0.01 * age
def appr_b(age: float) -> float: return 0.20 + 0.01 * age
def appr_c(age: float) -> float: return 0.70 - 0.005 * age
for age in (20.0, 40.0, 60.0): avg: float = (appr_a(age) + appr_b(age) + appr_c(age)) / 3.0 print(f"age {age:.0f} avg {avg:.2f}") # -> age 20 avg 0.43 # -> age 40 avg 0.53 # -> age 60 avg 0.63The printout matches the hand computation. Change C to rise with the others and watch the average steepen: agreement between ICE direction and PDP slope is the thing being tested.
Watch Out For
Sweeping age into impossible borrowers
Symptom: an ICE line for a 15-year-old with half a million income and twenty years of credit history. The model predicts there anyway, the line bends sharply at the edge, and a team reads the bend as a real age effect. It is noise on a row that cannot exist.
Fix: rug the axes with real data density and distrust every segment where the rug thins out. Clamp sweeps to observed ranges per subgroup, or condition the average on realistic rows instead of the full grid.
Reading a flat PDP as no effect
Symptom: the PDP runs horizontal, the team concludes age does nothing, and a subgroup complaint arrives a month later. Two ICE groups rose and fell symmetrically, and the average cancelled them. Flat described the compromise, not any borrower.
Fix: always plot ICE behind PDP and check the spread at each age. Wide spread with a flat mean means interaction, not irrelevance. Split by the interacting feature or move to a method that names interactions before shipping the flat line as a finding.
The Quick Version
- ICE draws one line per row: freeze everything else, sweep one feature, record the prediction.
- PDP is the average of those ICE lines at each swept value.
- Our three borrowers read 0.30, 0.40 and 0.60 at age 20, averaging to 0.43.
- At age 60 they read 0.70, 0.80 and 0.40, averaging to 0.63 while borrower C falls throughout.
- A flat PDP with fanned ICE means interaction cancelled in the average, not absence of effect.
- Sweeps invent impossible rows at the edges, so weigh the curve by where data actually sits.