Skip to content
AI360XpertBeta

SHAP Values

Split one prediction into a credit per feature, averaged over every order the features could arrive in. The credits always add up to the prediction exactly.

A base prediction of 0 plus feature credits of 2 and 3 stacks exactly to the model output of 5
A base prediction of 0 plus feature credits of 2 and 3 stacks exactly to the model output of 5

Why Does This Exist?

A loan model declines an applicant and the applicant asks why. "The model said so" ends the conversation badly, and in regulated lending it can end it unlawfully. Explanations are not decoration here. They are the product surface the decision stands on.

Local surrogate tools answer by fitting a simple stand-in model around the one prediction. That works, but the answer can wobble between runs as the sampling changes, and the feature weights almost never sum to the prediction they claim to explain. If the baseline is 50 and the output is 75, the cited reasons ought to account for exactly 25. Usually they do not.

SHAP, short for SHapley Additive exPlanations, exists to make the arithmetic close. It borrows Shapley values from cooperative game theory, the mathematics of splitting shared winnings fairly, and assigns every feature a credit such that base prediction plus all credits equals the model output, exactly, every time. Deterministic given the model and the background data. Auditable to the last decimal.

The one idea this page stands on is the baseline: the average prediction over a background set, your best guess before seeing the features. This page covers how SHAP walks from that baseline to the final number. The parent topic is model explainability, and the closest sibling method is global feature importance.

Think of It Like This

Splitting a project bonus three ways

Three consultants earn a shared bonus for a project. How do you divide it fairly? Look at every possible team: what each consultant adds alone, what each pair earns together, what all three earn as a trio. Average each person's marginal contribution across every order they could have joined, and you get the one split nobody can call unfair. That averaging-over-every-order is the Shapley value, and game theorists proved it is the only split with that property.

SHAP runs the same procedure on a prediction. The bonus is the gap between the baseline and the model output. The consultants are the features. Each feature's credit is its average marginal contribution across every order the features could arrive in.

The analogy stops where people start. Consultants negotiate, resent the split, and change their effort next quarter. Features do none of that, which makes the arithmetic cleaner than any bonus meeting. But the procedure also assumes every sub-team is meaningful, and features are not always independent the way consultants are. Correlated features break that assumption, which is exactly the second trap below.

How It Actually Works

Credits that sum, by construction

Write the prediction as the baseline plus one credit per feature:

f(x)=E[f]+ϕ1+ϕ2=0+2+3=5f(x) = E[f] + \phi_1 + \phi_2 = 0 + 2 + 3 = 5

In plain words: f(x)f(x) is the model output for this input, E[f]E[f] is the average prediction over the background set, and each ϕ\phi (phi) is one feature's Shapley credit. The equality is not an approximation. Additivity is the defining guarantee: base plus credits always equals the output.

Where one credit comes from

Take a feature and ask what the prediction would be with and without it, then repeat that question inside every possible coalition of the other features. Average the answers over every arrival order. With two features there are two orders, so each credit is the mean of two marginal contributions. With fifty features there are 2502^{50} coalitions, about a quadrillion, which is why exact SHAP stops being computable and approximations take over: fast exact routines for trees, sampling based ones for everything else.

Worked example

Model: f(x)=2x1+3x2f(x) = 2x_1 + 3x_2, applicant at point (1,1)(1, 1), baseline at (0,0)(0, 0) where the output is 0. The gap to explain is 5−0=55 - 0 = 5.

Order one, size first: size adds 2−0=22 - 0 = 2, then income adds 5−2=35 - 2 = 3. Order two, income first: income adds 3−0=33 - 0 = 3, then size adds 5−3=25 - 3 = 2. Average each feature over both orders: size (2+2)/2=2(2 + 2) / 2 = 2, income (3+3)/2=3(3 + 3) / 2 = 3.

Credits 2 and 3, baseline 0, output 5. The diagram above stacks exactly this: 0+2+3=50 + 2 + 3 = 5, with every unit of the gap assigned to somebody.

Watch Out For

Explaining through impossible feature combinations

Standard SHAP perturbs features as if they varied independently, so it will happily evaluate coalitions your data never contains: a teenager with thirty years of credit history, a studio flat with six bedrooms. The model still answers, and its reaction to nonsense leaks into real credits. The symptom is a feature earning large credit mostly through coalitions no applicant could ever belong to.

Check how your top-credited features correlate before quoting them. When two features move together, consider grouping them into one, and treat any credit built on impossible combinations as suspect rather than insightful.

Running exact SHAP where it cannot afford to run

Exact Shapley values grow with the coalitions, so per-prediction explanation on a large model can cost more than the prediction itself. The symptom is an explanation service that lags the product it explains, or a notebook that never finishes the test set.

Explain offline and in aggregate, not on the request path: global importance, representative examples, audit samples. Match the estimator to the model, fast tree routines for trees and sampling for the rest, and never promise real-time credits the arithmetic cannot deliver.

The Quick Version

  • SHAP assigns every feature a credit for one prediction, and base prediction plus all credits always equals the model output.
  • Each credit is the feature's average marginal contribution across every order the features could arrive in.
  • That averaging is the Shapley value, the only split with its fairness property, borrowed from game theory.
  • Exact computation is exponential in feature count, so trees get fast exact routines and everything else gets sampling.
  • Correlated features break the independence the perturbation assumes. Group them or distrust the split.
  • Explain offline and in aggregate. Per-request exact SHAP is a latency trap.