SHAP Values
Using cooperative game theory to distribute the exact 'credit' for a model's prediction among the features, ensuring a mathematically fair, perfectly additive explanation.
Why Does This Exist?
In the pursuit of model-explainability, techniques like lime were invented to explain complex models locally. However, LIME has a fatal flaw: it is non-deterministic. If you run LIME twice on the exact same prediction, the random sampling will yield slightly different explanations. Furthermore, LIME's explanations are not additive. If the baseline prediction is 50, and the final prediction is 75, LIME's feature weights will not mathematically sum up to exactly +25.
This lack of mathematical rigor is unacceptable in regulated industries. If you are audited by a financial regulator, you cannot provide an explanation that changes every time you press "run."
SHAP (SHapley Additive exPlanations) exists to solve this. Grounded in Nobel Prize-winning cooperative game theory, SHAP provides a mathematically guaranteed, deterministic, and perfectly additive explanation for any machine learning model.
Think of It Like This
Think of It Like This
Think of a machine learning prediction like a group of three consultants earning a $100,000 bonus for a project.
How do you divide the bonus fairly? If Consultant A worked alone, they might have earned 70,000. If B and C worked together, they might have earned $50,000.
Lloyd Shapley (a mathematician) proved there is only one mathematically fair way to distribute the money based on every possible combination of workers. SHAP values apply this exact same math to machine learning: the "bonus" is the model's final prediction, and the "consultants" are the features.
How It Actually Works
Calculating an exact Shapley value requires training the model on every possible combination of features. If you have 50 features, that is models, which is computationally impossible. Modern SHAP libraries use clever approximation algorithms (like TreeSHAP for random forests) to calculate these values efficiently.
1. The Base Value
SHAP always starts with the Base Value (or Expected Value). This is simply the average prediction of the model across the entire training dataset. If you know absolutely nothing about a user, your best guess for their prediction is the base value. Let's assume the base value is 50.
2. The Marginal Contributions
SHAP then evaluates the specific user (e.g., John Doe). It introduces John Doe's features one by one and calculates how much the prediction changes.
Because features interact (e.g., Age might matter more if Income is high), SHAP calculates the change across all possible orderings of introducing the features. The average of all these marginal contributions becomes the final SHAP value for that feature.
3. The Perfect Sum
The defining characteristic of SHAP is the additive property. The sum of all SHAP values for a prediction, plus the base value, will always equal the final prediction exactly.
- Base Value: 50
- Age (25): +20
- Income (40k): -10
- Debt (0): +15
- Final Prediction: 50 + 20 - 10 + 15 = 75.
This is usually visualized using a Waterfall Plot or a Force Plot, making it incredibly easy to explain a decision to a non-technical stakeholder.
Show Me the Code
The shap Python library is the industry standard for this technique. Here is how you generate SHAP values for a tree-based model.
import shapimport xgboost as xgb
def explain_with_shap(model: xgb.XGBClassifier, X_train, target_row): """ Uses TreeSHAP to explain a single prediction exactly. """ # 1. Initialize the Tree Explainer # TreeSHAP is highly optimized and runs in polynomial time # instead of exponential time. explainer = shap.TreeExplainer(model) # 2. Calculate SHAP values for the specific row shap_values = explainer.shap_values(target_row) # 3. Access the Base Value base_value = explainer.expected_value # The math guarantees this: # base_value + sum(shap_values) == model.predict_proba(target_row) # 4. Visualize it (generates an interactive HTML plot) shap.initjs() return shap.force_plot( base_value, shap_values, target_row )Watch Out For
The Computational Cost
While TreeSHAP is fast for tree-based models, calculating SHAP values for deep neural networks (KernelSHAP or DeepSHAP) is incredibly slow. If you need to generate real-time explanations for millions of API calls per second, SHAP is often too heavy to run in the critical path.
Feature Independence Assumption
Like many explainability tools, standard SHAP approximations assume that features are somewhat independent when it perturbs the data. If your dataset contains impossible combinations (e.g., it tests the model on a "Pregnant" feature being True while the "Biological Sex" feature is Male), the model's reaction to that impossible data will corrupt the SHAP value.
The Quick Version
- LIME is fast but unstable; SHAP is slower but mathematically rigorous and perfectly consistent.
- SHAP treats a prediction like a cooperative game, calculating the exact marginal contribution of every feature.
- The fundamental rule of SHAP:
Base Value + Sum(SHAP Values) = Final Prediction. - This exact additivity makes it the gold standard for explaining high-stakes models in regulated industries (like finance and healthcare).
- Visualizations like Waterfall and Force plots make SHAP values easily digestible for non-technical users.
What to Read Next
model-explainability— A review of the broader XAI landscape and why these tools are legally required.permutation-importance— A faster, albeit less rigorous, way to calculate global feature importance without using game theory.