Your gradient-boosted model flags a loan application as high risk. The applicant asks why. The regulator asks why. Your product manager asks why. Accuracy metrics answer none of these questions. SHAP and LIME do, and they answer them in very different ways. One gives you a mathematically grounded attribution. The other gives you a fast local approximation. Choosing wrong costs you either compute time or trust.
Quick Takeaways
- SHAP assigns each feature a Shapley value. It is consistent, additive, and supports both local and global explanations.
- LIME fits a simple surrogate model around one prediction. It is fast and model-agnostic, but its output varies between runs.
- Use SHAP (TreeExplainer) for tree ensembles. It is exact and runs in polynomial time.
- Use LIME for quick, instance-level checks on text, images, or black-box APIs where SHAP is too slow.
| Dimension | SHAP | LIME |
|---|---|---|
| Core idea | Shapley values from cooperative game theory | Local linear surrogate on perturbed samples |
| Scope | Local and global | Local only |
| Stability | Deterministic (Tree/Linear); stable estimates | Varies with random sampling |
| Theoretical guarantees | Efficiency, symmetry, dummy, additivity | None (fidelity is local and approximate) |
| Speed on trees | Very fast (TreeExplainer) | Slow (thousands of model calls) |
| Speed on black boxes | Slow (KernelExplainer) | Moderate (default 5,000 samples) |
| Data types | Tabular, text, images | Tabular, text, images |
| Best for | Audits, feature importance, production monitoring | Rapid debugging, ad hoc explanations |
What Model Explainability Actually Solves
Complex models trade transparency for accuracy. A random forest with 500 trees or a transformer with billions of parameters has no readable decision rule. Post-hoc explainers recover a human-readable story after training.
Two terms matter here:
- Local explanation: why the model made this prediction for this input.
- Global explanation: which features drive the model’s behavior across the whole dataset.
Both SHAP and LIME are model-agnostic at their core. They treat the model as a function f(x) and probe it. SHAP also ships model-specific accelerators that read the model structure directly.
How SHAP Works
SHAP (SHapley Additive exPlanations) borrows from game theory. Treat each feature as a player. The prediction is the payout. A feature’s Shapley value is its average marginal contribution across all possible feature orderings:
φ_i = Σ_{S ⊆ F\{i}} [ |S|! (|F| − |S| − 1)! / |F|! ] · [ f(S ∪ {i}) − f(S) ]
The explanation is additive:
f(x) = φ_0 + φ_1 + φ_2 + ... + φ_M
Here φ_0 is the base value (the mean model output) and each φ_i is a feature’s push up or down. This additivity is the efficiency property. The attributions always sum to the gap between the prediction and the baseline.
SHAP satisfies four axioms: efficiency, symmetry, dummy, and additivity. No other attribution method in this family gets all four.
SHAP Explainer Variants
Exact Shapley values need 2^M model evaluations. That is impractical beyond about 20 features. The library solves this with specialized algorithms:
| Explainer | Model type | Complexity | Exact? |
|---|---|---|---|
| TreeExplainer | XGBoost, LightGBM, CatBoost, random forest | Polynomial in tree size | Yes |
| LinearExplainer | Linear/logistic regression | Closed form | Yes |
| DeepExplainer | Neural networks (DeepLIFT-based) | Backprop-like pass | Approximate |
| KernelExplainer | Any black box | Exponential, sampled | Approximate |
| PermutationExplainer | Any model | Sampled permutations | Approximate |
SHAP in Python
import shap
import xgboost as xgb
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
data = load_breast_cancer(as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
data.data, data.target, test_size=0.2, random_state=42
)
model = xgb.XGBClassifier(
n_estimators=300, max_depth=4, learning_rate=0.05
).fit(X_train, y_train)
explainer = shap.TreeExplainer(model) # exact, fast for trees
sv = explainer(X_test) # Explanation object
shap.plots.waterfall(sv[0]) # local: one prediction
shap.plots.beeswarm(sv) # global: all predictions
shap.plots.bar(sv) # global: mean |SHAP| ranking
The waterfall plot shows how each feature moves one prediction from the base value to the final output. The beeswarm plot shows feature importance and direction of effect across every row.
For classifiers, TreeExplainer outputs values in log-odds space by default. Pass model_output="probability" with a background dataset if you need probability units.
How LIME Works
LIME (Local Interpretable Model-agnostic Explanations) skips the theory and goes empirical. It explains one prediction in four steps:
- Perturb the input to create many nearby synthetic samples (default 5,000 for tabular data).
- Query the black-box model for predictions on each sample.
- Weight each sample by its proximity to the original input using an exponential kernel.
- Fit a sparse, interpretable model (usually ridge regression) on the weighted samples.
The optimization looks like this:
ξ(x) = argmin_{g ∈ G} L(f, g, π_x) + Ω(g)
L measures how poorly the surrogate g mimics f near x. π_x is the proximity kernel. Ω(g) penalizes complexity, which keeps the explanation short. The surrogate’s coefficients become the feature weights.
LIME in Python
from lime.lime_tabular import LimeTabularExplainer
lime_explainer = LimeTabularExplainer(
X_train.values,
feature_names=list(X_train.columns),
class_names=data.target_names,
mode="classification",
discretize_continuous=True,
random_state=42, # pin the seed for repeatability
)
exp = lime_explainer.explain_instance(
X_test.values[0],
model.predict_proba,
num_features=8,
num_samples=5000,
)
print(exp.as_list()) # [(feature rule, weight), ...]
exp.show_in_notebook()
Set random_state every time. Without it, two runs on the same row can return different top features.
SHAP vs LIME: Head-to-Head Differences
Consistency and Stability
SHAP gives the same answer every time with TreeExplainer and LinearExplainer. LIME does not. Its random perturbations and kernel-width choice shift the surrogate. Run LIME ten times on one instance and the feature ranking can reorder.
Kernel width matters most. LIME’s default for tabular data is 0.75 × √(number of features). A narrow kernel makes the surrogate hyper-local and noisy. A wide kernel makes it less faithful to the local decision boundary.
Faithfulness
SHAP’s additivity means attributions reconcile exactly with the model output. LIME’s surrogate only approximates the model near the point. Check exp.score (the surrogate’s R²) after each explanation. A low score means the local linear assumption fails for that instance.
Handling Feature Correlation
Both methods struggle here, for different reasons. Standard SHAP with an interventional background breaks correlations and may evaluate the model on unrealistic feature combinations. LIME’s perturbation sampling ignores correlation entirely. When features are highly collinear, group them or drop redundant columns before explaining.
Compute Cost
| Scenario | SHAP | LIME |
|---|---|---|
| XGBoost, 100k rows, 50 features | Seconds to minutes (TreeExplainer) | Hours (5,000 calls per row) |
| Black-box API, one prediction | KernelExplainer: hundreds to thousands of calls | ~5,000 calls |
| Deep network, image | DeepExplainer: one backward pass | Superpixel perturbation: slow |
| Explaining the whole dataset | Native | Not designed for it |
Treat the numbers as relative. Actual runtime depends on model latency and hardware.
Decision Guide: Which Explainer Should You Use?
| Your situation | Pick | Why |
|---|---|---|
| Tree ensemble in production | SHAP TreeExplainer | Exact, fast, deterministic |
| Regulatory audit or adverse-action notice | SHAP | Additive, reproducible, defensible |
| Need global feature importance | SHAP | Aggregates local values cleanly |
| Closed API model, tight budget, one-off check | LIME | Cheap per instance, no model internals |
| Text classifier, want highlighted words | LIME or SHAP text | Both work; LIME is simpler to set up |
| Model monitoring and drift detection | SHAP | Track mean |SHAP| over time |
When in doubt, run both on a small sample. If they disagree on the top features, investigate. The disagreement usually points to correlated features or a nonlinear boundary.
Practical Workflows and Real-World Use Cases
Credit Scoring: Adverse-Action Reasons
A lender trains a LightGBM model on 120 features. Regulations require specific reasons for a denial. The team uses TreeExplainer to pull the top three negative SHAP contributions per applicant. Because attributions sum to the output, the reasons reconcile with the score. The same SHAP values feed a monthly dashboard that tracks whether feature importance drifts.
Healthcare: Sanity-Checking a Risk Model
A hospital builds a readmission model. A beeswarm plot reveals that an administrative field, such as discharge unit code, ranks above clinical variables. That is a data leakage signal. SHAP surfaced it in minutes. Without the global view, the team would have shipped a model keyed to a proxy.
Customer Support: Debugging a Text Classifier
A team classifies tickets with a fine-tuned transformer served behind an API. They cannot access gradients. LIME’s text explainer perturbs by removing words and highlights the tokens driving each label. They find the model keys on a boilerplate email footer. They strip the footer and retrain.
Workflow Template: Explanation Audit Loop
- Train the model and hold out a validation set.
- Compute SHAP values on the validation set.
- Rank features by mean |SHAP|. Flag any that look like leakage or proxies.
- Pick 20 borderline predictions. Compare SHAP and LIME on each.
- Log the SHAP base value and top features alongside every production prediction.
- Re-run the audit after each retrain.
Common Pitfalls
- Treating SHAP as causal. SHAP explains the model, not the world. A high attribution means the model relies on that feature. It does not mean changing the feature changes the outcome.
- Using too small a background set. For
KernelExplainer, summarize withshap.kmeans(X_train, 50)or sample 100 to 200 rows. A bad baseline skews every value. - Trusting one LIME run. Repeat with several seeds. Report only features that stay stable.
- Ignoring units. Tree SHAP values on classifiers default to log-odds. Convert before showing them to non-technical stakeholders.
- Explaining preprocessed features. If you one-hot encode a category, SHAP splits its credit across dummy columns. Sum them back to the original feature for reporting.
FAQ
What is the main difference between SHAP and LIME?
SHAP computes Shapley values, which fairly distribute a prediction across features and sum exactly to the model output. LIME fits a small linear model on perturbed samples around one prediction. SHAP offers theoretical guarantees and global views. LIME offers speed and simplicity for single predictions.
Is SHAP better than LIME?
SHAP is better when you need consistency, reproducibility, or global feature importance, especially for tree models. LIME is better for quick, low-cost explanations of black-box models or text. Neither wins everywhere. Match the tool to the model type and the audit requirement.
Why does LIME give different results each time?
LIME generates random perturbations for every explanation. Different samples produce slightly different surrogate models. Fix random_state, increase num_samples, and tune the kernel width to reduce variance.
Can SHAP and LIME explain deep learning models?
Yes. SHAP offers DeepExplainer and gradient-based variants for neural networks, plus text and image maskers for transformers and CNNs. LIME works on any model that returns predictions, using word removal for text and superpixel masking for images. Both get slower as input size grows.



