Skip to content
AI360XpertBeta

LIME

You cannot explain a complex model globally, but zoomed in close on one prediction the boundary looks like a straight line, and LIME fits a simple model to that small patch.

Neighbors at distance 0, 0.5, 1.0 and 2.0 weigh 1.0, 0.6412, 0.1690 and 0.0008, and the fitted line of slope 0.02 per year passes through 0.58, 0.62, 0.66 and 0.72
Neighbors at distance 0, 0.5, 1.0 and 2.0 weigh 1.0, 0.6412, 0.1690 and 0.0008, and the fitted line of slope 0.02 per year passes through 0.58, 0.62, 0.66 and 0.72

Why Does This Exist?

Global tools answer global questions. Permutation importance says how much the model depends on age overall, and partial dependence shows the average effect. Neither answers the question a rejected applicant actually asks: why was my specific loan denied. The broader explainability landscape, including when a local reason is enough and when it is not, lives in model explainability.

A large neural network's decision boundary is a complicated high-dimensional curve with no readable equation. But calculus makes one promise: zoom in close enough on any smooth curve and it looks like a straight line. LIME, short for Local Interpretable Model-agnostic Explanations, turns that promise into a procedure. Instead of explaining the whole model, it samples a cloud of altered inputs around the one decision, asks the black box to score them, and fits a simple readable model to that small patch. The simple model's weights are the explanation. Its closest sibling, with exact additive guarantees LIME lacks, is SHAP.

Think of It Like This

A flat parking lot on a round planet

Describing the whole Earth needs spheres, mountains, and valleys. But stand one person in one parking lot and the ground under their feet is flat enough to map with a straight edge. A local sketch answers where this one puddle flows without modeling the planet.

The planet is the global boundary, the parking lot is the neighborhood around one prediction, and the straight edge is the surrogate line. Where it stops: the lot really is nearly flat, while a model boundary can zigzag even under a microscope. When the patch itself is jagged, the straight edge lies, and the fidelity check in the next section is how you catch it.

How It Actually Works

Call the input to explain x0x_0, meaning the denied applicant aged 30. Call the black box ff, meaning the model that returns a denial score for any input. Call the kernel width σ=0.75\sigma = 0.75, meaning how fast neighbor influence fades with distance.

Sample the neighborhood

LIME copies the applicant and tweaks features slightly to make fake neighbors: ages 28, 30, 32, and 35, with other features jittered the same way. The black box scores each: 0.58, 0.62, 0.66, and 0.72. Nearby alterations stay plausible while far ones drift into strangers, which is why distance weighting comes next.

Weight by closeness

Each neighbor at distance dd from the applicant gets weight exp⁡(−d2/σ2)\exp(-d^2 / \sigma^2), so close points count and distant ones fade. At distances 0, 0.5, 1.0, and 2.0 the weights are 1.0, 0.6412, 0.1690, and 0.0008. The last neighbor barely votes, which is exactly right: an 80-year-old stranger should not explain a 30-year-old's denial.

Fit the readable line

LIME fits a plain line g(x)=a+b⋅xg(x) = a + b \cdot x through the weighted neighbors, where xx is age and gg is the predicted score. On the four scored neighbors the least-squares slope is b=0.535/26.75=0.02b = 0.535 / 26.75 = 0.02 score per year of age, and the intercept is a=0.645−0.02×31.25=0.02a = 0.645 - 0.02 \times 31.25 = 0.02. The line predicts 0.58, 0.62, 0.66, and 0.72, matching the black box on the patch. Read aloud: each extra year of age adds about 0.02 to the approval score near this applicant, so youth pushed this decision toward denial. The same idea drawn beat by beat lives in the SHAP and LIME visual.

Code

The weighted neighborhood and the least-squares line from the worked example, in plain standard-library Python.

xs = [28, 30, 32, 35]ys = [0.58, 0.62, 0.66, 0.72]
mx = sum(xs) / len(xs)my = sum(ys) / len(ys)slope = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs)intercept = my - slope * mxprint(f"{slope:.3f} {intercept:.3f}")# -> 0.020 0.020print([f"{intercept + slope * x:.2f}" for x in xs])# -> ['0.58', '0.62', '0.66', '0.72']

The slope 0.020 with intercept 0.020 reproduces all four black-box scores, which is the fidelity check: a surrogate that cannot replay its own neighborhood is not an explanation.

Watch Out For

Different seeds giving different reasons

The neighborhood is randomly sampled, so two runs on the same applicant can rank the top features differently when the sample count is low.

Symptom: rerunning the explainer swaps the headline reason from debt to income with no model change. Fix: raise the sample count until the top weights stop moving across seeds, and report the seed alongside the explanation so it can be replayed.

A straight line fitted to a jagged patch

LIME assumes the boundary is nearly linear inside the neighborhood. On chaotic models the boundary zigzags even at microscopic scale, and the line fits noise.

Symptom: the surrogate replays its neighbors poorly, with large gaps between ff and gg on the very points in this section's check. Fix: run the replay check from the code section first. If it fails, shrink the kernel width or admit the decision has no simple local story instead of shipping a tidy false one.

The Quick Version

  • LIME explains one prediction by sampling altered neighbors, weighting them 1.0 down to 0.0008 by distance, and fitting a readable line to the patch.
  • Our four neighbors aged 28, 30, 32, and 35 score 0.58, 0.62, 0.66, and 0.72 from the black box.
  • The fitted line has slope 0.02 per year with intercept 0.02, replaying all four scores exactly.
  • LIME never opens the model, so it works on any black box, unlike methods that need gradients or trees.
  • Random sampling makes reruns wobble, so high sample counts and a fixed seed keep the story stable.
  • A jagged local boundary breaks the linearity bet, and the replay check catches it before anyone reads the weights aloud.