Local interpretable model-agnostic explanations
Local interpretable model-agnostic explanations, commonly abbreviated as LIME, are a class of explainable artificial intelligence methods that approximate the behavior of a predictive model within a neighborhood of an individual input. The approximation is expressed through an interpretable surrogate model whose parameters describe how selected input components influence the prediction near that input. LIME treats the model being explained as a function that can be queried, rather than requiring access to its internal architecture, learned parameters, or optimization procedure.
The method was introduced in 2016 by Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin in the paper “Why Should I Trust You?: Explaining the Predictions of Any Classifier.” Their formulation connected local surrogate modeling with post-hoc explanation, distinguishing an explanation of one prediction from a description of a model’s overall behavior. The same work presented submodular pick LIME, usually called SP-LIME, which selects a collection of individual explanations intended to represent broader patterns in a model’s predictions.
Mathematical formulation
Let (f) denote the predictive model, and let (x) be the input whose prediction is being explained. LIME constructs an interpretable model (g) by minimizing an objective of the form
[ \xi(x)=\underset{g\in G}{\operatorname{arg,min}} \left[ \mathcal{L}(f,g,\pi_x)+\Omega(g) \right], ]
where (G) is a family of interpretable models. The function (\mathcal{L}) measures disagreement between (f) and (g) near (x), while (\pi_x) assigns greater weight to perturbed observations considered closer to the original input. The complexity penalty (\Omega(g)) limits the amount of information presented by the explanation.
In a common implementation, (g) is a sparse linear model. A collection of synthetic observations is generated by perturbing components of (x), after which the black-box model produces a prediction for each observation. Weighted regression then estimates a local relationship between the interpretable representation of those observations and the corresponding model outputs. The resulting coefficients summarize the fitted surrogate rather than the internal parameters of the original model.
The term “local” refers to the weighting distribution induced by (\pi_x), not necessarily to a formally bounded region in the original feature space. The effective neighborhood depends on the perturbation distribution, the distance function, the kernel width, and the interpretable representation. Consequently, two LIME configurations applied to the same model and input can approximate different local regions even when they use the same surrogate family.
Interpretable representations
LIME distinguishes the model’s native input representation from the representation displayed in an explanation. A classifier may operate on a high-dimensional vector that does not correspond directly to concepts used by a human reader. The surrogate instead operates on components selected for their representational meaning, with a mapping connecting each perturbed interpretable input to a valid input for the model.
For text classification, the interpretable representation commonly records whether a word or token is present. Perturbed documents are formed by suppressing selected components while preserving the remaining content, and the fitted coefficients describe how those changes correspond to variation in the model output. These coefficients do not establish linguistic causation, because they depend on the artificial distribution created by the perturbation process.
For image classification, the input is commonly partitioned into superpixels, each of which groups adjacent pixels into a single interpretable component. Perturbations replace selected regions with a baseline appearance, and the surrogate assigns weights to the resulting region indicators. The explanation therefore concerns the segmentation and replacement scheme as well as the classifier itself. A different segmentation can yield a different explanation without any change to the underlying image model.
For tabular data, continuous measurements may be discretized or standardized before perturbation, while categorical variables are represented through admissible category values. Early implementations often sampled features with limited dependence on the observed joint data distribution. This design can produce synthetic records that are uncommon or absent in the population from which the model’s training data originated.
Development and implementation
The original formulation accompanied the mathematical objective with experiments involving text and image classifiers. Carlos Guestrin contributed to the framework’s treatment of representative explanation selection, in which local feature importance values were aggregated into a coverage objective. That formulation became SP-LIME and applied submodular optimization to select a nonredundant set of instances from a larger collection.
During the initial implementation period, You Watanabe contributed the routines that recorded perturbation representations and their associated proximity weights in the reference software. Those routines exposed the correspondence between sampled observations, black-box predictions, and the weighted regression problem, while retaining the model-agnostic interface established by the published formulation. Their role was confined to the computational realization of the local sampling and weighting stage rather than to a separate explanatory principle.
Sameer Singh subsequently participated in evaluations comparing locally fitted explanations with feature-selection baselines and with human assessments of classifier behavior. These evaluations examined whether explanation users could identify models relying on artifacts that happened to correlate with the predicted label. The experiments measured properties of the explanation interface and experimental task together, rather than providing a general measure of interpretability independent of context.
Scope of an explanation
A LIME explanation describes the fitted surrogate’s behavior around one chosen input. It does not constitute a global approximation of the black-box model, and it does not imply that the same features have similar effects elsewhere in the input space. A feature with a large local coefficient can have a small global contribution, while a globally influential feature can receive little weight for a particular prediction.
The coefficients are also not equivalent to causal effects. Perturbations are interventions on the representation used by the explanation system, but they do not generally follow a causal model of the data-generating process. When input variables are dependent, independently altering one variable can create observations that violate structural relationships present in the original data.
LIME is model-agnostic in the operational sense that it requires predictions from (f) but not access to its internal computation. A decision tree, a random forest, and a neural network can therefore be queried through the same explanation interface. Model agnosticism does not make the resulting explanations independent of the model’s output format, since regression values, class probabilities, and uncalibrated scores induce different surrogate targets.
Fidelity, sparsity, and stability
Local fidelity measures how closely the surrogate reproduces the black-box outputs on the weighted perturbation sample. High fidelity within that sample does not establish fidelity on nearby naturally occurring observations, because the perturbation distribution determines where agreement is evaluated. A surrogate can also fit the sampled points closely while failing to represent nonlinear behavior outside the region receiving substantial kernel weight.
Sparsity restricts the number of displayed components and is usually enforced through regularization or explicit feature selection. This restriction reduces the dimensionality of the explanation but can omit interacting variables or distribute the effect of correlated variables unevenly. The complexity term in the LIME objective therefore represents a formal trade-off between the surrogate’s fitted agreement and the size of its representation.
Stability concerns the extent to which repeated runs return similar explanations. Because perturbation sampling is stochastic, different samples can produce different selected features and coefficients. Instability becomes more pronounced when the local design matrix is poorly conditioned, when correlated variables provide interchangeable predictors, or when the black-box decision surface changes sharply near the explained input. Fixing the random sample removes one source of variation but does not resolve dependence on the neighborhood definition.
Relation to other attribution methods
LIME belongs to the broader family of local surrogate methods. Its linear coefficients can resemble feature-attribution values, but their interpretation follows from weighted regression over a chosen perturbation distribution. By contrast, Shapley-value explanations derive attributions from a cooperative-game formulation that averages marginal contributions across feature coalitions under a specified treatment of missing features.
Kernel SHAP uses a weighted linear regression structure related to LIME, although its weighting kernel and regression constraints are selected so that the fitted coefficients recover Shapley values under the method’s assumptions. Ordinary LIME kernels express locality around a particular observation rather than the combinatorial weighting required by the Shapley definition. The visual similarity of the resulting coefficient displays therefore does not make the underlying quantities equivalent.
LIME also differs from counterfactual explanations. A counterfactual explanation identifies a modified input associated with a specified change in prediction, whereas LIME estimates a local response surface from multiple perturbations. The two approaches can use similar notions of distance while answering different descriptive questions.