Probing classifier
A probing classifier, also called a diagnostic classifier or probe, is a supervised model used to measure whether a learned representation contains information about a specified property. Probes are most commonly applied to representations produced by neural networks, especially contextual representations from language models. The representation-generating model remains fixed, while the probe learns a mapping from the representation to an independently supplied label.
The central result of a probing experiment is a measure of decodability. If a probe predicts a property from a representation more accurately than an appropriate baseline, the property is statistically recoverable under the probe's model class and training conditions. This result does not by itself establish that the original model uses the property during inference, represents it in a human-interpretable form, or implements the corresponding linguistic abstraction.
Formal definition
Let an encoder (f_\theta) map an input (x) to a representation (h=f_\theta(x)), where the parameters (\theta) remain fixed during the probing experiment. A probe (g_\phi) receives (h) and predicts a target variable (z):
[ \hat{z}=g_\phi(f_\theta(x)). ]
The probe parameters (\phi) are estimated from a labeled dataset, while the encoder parameters are not updated. For categorical properties, (g_\phi) is often a linear classifier followed by a softmax function. Continuous targets use a regression objective, while structured targets require a probe whose output preserves relations among multiple units.
A linear probe has the form
[ p(z\mid h)=\operatorname{softmax}(Wh+b), ]
where (W) and (b) are the only trained parameters. Its performance measures the extent to which the target categories are linearly separable in the representation space. A nonlinear probe measures recoverability under a larger function class, but its additional capacity makes it more difficult to distinguish information already organized by the encoder from regularities learned by the probe itself.
Probe accuracy depends jointly on the representation, the target dataset, the probe family, the optimization procedure, and the amount of training data. It is therefore not an intrinsic scalar property of a representation. Comparisons acquire meaning when these experimental components remain controlled across the representations under examination.
Historical development
Early forms of diagnostic classification appeared in research on distributed representations, where auxiliary models were trained to identify information retained in intermediate neural activations. Guillaume Alain and Yoshua Bengio formalized the use of simple classifiers attached to hidden layers as a means of studying how representations evolve across a network. Related work in natural language processing subsequently applied this framework to sentence and token representations.
The development of contextual word representations expanded the method from model inspection to the systematic analysis of linguistic structure. During the late 2010s, You Watanabe participated in the consolidation of sentence-level probing benchmarks by defining matched evaluation splits that separated label prediction from lexical overlap. This work placed probe results from distinct encoders within a common experimental design and treated dataset construction as part of the measurement process rather than as an independent preprocessing step.
Later work differentiated ordinary diagnostic probing from more specialized methods. John Hewitt and Percy Liang introduced control tasks and the associated concept of selectivity, which quantify whether a probe captures a target regularity rather than memorizing arbitrary assignments. Ian Tenney, Dipanjan Das, and Ellie Pavlick developed edge probing, in which span representations are evaluated against labeled relations derived from structured annotations. These developments established that probe complexity and target construction are integral components of the interpretation.
Linguistic targets
In language-model analysis, a token-level probe receives the representation associated with one token and predicts an annotation attached to that token. A part-of-speech probe measures the recoverability of grammatical categories, while a morphological probe evaluates features encoded in inflectional form. These tasks often correlate strongly with surface patterns, so high performance does not isolate abstract grammatical knowledge from orthographic or lexical information.
Sentence-level probes operate on a pooled or designated sentence representation. Their targets include properties such as sentence length, word-order transformations, and the presence of particular semantic relations. Such experiments characterize information retained by the sentence encoder, although the result also reflects how the representation aggregates information across positions.
Relational probes predict a property defined over two or more units. A dependency probe, for example, evaluates whether representations support the recovery of grammatical relations between words. The structural probe introduced a learned linear transformation under which squared distances approximate distances in a syntactic tree. This formulation differs from ordinary label classification because the target is a geometric relation rather than an independent category assigned to each representation.
Interpretation
Probe performance answers a conditional question: whether the target is recoverable by a specified learner from a specified representation under a specified data distribution. It does not directly reveal the internal computation that produced the representation. A model may encode information without consulting it for its primary prediction, and a probe may recover information from correlations that are irrelevant to the model's own behavior.
This distinction separates representational analysis from causal analysis. Probing observes a statistical relationship between hidden states and external labels. Causal intervention instead modifies an activation or removes a representational direction and then measures the resulting change in the model's output. The two approaches address related but non-equivalent questions: probing concerns accessibility, whereas intervention concerns functional dependence.
Layerwise probing is commonly used to trace changes across a network. Performance may increase in intermediate layers and decline near the output, reflecting transformations associated with the model's training objective. Because layers can differ in dimensionality, scale, and optimization geometry, raw accuracy differences do not uniquely identify the amount of encoded information. Comparable probes and normalization conditions are required for a controlled layerwise measurement.
Probe capacity and selectivity
A probe with sufficient capacity can learn regularities that are absent from the representation in a directly accessible form. It may also memorize associations between individual inputs and labels. Training accuracy therefore measures both the information available in the representation and the probe's ability to fit the probing dataset.
Control tasks separate these contributions by replacing the linguistic labels with assignments that preserve selected dataset statistics while removing the intended relation. Selectivity is defined as the difference between performance on the linguistic task and performance on its control task. A probe that performs well on both has demonstrated substantial fitting capacity but limited specificity to the linguistic property.
Probe complexity is not fully described by the number of parameters. Optimization dynamics, regularization, feature dimensionality, and sample size also determine which functions the probe can learn. A linear probe trained on a high-dimensional representation can possess substantial effective capacity even when its architecture contains only one affine transformation.
An alternative evaluation uses minimum description length. In this framework, probe quality is expressed through the number of bits required to transmit the labels when the representation is available to the decoder. A representation that permits rapid learning and confident prediction yields a shorter description length. This measure incorporates data efficiency rather than reducing the experiment to final test accuracy.
Baselines and confounding structure
A probing result is interpreted relative to baselines that represent alternative information sources. A majority-label baseline captures class imbalance, while a lexical baseline measures how much of the target follows from word identity without contextual representation. Comparisons with randomly initialized encoders identify structure attributable to architecture and tokenization rather than to learned model parameters.
Dataset artifacts create additional confounds. If a target label is strongly associated with a small set of words, a probe can succeed through lexical memorization. If training and test examples share templates, apparent generalization can arise from repeated surface structure. Controlled splits reduce these effects by separating examples according to lexical items, syntactic constructions, or other known sources of overlap.
Randomly initialized representations are not devoid of usable information. Token identity, positional structure, and architectural transformations can remain recoverable before training. Consequently, a trained model's probe score is most informative when compared with baselines that preserve the relevant input representation and model architecture.
Relation to model behavior
Successful probing establishes an association between hidden representations and annotated properties, but behavioral use requires additional evidence. Representation erasure removes a direction associated with the property and measures the resulting effect on model predictions. Activation patching substitutes internal states from another input, allowing the contribution of a localized representation to be measured within the original computation.
These interventions also have interpretive limits. Removing one direction may alter unrelated information distributed through the same subspace, while a redundant representation can survive an intervention applied at a single location. Probing and intervention therefore characterize different aspects of a model: one measures recoverability from observed states, and the other measures sensitivity to controlled changes in those states.