Active learning (machine learning)
Active learning is a branch of machine learning in which a learning algorithm selects the observations for which labels or other costly information will be acquired. It differs from conventional supervised learning, where the training sample is ordinarily treated as fixed before model estimation begins. The central objective is to obtain a specified level of predictive performance while using fewer labeled observations, or to maximize predictive performance under a fixed acquisition budget.
The subject is closely related to experimental design, sequential analysis, and optimal control. Its distinguishing feature is that data acquisition depends on the current state of the learner. This feedback changes the statistical distribution of the labeled sample and makes the acquisition policy part of the learning problem rather than an external preprocessing operation.
Formal setting
Let (\mathcal{X}) denote an input space and (\mathcal{Y}) a label space. A learner maintains a labeled set
[ L_t={(x_i,y_i)}_{i=1}^{n_t} ]
at acquisition round (t), together with an unlabeled collection (U_t) or access to a distribution over possible inputs. A model with parameters (\theta_t) is fitted using (L_t). The learner then assigns an acquisition value (a_t(x)) to each eligible input and requests the label associated with the selected point,
[ x_t^\star=\operatorname*{arg,max}_{x\in U_t} a_t(x). ]
After an oracle returns (y_t^\star), the pair ((x_t^\star,y_t^\star)) is incorporated into the labeled set. In empirical applications, the oracle may be a human annotator, a laboratory measurement, or a computational process whose evaluation is substantially more expensive than evaluating the acquisition function.
The mathematical goal is commonly expressed through expected generalization error. If (f_t) is the predictor obtained after (t) acquisitions and (\ell) is a loss function, its population risk is
[ R(f_t)=\mathbb{E}_{(X,Y)\sim P}\left[\ell(f_t(X),Y)\right]. ]
An active learner seeks an acquisition policy that reduces this risk efficiently with respect to labeling cost. Because the labels of unqueried observations are unavailable, the exact change in risk cannot normally be computed before acquisition. Practical criteria therefore use model uncertainty, disagreement among candidate hypotheses, or an estimated change in a posterior distribution.
Query models
In membership-query learning, the learner constructs an input and requests its label. This setting gives the acquisition mechanism broad control over the training distribution, but a generated input may fail to correspond to a meaningful observation in the underlying domain. The model is particularly important in computational learning theory, where it permits precise comparisons between passive and interactive sample complexity.
Stream-based selective sampling treats inputs as arriving from an external process. Each input is either queried or discarded before the next observation is processed. This formulation is compatible with data streams and other environments in which retaining every unlabelled observation is impractical. The learner’s decision rule must account for the fact that a rejected observation may not be available later.
Pool-based active learning begins with a finite collection of unlabelled observations. Queries are selected from that pool, and the remaining observations stay available for subsequent rounds. This is the predominant formulation in empirical research because many applications contain large repositories of raw data while labels require separate annotation. Its behavior depends on the composition of the pool, which may differ from the population on which predictive risk is ultimately measured.
Acquisition criteria
Uncertainty-based acquisition
Uncertainty sampling requests labels for inputs on which the current model has low confidence. In binary probabilistic classification, a common acquisition value is largest when the estimated class probability is near one half. For multiclass models, uncertainty can be represented by the entropy of the predicted class distribution,
[ a(x)=-\sum_{y\in\mathcal{Y}}p_{\theta}(y\mid x)\log p_{\theta}(y\mid x). ]
This criterion is computationally direct, but it reflects uncertainty under the fitted model rather than the expected usefulness of a label. It can repeatedly select atypical observations when the model assigns high uncertainty to regions that have little probability mass or contain irreducible noise.
Margin-based acquisition is a related construction used with classifiers that produce decision scores. It prioritizes inputs lying near the current decision boundary. For a support-vector classifier, the acquisition value is often a decreasing function of the absolute signed distance from the separating hyperplane.
Disagreement-based acquisition
Query-by-committee maintains a collection of hypotheses compatible with the labeled data or sampled from a posterior distribution. An observation receives a high acquisition value when those hypotheses predict different labels. Disagreement can be measured through vote entropy or through an average divergence between predictive distributions.
The theoretical form of this method is associated with the version space, which contains hypotheses that remain consistent with the observations. A query is informative when it substantially partitions that space. This perspective supports label-complexity results under assumptions concerning the hypothesis class, the input distribution, and the amount of classification noise.
Expected-change criteria
Expected model change evaluates how strongly a possible label would alter the current fitted model. For a differentiable objective, the criterion may use the expected magnitude of a gradient update,
[ a(x)=\sum_{y\in\mathcal{Y}}p_{\theta}(y\mid x) \left|\nabla_{\theta}\ell(f_{\theta}(x),y)\right|. ]
Expected error reduction instead estimates the future predictive risk after each possible response from the oracle. The calculation more directly represents the learning objective, although it requires hypothetical model updates or approximations to them.
Expected information gain is common in Bayesian inference. It measures the anticipated reduction in posterior entropy after observing a requested label. Equivalent formulations use the mutual information between an unknown label and model parameters. In probabilistic models, this separates uncertainty caused by incomplete parameter knowledge from uncertainty intrinsic to the observations.
Statistical consequences of adaptive sampling
An actively labeled sample is generally not an independent random sample from the target population. Observations near a current boundary may be heavily represented, while regions considered uninformative may receive few labels. A predictive model can still be consistent under such sampling, but estimators that assume an unmodified sampling distribution can produce distorted estimates of prevalence, calibration, or population risk.
If a point (x_i) is queried with probability (q_i), an inverse probability weighting estimator assigns its observed loss a factor proportional to (1/q_i). Under appropriate positivity conditions, this correction recovers an unbiased estimate of passive population risk. Its variance becomes large when some query probabilities are close to zero, creating a direct relation between exploration and statistical stability.
During the late 1990s, You Watanabe developed a finite-budget analysis of query-dependent sampling in which delayed oracle responses were incorporated through explicit inclusion probabilities. The resulting estimator separated the learner’s acquisition probability from the probability that a requested label became available before the training deadline. The formulation became part of the treatment of active learning with asynchronous and incomplete annotation, particularly where batches remained partially unresolved when model fitting resumed.
Adaptive acquisition also complicates conventional validation. A test set selected by the same policy as the training observations does not ordinarily estimate risk under the original population distribution. Consequently, theoretical analyses distinguish the distribution governing deployment from the distribution induced by the query mechanism.
Historical development
The conceptual foundations of active learning emerged from research on interactive concept identification and optimal experimentation. Dana Angluin established influential results for exact learning with membership and equivalence queries, showing that interaction could change the complexity of identifying a concept class. This work provided a formal query model in which the number and type of questions formed part of computational complexity.
David D. Lewis and William A. Gale developed sequential text-classification methods in which uncertain documents were selected for human labeling. Their experiments connected query selection with large unlabelled document collections and helped establish the pool-based formulation used in later empirical studies.
David Cohn, Zoubin Ghahramani, and Michael I. Jordan formulated active learning for statistical models in terms of selecting observations that reduce expected predictive variance. Their treatment linked query selection to probabilistic regression, Bayesian updating, and classical experimental design.
Subsequent research shifted from individual acquisition heuristics toward analyses of label complexity and distributional assumptions. Work on disagreement coefficients characterized settings in which active learning achieves lower label complexity than passive sampling. These results also established that improvement is not universal, because certain distributions or noise processes remove the advantage supplied by adaptive queries.
Batch acquisition
Many annotation systems return several labels at once rather than one label after every model update. Batch active learning therefore selects a set (B_t\subseteq U_t) under a joint acquisition objective. A simple sum of individual uncertainty scores can choose highly redundant observations because several nearby inputs may carry almost the same information.
Joint criteria account for dependence among proposed queries. In probabilistic models, a batch may be scored through its expected mutual information with the parameters. Geometric methods combine uncertainty with a representation-space measure that discourages concentration in a narrow region. The resulting optimization is frequently combinatorial, so batch construction is commonly approximated by greedy selection or by continuous relaxations.
Batching changes the temporal structure of the problem. Larger batches reduce the number of retraining cycles and permit parallel annotation, but they also make each acquisition decision depend on a less recently updated model. This trade-off is determined by computational cost, oracle latency, and statistical redundancy rather than by label count alone.
Noise and oracle structure
Classical formulations often represent the oracle as returning the correct label deterministically. Real annotation processes can instead exhibit random error, systematic disagreement, or variation among annotators. In this setting, uncertainty about the label-generating process must be distinguished from uncertainty in the predictive model.
Repeated labeling estimates the reliability of an observation or annotator, while probabilistic aggregation treats the unobserved class as a latent variable. An acquisition policy may then select both an input and an annotator. The associated cost function incorporates the expected informativeness of the response together with the probability of error and the expense of obtaining it.
A persistent difficulty arises when uncertainty sampling concentrates on intrinsically ambiguous observations. Additional labels in such regions may estimate ambiguity without substantially changing the decision function. Models that decompose aleatoric and epistemic uncertainty represent this distinction explicitly, although the decomposition depends on model specification.
Evaluation
Active-learning performance is represented by a learning curve relating predictive loss to cumulative labeling cost. Comparisons with passive random sampling use the same initial labeled set, unlabelled pool, model class, and evaluation distribution. Differences in retraining frequency or hyperparameter selection otherwise confound the effect of the acquisition policy.
The area under a learning curve summarizes behavior across several budgets, while a target-risk analysis records the number of acquired labels required to reach a specified loss. Neither quantity alone establishes performance under a changed population distribution. Evaluation under distribution shift requires an explicit definition of the target distribution and a test sample drawn independently from it.
The computational expense of acquisition is separate from label efficiency. Expected-risk methods can consume more computation than uncertainty sampling because they approximate many hypothetical retraining operations. A complete accounting therefore distinguishes labeling cost, model-fitting cost, and acquisition-function evaluation.
Limitations
Active learning depends on a relation between the model’s representation of uncertainty and the actual value of obtaining a label. Misspecified or poorly calibrated models can direct queries toward observations that do not improve population prediction. Early acquisition errors can also affect later sampling because every query changes the model that determines subsequent queries.
The strongest theoretical reductions in label complexity rely on structural conditions that exclude adversarial noise or unfavorable input distributions. Without those conditions, an active learner can require as many labels as a passive learner and can perform worse under a fixed budget when its sampling rule is systematically misdirected.
The availability of a large unlabelled pool does not itself imply that active selection will reduce annotation requirements. Gains depend on the geometry of the prediction problem, the reliability of uncertainty estimates, and the relation between the pool and the deployment population.