Active learning

Overview

Labels are often the expensive part of supervised learning: a clinician has to outline each lesion, an expert has to tag each sentence. Active learning spends that budget selectively. A model is trained on a small labeled set, scores the unlabeled pool, and sends the most useful examples to an oracle (usually a human annotator). The loop repeats until the budget or a stopping rule is reached 2009.00236 §1.2, §3.

The invariant shared by all variants is that the oracle is the only source of new ground-truth labels, and each round adds a batch of them to the labeled set. What differs is how the batch is scored and how the unlabeled pool is used besides querying.

Classic active learning assumes fixed features and one label at a time. Deep networks learn their features, need many labels, and are costly to retrain after every label. The survey names three obstacles to combining the two 2009.00236 §2:

1. Model uncertainty. Softmax outputs look like confidences but are often overconfident, so ranking by them can do worse than random sampling 2009.00236 §2.
2. Too few labels per round. Deep training needs many labels, while querying one label at a time retrains often for little new data 2009.00236 §2, §3.1.1.
3. Split pipelines. Query strategies assume fixed features, while deep models learn features and classifier together. Treating these as separate problems can make the two parts diverge 2009.00236 §2, §3.3.

This page draws on nine cards. The survey A Survey of Deep Active Learning has no experiments of its own, and its taxonomy is the backbone here. A survey of active learning algorithms for supervised remote sensing image classification tests classic heuristics on SVMs and LDA with three image datasets; its results are plots only. GFlowNet contributes one multi-round loop with a learned proxy and an oracle. Leverage SHAP uses an active-learning idea outside labeling. Machine learning methods for histopathological image analysis is a review that names active learning as one answer to label scarcity in pathology; it has no experiments either. A Survey on Human-AI Collaboration with Large Foundation Models describes a human-in-the-loop threshold round, with no numbers of its own. Machine learning and information theory concepts towards an AI Mathematician is tagged with this concept but does not select labels; it is listed under Papers only. A Holistic Approach to Undesired Content Detection in the Real World applies the loop to rare categories in production traffic. Boosting the Learning for Ranking Patterns applies a similar query idea to asking a user for preferences rather than labels. Method details below that come from the survey are its descriptions of other papers; primary sources are not yet on cards, so check them before implementing.

Variants

The survey groups deep active learning into query-strategy optimization, label expansion, generic frameworks and stopping strategies 2009.00236 tab. 1, §3.

Batch-mode uncertainty and hybrid scores

Batch mode is the survey's main departure from classic one-at-a-time querying: pick a whole batch per round 2009.00236 §3.1.1. Per-sample scores then need a batch rule. Summing independent scores (top-K BALD) ignores redundancy, so a batch can fill with near-duplicates; a joint batch score (BatchBALD) avoids this 2009.00236 §3.1.1, fig. 2.

Common per-sample scores are margin sampling (the gap between the two most probable classes, smallest first) and entropy (largest first) 2009.00236 §3.1.2, eq. 5. Hybrid methods add a diversity term. Exploration-P, for example, trades total uncertainty of a selected set against pairwise feature similarity within it, with a trade-off weight \alpha 2009.00236 §3.1.2, eq. 6–7. Other named hybrids are DMBAL (clustering with informativeness as weights), BADGE (diversity and magnitude in gradient space, with no trade-off hyperparameter) and WAAL (distribution matching with a Wasserstein distance) 2009.00236 §3.1.2.

Adversarial examples can also serve as a boundary-distance proxy: DFAL uses DeepFool-generated points to estimate how far each sample is from the decision boundary 2009.00236 §3.1.2.

Classic heuristics with SVMs: committee, margin and posterior scores

Outside deep networks, the same loop is often run with an RBF-kernel SVM or LDA. A remote sensing survey groups the heuristics into three families 2104.07784 §2:

• Committee. Normalised entropy query-by-bagging trains several models on bootstrap bags and scores a pixel by the entropy of their votes, divided by the log of the number of classes predicted for it 2104.07784 §3.1, eq. 1. The paper's runs use 7 SVMs on 75% bags, and 12 LDAs on 85% bags 2104.07784 §6.2.
• Large margin. Margin sampling takes the candidate whose smallest absolute one-against-all SVM decision value is smallest, i.e. the one nearest a boundary 2104.07784 §4.1. MCLU takes the smallest gap between the largest and second-largest absolute decision values 2104.07784 §4.2, eq. 10.
• Posterior probability. Breaking Ties converts SVM outputs to probabilities with Platt's sigmoid and picks the candidate whose two top class probabilities are closest 2104.07784 §5.2. KL-max is also in this family; it retrains once per candidate, so the paper says it is hard to use with SVMs 2104.07784 §5.1.

A diversity step can be added to any of these. Given a shortlist of uncertain candidates, it adds the most uncertain one first and then repeatedly adds the candidate most different from the batch so far. The paper gives several distance measures, including an angle in feature space and distance to support vectors 2104.07784 §4.4, alg. 2.

The paper's own summary is that, with SVMs, large-margin heuristics with a diversity term suit best, and margin sampling was generally the strongest curve; with LDA the families performed similarly 2104.07784 §7, fig. 4, fig. 7. It also says its experiments are not meant to rank the heuristics 2104.07784 §7. Treat the ranking as the authors' reading of plotted curves, not a measured table.

Bayesian acquisition

Place a prior over the network weights and score each sample by how much its label is expected to tell you about the weights. That is BALD's mutual information 2009.00236 §3.1.3, eq. 3. In practice, dropout kept on at prediction time supplies approximate posterior samples (MC-dropout), and the survey notes that dropout training has been shown to be equivalent to variational inference 2009.00236 §3.1.3. The posterior predictive averages over those samples 2009.00236 eq. 9.

The survey reports several caveats here. An analysis it cites finds MC-dropout uncertainty performs about as well as softmax response 2009.00236 §3.1.3. Ensembles are reported to give better-calibrated uncertainty and to counter class imbalance during acquisition 2009.00236 §3.1.4. DEBAL combines ensembles with MC-dropout to address overconfidence from mode collapse 2009.00236 §3.1.3. ACS-FW frames batch selection as a sparse approximation of the full-data posterior 2009.00236 §3.1.3.

Card: A Survey of Deep Active Learning.

Density and core-set coverage

These methods ignore the model's confidence and pick a set that covers the data. Farthest-first traversal in an activation space and the k-Center core-set are the classic examples 2009.00236 §3.1.4. Core-set needs distances across the whole unlabeled pool, which gets expensive on large pools 2009.00236 §3.1.4. Discriminative Active Learning instead trains a classifier to separate labeled from unlabeled samples and queries the unlabeled samples it can most easily tell apart from the labeled set, so the labeled set comes to match the pool's distribution 2009.00236 §3.1.4.

At large scale, one reported study queries 10k or 500k samples per round and finds that training on an AL-chosen subset can beat training on the full dataset 2009.00236 §3.1.4. The survey also notes that for some datasets per-sample importance differs too little for subset training to pay off 2009.00236 §3.1.4.

Label expansion: pseudo-labels, generators, semi-supervised training

Instead of only asking for labels, use the unlabeled pool as well. CEAL sends uncertain samples to the oracle and assigns confident ones a pseudo-label from the model 2009.00236 §3.2, fig. 5. VAAL trains a VAE and a discriminator so that the discriminator's view of the latent space picks which samples to query 2009.00236 §3.2. GAN-based methods (GAAL, BGADL, ARAL) generate candidate samples in regions of model disagreement 2009.00236 §3.2, fig. 6–7. SSAL pretrains on all data before the loop, then alternates supervised and semi-supervised training in each cycle 2009.00236 §3.2.

Frameworks that learn the query

RAL replaces the fixed acquisition function with a policy network trained by reinforcement learning on oracle feedback. Active-iNAS runs architecture search during the loop instead of fixing the network 2009.00236 §3.1.5, fig. 4. Uncertainty can also come from several layers rather than only the last one. LLAL attaches a small module that predicts per-sample loss from hidden layers, trains it jointly with the main loss, and queries the highest predicted losses 2009.00236 §3.3, fig. 9.

Multi-round generative search with a proxy

When the oracle is an expensive simulator or assay rather than a labeler, the loop can be framed as design. The GFlowNet paper runs a multi-round loop: fit a proxy on the data so far, train the generator to sample objects in proportion to the proxy's reward, query a batch, label it with the oracle, add it to the dataset and repeat 2106.04399 §4.3, alg. 1. The proxy is a Gaussian process in a 4-D hypergrid and an MPNN for molecules 2106.04399 §4.3, app. A.5.1, app. A.5.2.

Author-reported molecule result: after 1800 docking evaluations, GFlowNet's top-10 mean is 8.83 against 8.27 for MARS. PPO training diverged, so no numbers are reported for it 2106.04399 app. A.5.2. The paper's hypergrid multi-round figures are plots only 2106.04399 §4.3, fig. 6. Card: Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation.

Leverage-score sampling outside labeling

Leverage SHAP does not choose labels. It chooses which feature subsets to evaluate when estimating Shapley values, using leverage-score sampling for regression, an idea the paper says it takes from active learning 2410.01917 §1.3. Use it only as a borrowed technique; the loop above does not apply to it. Card: Provably Accurate Shapley Value Estimation via Leverage Score Sampling.

Pairwise queries for user preferences

Here the oracle is a user, and the thing learned is a weight vector over interestingness measures, not labels. The user ranks small sets of patterns. Each measure is scored for how well it agrees with that ranking, and the pairwise differences between measures feed an Analytic Hierarchy Process eigenvector solve 2610.16511 §4.1, alg. 1.

The active mode draws a sample of 10^3 patterns, orders it by the current score, and asks about the adjacent pair with the smallest ratio of score gap to summed measure gap 2610.16511 §4.3.1, eq. 7, alg. 2. The rule resembles margin-based uncertainty sampling; that comparison is our reading, not the paper's. Over 10, 50 and 100 queries, AHP's Spearman rho beats the SVM baseline on Hepatitis, Mushroom and Retail, ties on T40, and is below it on Connect at 50 and 100 queries 2610.16511 tab. 7. With simulated flipped answers on one emulator, AHP keeps rho near 77% without errors and near 70% at 40% errors; SVM falls from about 71% to 54% 2610.16511 §6.8. Passive learning is the weaker side: SVM is more accurate than AHP on Hepatitis, and SVM timed out on every other dataset 2610.16511 tab. 5, §6.6 B. All of this uses simulated users. Card: Boosting the Learning for Ranking Patterns.

Rare classes in live traffic: mixed sub-strategies

Public datasets rarely match production traffic, and some categories are rare in it. The authors score a large production sample each round, then draw a batch from three parts: random sampling, samples whose score for a category passes a threshold, and samples whose score is closest to 0.5 2208.03274 §3.1. The mix of parts is adjusted between rounds, and batches are re-weighted by metadata counts with weight proportional to the square root of the count 2208.03274 §3.1. The metadata, the threshold value and the random fraction are not stated 2208.03274 §3.1. Their run scores about 5×10^5 production samples and selects about 2000 per round, for three rounds 2208.03274 §4.3.

The reported gain is in label share, not model quality: the share of labeled samples carrying each undesired label is 10 to 22 times higher under active learning than under random sampling 2208.03274 tab. 4. The introduction's "up to 10×" is below the table's 22.3× maximum 2208.03274 §1, tab. 4. The body gives no per-category AUPRC numbers for active learning, only a trend 2208.03274 §4.3, fig. 2. Card: A Holistic Approach to Undesired Content Detection in the Real World.

Histopathology: one label-saving option among several

A review of machine learning for whole-slide pathology images lists active learning among its answers to scarce labels. It describes the method as choosing the most useful unlabelled sample to label next, and names uncertainty sampling, query-by-committee, variance reduction and hypothesis-space reduction as criteria for that choice 1709.00786 §4.2.1. The review places it beside multiple instance learning, semi-supervised learning and transfer learning, and gives no comparison between them 1709.00786 §4.2.1–§4.2.3. Its problem-to-solution table cites a web page for the uncertainty-sampling entry, so that attribution is unclear 1709.00786 tab. 1. Card: Machine learning methods for histopathological image analysis.

Confidence-threshold rounds with a human in the loop

The human-AI survey describes a simple round: predict a label and a confidence for each unlabeled sample, send those below a threshold to annotators, retrain on the enlarged labeled set, and repeat until the model is confident 2403.04931 §2.1. The survey does not give the threshold value or a stopping count. Because the rule relies on softmax-style confidence, it inherits the overconfidence problem the deep survey names 2009.00236 §2. Card: A Survey on Human-AI Collaboration with Large Foundation Models.

Stopping

Most work uses predefined rules: a maximum number of iterations, a minimum change in accuracy, a minimum number of labels, or a target accuracy 2009.00236 §3.4. Stabilizing Predictions stops when predictions on a held-out unlabeled stop set stop changing, which needs no extra labels 2009.00236 §3.4. The survey does not give thresholds for either rule.

When to use which

The survey gives no end-to-end benchmark, so the table below follows its stated conditions, not measured rankings.

Two cross-cutting points: the training recipe can matter more than the sampler (SSAL reports this 2009.00236 §3.2), and retraining from scratch each round is costly while naive incremental training biases the model toward early or new samples 2009.00236 §3.1.3, §5.

Implementation sketch

The loop below follows the survey's generic DeepAL loop 2009.00236 §3, fig. 1c. The scoring step is where the variants differ. The SVM-based remote sensing survey uses the same loop with a fixed number of iterations and no stopping rule 2104.07784 alg. 1, §6.2.

# Train on the initial labeled set L0 [arxiv:2009.00236 §3]
model = train(L, theta_init)
while budget_left and not stop(model):
    # Score the unlabeled pool with the current model [arxiv:2009.00236 §3.1]
    scores = score(model, U)
    # Choose a batch; a joint or diversity-aware rule avoids duplicates
    # [arxiv:2009.00236 §3.1.1, §3.1.2]. Mixing random, threshold and
    # uncertainty parts is another option [arxiv:2208.03274 §3.1]
    batch = select_batch(scores, U, k)
    # Ask the oracle; move the batch from U to L [arxiv:2009.00236 §3]
    L = L + oracle(batch)
    U = U - batch
    # Retrain; cycle-level choices here, such as pseudo-labels or
    # SSAL-style semi-supervised steps, are described in [§3.2]
    model = train(L, model)

For the Bayesian score, average the class probabilities over posterior samples and compute entropy from that average, then the mutual information from the two terms 2009.00236 §3.1.3, eq. 3, eq. 9. The survey writes eq. 9 with the expectation of the network output rather than of the softmax, so averaging softmax probabilities is an interpretation of the notation, not a stated step.

Scoring direction differs by heuristic. In the SVM survey, margin, MCLU, the diversity variants and Breaking Ties pick the candidate with the smallest value, while nEQB, AMD and KL-max pick the largest 2104.07784 §3–§5. Writing one argmax for all of them is a common error; check each score's direction against its citation before reuse.

Contradictions

• Random sampling is a moving baseline. Two papers report random-sampling accuracy at the same CIFAR-10 setting that differ by 13%; one method's CIFAR-100 result differs by 8% between two papers 2009.00236 §5. The survey recommends common evaluation platforms, several runs with different initializations and ablations 2009.00236 §5.
• Diversity versus uncertainty. The core-set and DFAL papers, as the survey reports them, find diversity methods always beat uncertainty methods and uncertainty methods do worse than random sampling. LLAL reports otherwise 2009.00236 §5. The SVM remote sensing survey reports a smaller effect: a diversity term on MCLU seems to help slightly, and does not help on one dataset, though no run degrades 2104.07784 §7, fig. 6. Its datasets and models differ from the deep settings above, so the two are not directly comparable.
• MC-dropout versus softmax and ensembles. One analysis finds MC-dropout uncertainty about as good as softmax response 2009.00236 §3.1.3. Beluch et al. report ensembles beat MC-dropout and density-based selection 2009.00236 §3.1.4.
• Mode counts depend on the definition. The GFlowNet paper's main text counts more than 1500 Bemis-Murcko scaffolds above reward 8 for GFlowNet, against fewer than 100 for MARS. Its appendix counts Tanimoto-separated modes above reward 7 and reports more than 500 for GFlowNet, against fewer than 100 for MARS 2106.04399 §4.2, app. A.7. These numbers use different definitions and should not be compared.
• Subset training versus full data. Chitta et al. report AL-chosen subsets can outperform full-data training 2009.00236 §3.1.4; the survey also says some datasets gain little from subset selection 2009.00236 §3.1.4.
• Text versus table in the preference-query paper. The text reports AHP as better overall and does not mention Connect. The table has AHP at a constant 0.10 on Connect, below SVM at 50 and 100 queries; the paper's own text does not explain this, and the constant may be a copy error 2610.16511 tab. 7, §6.7 B. Check the source before relying on the Connect row.

Papers

• A Survey of Deep Active Learning: taxonomy of deep active learning (query strategies, label expansion, frameworks, stopping) with an application table and a reproducibility discussion; no experiments of its own.
• Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation: generative sampler for diverse high-reward objects, with a multi-round proxy-and-oracle loop in its experiments.
• A survey of active learning algorithms for supervised remote sensing image classification: committee, large-margin and posterior heuristics, with a diversity term, tested with SVMs and LDA on three remote sensing datasets; results are plots only.
• Provably Accurate Shapley Value Estimation via Leverage Score Sampling: leverage-score sampling for Shapley estimation, an idea borrowed from active learning; not a labeling method.
• Machine learning methods for histopathological image analysis: review of pathology image analysis that lists active learning among its answers to scarce labels; no experiments.
• A Survey on Human-AI Collaboration with Large Foundation Models: survey of human-AI collaboration; its build-time section describes a confidence-threshold annotation round; no experiments.
• Machine learning and information theory concepts towards an AI Mathematician: position essay on rewarding compact sets of theorems; it proposes reward-proportional conjecture sampling 2403.04571 §8 and no label-selection rule, so it is related only and not an active-learning method.
• A Holistic Approach to Undesired Content Detection in the Real World: classifier for undesired content; active learning on production traffic with mixed random, threshold and uncertainty sampling, reported as a 10–22× gain in label share over random sampling.
• Boosting the Learning for Ranking Patterns: learns a user's ranking as weights over interestingness measures with AHP, and asks pairwise questions in an active mode; simulated users only.