Active Learning by Acquiring Contrastive Examples

In plain words

Labeling data costs time and money, so active learning asks which unlabeled examples are worth labeling next. Two common answers are to pick the examples the model is least sure about, or to pick a spread of examples that covers the data. Both have a known weakness: uncertain picks can be repetitive, and spread-out picks can be too easy for the model.

This paper proposes a different choice, called CAL. It looks for examples that the model places near examples it has already labeled, but that the model predicts very differently from those neighbours. The authors expect such examples to sit close to the line where the model changes its answer, which is where a new label helps most.

The authors test CAL on sentiment, topic, language-inference and paraphrase tasks with a BERT model. They report that it matches or beats the other strategies they compared, including on data from a different source than the training data.

TL;DR

• Active learning picks which unlabeled examples to label next; CAL picks ones that look similar to labeled data in the model's feature space but get very different predicted probabilities §2.1.
• Score: for each pool example, average the KL divergence between the predicted distribution of each of its k nearest labeled neighbours and its own §2.2 alg. 1.
• Author-reported: best out-of-domain accuracy on SST-2→IMDb (79.00 ± 1.39) and QQP→TwitterPPDB (86.20 ± 0.22); on IMDb→SST-2 Entropy is best (85.54 vs 84.96) tab. 2.
• In-domain results are shown only as curves fig. 2; the ablations are shown only as curves on AG News fig. 3.
• Costs more than entropy sampling: one extra pass over the labeled set plus a neighbour search each round app. A.2 tab. 4.

Problem

Pool-based active learning labels data in rounds: train a model on the labeled set, score the unlabeled pool, label a batch, repeat until a budget is spent §2.2. The two dominant score families have known limits. Uncertainty-based scores can select uncertain but redundant points, and diversity-based scores can select diverse but easy points §1. Hybrid methods that cluster uncertainty-derived representations (BADGE, BertKM) are described as computationally heavy, with cost growing with the acquisition size §3.2, footnote app. A.2.

The method assumes a simulated loop: the pool's labels are known and revealed only when an example is acquired §2.2, footnote. Models are BERT-base with a task-specific classification layer §3.3.

Method

CAL scores each unlabeled example by how strongly its predicted class distribution disagrees with those of its nearest labeled neighbours, measured in the model's own [CLS] feature space. The "neighbourhood" is the top-k labeled examples, not a distance threshold §2.2. The score is a mean of KL divergences, and the batch is the b highest-scoring pool examples alg. 1.

1. Fine-tune BERT-base with a classification layer on the current labeled set D_lab §2.2 §3.3.
2. Represent every labeled and pool example by its [CLS] embedding Φ(·) §2.2.
3. For each pool example x_p, find the k nearest labeled examples in Euclidean distance; k is fixed, not thresholded §2.2.
4. Compute the predicted distributions p(y|x) of the neighbours and of x_p with the current model §2.2 alg. 1.
5. Score x_p as the mean over neighbours of KL(p(y|x_l) ‖ p(y|x_p)); the neighbour's distribution comes first §2.1, footnote alg. 1.
6. Select the b pool examples with the largest scores; reveal their labels, move them to D_lab, and repeat §2.2 alg. 1.

The paper defines contrastive examples with two criteria, a similarity bound and a divergence condition eq. 1 eq. 2. The bound ε in eq. 1 is replaced in practice by a fixed k nearest neighbours §2.2, and the divergence condition in eq. 2 is realised as the mean KL score in step 5 alg. 1.

d\big(\Phi(x_i),\Phi(x_j)\big) < \epsilon eq. 1

\mathrm{KL}\big(p(y|x_i)\,\|\,p(y|x_j)\big) \rightarrow \infty eq. 2

Selection for one round, as the paper's Algorithm 1 gives it, in our own pseudocode. The paper's KL is written as KL(neighbour ‖ candidate) §2.1.

# CAL: score each pool example, keep the top b  [alg. 1] [§2.2]
# Input:  labeled set (m examples), pool (n examples), batch size b,
#         fine-tuned model, neighbour count k, encoder phi (the [CLS] vector)
# Output: indices of b pool examples

def cal_select(labeled, pool, b, model, k, phi):
    # Embed labeled and pool examples with the model's [CLS] vector  [§2.2]
    labeled_emb = phi(labeled)                  # (m, d)
    pool_emb = phi(pool)                        # (n, d)
    # Predicted class distributions of labeled and pool examples  [alg. 1]
    labeled_prob = softmax(model(labeled))      # (m, C)
    pool_prob = softmax(model(pool))            # (n, C)
    scores = []
    for pool_index in range(n):
        # The k labeled examples nearest in Euclidean distance  [§2.2]
        neighbours = k_nearest(pool_emb[pool_index], labeled_emb, k)
        # KL(neighbour || candidate) for each neighbour  [§2.1] [alg. 1]
        kl_values = []
        for labeled_index in neighbours:
            kl_values.append(kl(labeled_prob[labeled_index],
                                pool_prob[pool_index]))
        # Mean over the neighbourhood is the score  [alg. 1] [§5]
        scores.append(mean(kl_values))
    # The b largest scores form the batch  [alg. 1]
    return top_b_indices(scores, b)

The surrounding loop, with the simulated labels. Starting-set and budget rules are in §2.2 and §3.3.

# Simulated active learning loop  [§2.2] [§3.3]
# Pool labels exist but are hidden until acquired.

def run_active_learning(train_set, total_budget, b, init_size, seed):
    # Random label-stratified initial set  [§2.2] [§3.3]
    labeled = stratified_sample(train_set, init_size, seed)
    pool = train_set minus labeled
    while len(labeled) < total_budget:
        # Start each round from the pretrained checkpoint, not the last model
        # [code: utilities/trainers.py#L828-L833]
        model = fine_tune_pretrained_bert(labeled)
        # Score the pool and select a batch  [alg. 1]
        batch = cal_select(labeled, pool, b, model, k, phi)
        # Reveal labels and move the batch into the labeled set  [§2.2]
        labeled = labeled + batch
        pool = pool minus batch
    return labeled

Other measures used in the analysis are described in words, with their formulas, in §6.1: an input-space diversity (Jaccard overlap of token sets), a feature-space diversity (inverse mean nearest-neighbour distance of [CLS] vectors), uncertainty (mean predictive entropy of a fully trained model), and representativeness (inverse mean KNN distance with K = 10).

Hyperparameters

Values stated in the paper are cited to the paper. Values taken only from the official code are marked "code default" or "code only" and cited to the code. The paper states no value for epochs, warmup, gradient clipping or early stopping.

Results

Author-reported. Setup: BERT-base; seven datasets for four tasks tab. 1; five seeds app. A.1; out-of-domain (OOD) accuracy is measured on the models trained on the final acquired dataset (15% of the pool) §4.2. The paper does not define the ± value in the table caption.

Dataset statistics as the paper gives them tab. 1. DBpedia counts are after undersampling; the original DBpedia training set has 560K examples app. A.1.

Out-of-domain accuracy of models trained on the datasets acquired by each strategy tab. 2. Each column trains on the first named dataset and tests on the second. Bold marks the best value per column, as in the paper.

Headline statements in the text, all author-reported:

• Out-of-domain: CAL gives the highest SST-2→IMDb accuracy "by a large margin", and the highest QQP→TwitterPPDB accuracy §4.2.
• The authors say CAL is "consistently similar or better" than the other approaches on out-of-domain data §4.2. The table shows Entropy above CAL on IMDb→SST-2, within the spread of both.
• In-domain: CAL is said to be top-performing, especially on DBpedia, PubMed and AG News; Entropy is second overall; BertKM is competitive on SST-2 and QNLI but always below CAL and Entropy; BADGE is described as low; ALPS is close to random except on SST-2 §4.1 fig. 2. The numbers are only in fig. 2.
• The abstract and conclusion say CAL is "better or equal" to the best baseline in all tasks §1 §8. The in-domain claim has no table in the paper.

Analysis, averaged over datasets, on each strategy's full acquired set at 15% of the pool tab. 3 §6.1:

The paper reads these as: CAL and Entropy acquire the most uncertain data; CAL acquires the most representative data; ALPS acquires the most diverse input text §6.2.

Ablations on AG News, in-domain accuracy fig. 3 §5, all described in words:

• Choosing the least-divergent candidates instead (argmin) leaves accuracy unchanged or lower than the first model: evidence that the ranking carries signal.
• Building neighbourhoods from each labeled example instead of each pool example is slightly worse.
• Cross-entropy with gold labels instead of KL is slightly worse.
• Max and median in place of the mean perform similarly to the mean.
• The [CLS] vector is the best feature space; mean embeddings, mean outputs, pretrained-BERT [CLS] and TF-IDF are worse.

Efficiency app. A.2 tab. 4 tab. 5. Runtimes in seconds, one V100. Each cell in tab. 4 is (inference, selection); the values are medians, because the paper could not report every round's runtime tab. 4 caption. Averages in tab. 5 are over all datasets (avg-all), the first three (avg-3), or all except QQP (avg-6).

The text says CAL's inference cost is higher than Entropy's because it needs a forward pass over both the labeled and the pool sets, and that its selection step is cheaper than inference in most cases app. A.2.

Paper vs code

The official repository is mourga/contrastive-active-learning at the commit above. The paper links it §1, footnote. It is GPL-3.0 licensed, so this card cites the code and writes its own pseudocode; it quotes no code. Files used: acquisition/cal.py, run_al.py, utilities/trainers.py.

• Distance space. The paper says Euclidean distance on [CLS] vectors §2.2. The code L2-normalises the labeled and pool embeddings first cal.py. For unit-length vectors, Euclidean ranking equals cosine ranking, so the neighbours can differ from the paper's stated method.
• Model at each round. The paper says the model is "fine-tuned at each AL iteration using the current D_lab" §2.2. The code builds a new model from the pretrained checkpoint on every round trainers.py. The paper's wording does not say which.
• Percentages. The paper gives the budget as 15% "of D_pool" §3.3. The code computes the budget, the initial set and the acquisition size as percentages of the full training set after DBpedia undersampling run_al.py run_al.py. The difference is small on the first round but changes the absolute counts.
• Acquisition size. The paper gives b = 2% §3.3. The complexity section uses k = 100 as the acquisition size app. A.2 and gives QQP's acquisition size as 3270 app. A.2. 3270 is 1% of the QQP training count in tab. 1 (327K), not 2%. The paper does not reconcile these.
• Max sequence length. The paper says 128, and 256 for IMDb app. A.1. The code default is 256 for every dataset run_al.py, and the README's only example sets no length [README.md]. The paper's runs cannot be pinned to one value from the code alone.
• Neighbour count. The paper never gives k for CAL. The code default is 10 run_al.py. Treat 10 as code only.
• Evaluation frequency. The paper says 5 evaluations per epoch §3.3. The code computes the interval as one fifth of the epoch in steps, capped at 500 trainers.py. Above about 40K labeled examples (2,500 steps per epoch at batch 16) the cap gives more than 5 per epoch.
• Retry branch. The code can retrain from scratch up to three times when validation accuracy drops by more than 0.5 trainers.py. Accuracy is a fraction in this code, so the branch almost never fires. The paper does not mention it.
• Ablation path. The labeled-centroid ablation (--knn_lab) calls faiss, but its import is commented out cal.py and it is used at cal.py. As shipped, that path cannot run.

No discrepancies were found in the KL direction, the top-b selection, or the mean aggregation: the code matches alg. 1 and §2.1.

Pitfalls

• KL direction. The score is KL(neighbour ‖ candidate). Swapping the arguments changes the ranking. The code uses the paper's order cal.py.
• Unbounded scores. A neighbour's probability mass on a class the candidate almost rules out makes the KL term very large, because the candidate's log-probabilities are unbounded below. A few confident-but-different candidates can dominate the top b. The paper's ablations do not test this.
• Ties. The top-b choice uses an argpartition, so among equal scores which examples are picked is arbitrary cal.py.
• Batch independence. Each candidate is scored on its own; nothing penalises picking near-duplicates within one batch. The paper does not discuss this.
• Cost grows with the labeled set. Each round needs a forward pass over the labeled set and the pool, plus a neighbour search over the labeled set app. A.2. The paper calls the cost linear in pool size app. A.2; the timings are on one V100 and may not transfer.
• Labels are simulated. The pool's labels exist and are hidden until an example is acquired §2.2, footnote. Human-labeling effects are not studied.
• Gold labels are not used by default. The default score uses the model's predicted distributions of the neighbours, not their gold labels. The gold-label variant is an ablation §5.
• Variance is unclear. The ± values are not defined in the table caption, and the paper gives only five seeds app. A.1. Overlapping ± ranges mean a claim of "best" is weak evidence.
• Wording vs numbers. The abstract and conclusion say "better or equal" in all tasks §1 §8. Table 2 has Entropy above CAL on IMDb→SST-2, and the in-domain claims rest on a figure.
• Table 1 counts. The DBpedia train and validation counts are after undersampling, not the original DBpedia split app. A.1.
• Overloaded k. The complexity section uses k for the acquisition size app. A.2, while Algorithm 1 uses k for the neighbour count alg. 1. Keep them apart in an implementation.
• Fresh vs continued training. The paper's "fine-tune at each iteration" can be read as continuing from the previous model. The code restarts from the pretrained checkpoint every round trainers.py. Results will differ between these.

Lineage

• Builds on: uncertainty sampling, least confidence and predictive entropy §7; contrastive learning is cited as the inspiration for pairing similar inputs with different outputs §2.1.
• Compared against: Random, Entropy, BertKM and BADGE (Ash et al., 2020), and ALPS (Yuan et al., 2020) §3.2.
• Superseded by: not stated in this paper; no later work is linked from this wiki.