Partial label learning

Overview

In partial label learning, each training example comes with a set of candidate labels that contains the true ones plus some wrong ones, and the wrong ones cannot be told apart from the true ones by annotation alone. Crowd-sourced image tags are a typical case: a worker ticks every tag that might apply 2610.17635 §1.

The wiki has one card for this slug, and it covers the multi-label form: each image has several candidate tags, and each tag is either correct or not. The card's method weights every candidate positive by an estimate of the chance that it is a true positive, given the model's confidence in it 2610.17635 §1 2610.17635 §3. Negatives keep weight 1 2610.17635 alg. 1. Single-label partial label learning has no card in the wiki yet, so nothing on this page applies to it.

Methods in this area fall into two groups. The first group uses a fixed weighting rule. The second group estimates the weight from data. For the memorisation and small-loss ideas behind both groups, see noisy labels 2610.17635 §2.

Variants

Hand-set weighting rules

Each candidate's loss is multiplied by a fixed function of its confidence t. The paper compares three of these in its ablation 2610.17635 §5.3:

• Hard threshold: weight 1 above a cut-off of 0.6, otherwise 0 2610.17635 §5.3.
• Soft weighting: weight t² 2610.17635 §5.3.
• Identity weighting: weight t 2610.17635 §5.3.

The paper argues that such rules have no stated optimum, and that the best threshold moves with the dataset, the noise rate and the epoch 2610.17635 §1. Its COCO run at 80% noise, epoch 1, shows the three rules giving nearly the same mAP (59.79, 60.08 and 59.93). Their error in the weights differs much more, at 92.96, 61.68 and 76.78 2610.17635 tab. 2. Card: Calibrated Disambiguation for Partial Multi-label Learning.

Calibrated weights from a pre-trained reliability estimator (PML-CD)

The weight for a candidate is the reliability of its confidence bin. The method works in two phases 2610.17635 §4 2610.17635 alg. 1:

1. Warm-up: train with plain binary cross-entropy on the raw candidate labels until early stopping.
2. Disambiguation: each epoch, build a 20-bin histogram of confidences on candidate positives 2610.17635 eq. 5. A small two-layer network maps the histogram to a reliability per bin 2610.17635 §4.1. Each candidate's weight interpolates between the two neighbouring bins 2610.17635 eq. 7. Train on the weighted loss 2610.17635 eq. 9.

The network is trained once, on VOC 2007, with identity-weighted runs at noise rates from 10% to 90%. It is then used as a fixed plug-in for other datasets and noise rates 2610.17635 §5.1 2610.17635 §4.1. The paper reports the best mAP in 15 of 16 settings against nine baselines 2610.17635 §5.2 2610.17635 tab. 1. Card: Calibrated Disambiguation for Partial Multi-label Learning.

Prototype-alignment auxiliary loss

A second head scores each class embedding by its cosine to a learned per-class prototype 2610.17635 eq. 10. This loss is added only in the disambiguation phase 2610.17635 eq. 13. Its effect is strongest at high noise. On VOC at 80% noise, mAP rises from 66.91 without either prototype term to 71.70 with the second-phase term only. At 20% on VOC the same term lowers mAP, from 89.28 to 88.77 2610.17635 tab. 3. The paper concludes the term helps mainly at high noise 2610.17635 §5.3.

When to use which

The "best in 15 of 16" claim comes from the authors' own table and is not independently checked 2610.17635 §5.2. The paper does not report the number of runs or seeds behind each cell, and gives no variance 2610.17635 §5.1.

Implementation sketch

Common core of the calibrated variant, in the same style as the paper's algorithm. Variants differ in how the weight is set (see above). The printed interpolation in eq. 7 is off by a factor of 1/K, so use a convex combination of the two neighbouring reliabilities 2610.17635 eq. 7.

# Two-phase training  [arxiv:2610.17635 alg. 1] [arxiv:2610.17635 §4]
# Input: images, partial labels in {0, 1} (1 = candidate)
# Output: network with a classifier head and a prototype head

def train(images, partial_labels, calibrator):
    # Warm-up: plain BCE on the raw candidates  [arxiv:2610.17635 eq. 12]
    weights = ones_like(partial_labels)
    while not early_stopping():
        clf, proto = network(images)
        update(network, weighted_bce(clf, partial_labels, weights))

    # Disambiguation: recompute weights every epoch  [arxiv:2610.17635 §4.1]
    while not early_stopping():
        # Confidences on the training set, with no gradient  [alg. 1]
        conf = no_grad(network_clf(images))
        # Per-class histogram over candidate positives only  [eq. 5]
        hist = histogram_of_candidates(conf, partial_labels, K=20)
        # Reliability per bin from the pre-trained calibrator  [eq. 6]
        reliability = calibrator(hist)
        # Weight each candidate by interpolating its bin  [eq. 7]
        weights = interpolate(conf, reliability, partial_labels)
        # Classification loss plus prototype loss  [arxiv:2610.17635 eq. 13]
        clf, proto = network(images)
        loss = weighted_bce(clf, partial_labels, weights)
        loss = loss + weighted_bce(proto, partial_labels, weights)
        update(network, loss)
    return network

For the hand-set variant, replace the calibrator step with a fixed function of conf, such as conf itself (identity) or conf**2 (soft) 2610.17635 §5.3.

Contradictions

• Table 1 and Table 3 disagree on VOC. The "2nd phase only" VOC row of 2610.17635 tab. 3 differs from the PML-CD VOC row of 2610.17635 tab. 1 at 20%, 40% and 60% noise (88.77 vs 88.83, 88.12 vs 87.12, 85.07 vs 83.87). Only the 80% value matches. The COCO rows match. The paper does not explain the gap 2610.17635 tab. 1 2610.17635 tab. 3.
• Paper and released code differ. The shipped code uses a hard threshold of 0.7, where the paper says 0.6 2610.17635 §5.3. It uses one learning rate of 1e-4 for all datasets, where the paper gives 1e-5 for COCO and NUS-WIDE 2610.17635 §5.1. It also rescales weights and smooths the calibrator targets, and the paper describes neither step. Numbers reproduced from the code may not match the paper's tables. Full list in the card's "Paper vs code" section.
• Identical weight error for two rows. At COCO 80%, the calibrator and the optimal weighting both show a weight error of 3.74 2610.17635 tab. 2. The paper does not discuss this. It may be a coincidence or a copy error.

Papers

• Calibrated Disambiguation for Partial Multi-label Learning: calibrates the weight of each candidate tag with a pre-trained reliability estimator and adds a prototype-cosine loss. Author-reported gains on VOC, COCO, NUS-WIDE and CUB at 20–80% noise.