Adversarial-Learned Loss for Domain Adaptation

In plain words

A model trained on labelled data from one setting often performs worse on data from a different setting, such as images with other lighting or backgrounds. Adapting it to the new setting without labels for the new data is called domain adaptation. Two common approaches exist. One makes the model's internal features look alike across the two settings, using an adversary that tries to tell them apart. The other lets the model label the new data itself and trains on its own confident guesses, which is called self-training. Each approach has a weakness: the first does not make the new data's features sharp enough to classify, and the second's guesses are sometimes wrong.

The paper keeps the guesses but corrects them. A second network estimates, for each class, how often a guess of that class is right. These estimates form a small table that is multiplied with the guess, giving a corrected label that becomes the training target for the new data. The correcting network is itself trained adversarially, so that the corrections help the two settings' features look alike. The authors report that this combination beats earlier methods on four standard benchmarks.

TL;DR

ALDA is an unsupervised domain adaptation method for classification. It replaces the pseudo-label loss on unlabelled target data with an unhinged loss weighted by a corrected label vector. The corrections come from a confusion-matrix estimate that a discriminator learns adversarially, with a gradient reversal layer. Author-reported results: Office-31 average 88.7% (ResNet-50), Office-Home 66.6% (ResNet-50), digits 97.6% (threshold 0.6). No VisDA training code is released.

Problem

Unsupervised domain adaptation (UDA) must transfer a classifier from a labelled source domain to an unlabelled target domain § Introduction. Domain-adversarial learning aligns feature distributions with a theoretical guarantee, but it does not check whether target features are discriminative for the classifier § Introduction. Self-training with pseudo-labels makes target features more discriminative but aligns the domains only implicitly, with no theoretical guarantee; it can reduce performance for shallow networks § Introduction.

The paper's analysis shows that the cross-entropy loss on pseudo-labels differs from the ideal loss (which uses the unknown true label) by a confusion matrix conditioned on the prediction § Methods (Preliminaries) eq. 2–eq. 4. The noisy-label literature handles such a matrix under uniform noise independent of the input. The authors state that these assumptions do not hold for pseudo-labels § Methods (Preliminaries).

Method

Notation. A generator G gives features, a classifier C predicts K-class probabilities, and a discriminator D outputs one value per class. Source samples (x_s, y_s) are labelled; target samples x_t are not § Methods (Preliminaries). Because the paper has no section numbers (the LaTeX sets section numbering to zero), sections are cited by title, for example [§ Methods] (Setup). Equation numbers were counted from the LaTeX source in display order and were not checked against the rendered PDF.

Step 1: Ideal and pseudo-label losses. The ideal target loss weights a basic loss by the true label probability eq. 1. Replacing the true label with the thresholded prediction (the pseudo-label, kept only when its confidence exceeds a threshold delta) gives a sum with a confusion matrix eta, where eta[k, l] = p(y = k | yhat = l, x) eq. 2–eq. 4 § Methods (Preliminaries). The paper calls c_k = sum_l eta[k, l] p(yhat = l | x) the corrected label vector § Methods (Preliminaries).

Step 2: Class-wise uniform noise. To make eta tractable, the paper assumes noise is class-wise uniform with a vector xi. Definition 1 sets eta[k, k] = xi_k and eta[k, l] = (1 - xi_l) / (K - 1) for k different from l § Methods (Adversarial-Learned Loss), Definition 1. Column l sums to one, so each column is a valid conditional distribution. With a hard pseudo-label yhat, the corrected vector reduces to one column of eta: it equals xi_yhat at the predicted class and (1 - xi_yhat) / (K - 1) elsewhere.

Step 3: Discriminator outputs. D maps the feature G(x) to K scores, and a sigmoid per class gives xi(x) = sigmoid(D(G(x))). Each component is read as the probability that the pseudo-label for that class is correct § Methods (Noise-correcting Domain Discrimination).

Step 4: Adversarial term. For source samples, the discriminator pushes the corrected vector c(x_s) toward the one-hot ground truth y_s, using binary cross-entropy summed over classes eq. 6–eq. 7. For target samples, it pushes c(x_t) toward the opposite distribution u(yhat_t), which is 0 at the predicted class and 1 / (K - 1) elsewhere eq. 8. The total adversarial term sums both eq. 9 § Methods (Noise-correcting Domain Discrimination). Source and target use the same discriminator, so the same network learns the noise estimate for both domains § Methods (Noise-correcting Domain Discrimination).

Step 5: Regularisation of the discriminator. The discriminator also gets a cross-entropy term on the source samples, computed from its K outputs with a softmax, so it also classifies source data eq. 10 § Methods (Regularization Term). The discriminator minimises adversarial plus regularisation loss eq. 11. The generator maximises the adversarial term through a gradient reversal layer fig. 2 § Methods (Noise-correcting Domain Discrimination).

Step 6: Corrected target loss. The paper uses the unhinged loss L_unh(p, k) = 1 - p_k as the basic loss eq. 5 § Methods (Corrected Loss Function). The target loss is sum_k c_k L_unh(p_t, k) eq. 12–eq. 13. The paper argues the unhinged loss is robust to uniform noise, which removes the uniform part of the noise § Methods (Preliminaries) § Experiments (Analysis).

Step 7: Training objectives. The classifier minimises source cross-entropy plus lambda times the corrected target loss eq. 14. The generator minimises the same expression but with the adversarial term subtracted eq. 15. The trade-off lambda lies in [0, 1] and follows a sigmoid ramp over training progress q § Methods (Setup) § Methods (Corrected Loss Function).

Step 8: Theory. Theorem 1 states that at the optimum of the max-min adversarial objective eq. 16, the source and target feature distributions are aligned. Theorem 2, under an assumed optimal labelling function on the feature space, states that the target corrected vector equals h at the predicted class (one half at the predicted class, 1 / (2K - 2) elsewhere) when the prediction is correct, and equals u otherwise § Theoretical Insight (Theorem 2). Proofs are in the supplementary material, which also contains the training-accuracy and pseudo-label-weight plot [supp. p. 2].

Pseudocode for one training step, following the paper's notation and the official code's logic (the code is cited in Paper vs code):

# ALDA losses for one training step  [§ Methods] (Corrected Loss Function)
# Inputs: x_s, y_s (source batch), x_t (target batch), delta, lam
# Outputs: loss_c, loss_g, loss_d (one objective per player)

def corrected(xi, yhat, num_classes):
    # Column yhat of the class-wise uniform matrix  [Definition 1]
    # c[k] = xi[yhat] if k == yhat, else (1 - xi[yhat]) / (K - 1)
    xi_hat = gather(xi, yhat)                   # (B, 1)
    hot = one_hot(yhat, num_classes)            # (B, K)
    off = (1 - xi_hat) / (num_classes - 1)      # (B, 1)
    return hot * xi_hat + (1 - hot) * off       # (B, K)

def alda_losses(x_s, y_s, x_t, delta, lam, num_classes):
    # 1. Features and class probabilities per domain  [§ Methods] (Preliminaries)
    feat_s = G(x_s)                             # (B, d)
    feat_t = G(x_t)                             # (B, d)
    prob_s = softmax(C(feat_s))                 # (B, K)
    prob_t = softmax(C(feat_t))                 # (B, K)

    # 2. Hard pseudo-labels; target ones are kept only above delta
    #    [§ Methods] (Preliminaries)
    yhat_s = argmax(prob_s.detach())            # (B,)
    yhat_t = argmax(prob_t.detach())            # (B,)
    keep_t = max(prob_t.detach()) > delta       # (B,) bool

    # 3. Per-class noise estimates  [§ Methods] (Noise-correcting Domain
    #    Discrimination)
    xi_s = sigmoid(D(feat_s))                   # (B, K)
    xi_t = sigmoid(D(feat_t))                   # (B, K)

    # 4. Corrected label vectors  [§ Methods] (Adversarial-Learned Loss)
    c_s = corrected(xi_s, yhat_s, num_classes)  # (B, K)
    c_t = corrected(xi_t, yhat_t, num_classes)  # (B, K)

    # 5. Adversarial term: source toward its true label  [eq. 6]–[eq. 7]
    loss_adv_s = bce(c_s, one_hot(y_s, num_classes)).sum(axis=1)

    # 6. Adversarial term: confident targets toward the opposite
    #    distribution  [eq. 8]
    u_t = (1 - one_hot(yhat_t, num_classes)) / (num_classes - 1)
    loss_adv_t = bce(c_t, u_t).sum(axis=1) * keep_t

    # 7. Discriminator regulariser on source samples  [eq. 10]
    loss_reg = cross_entropy(D_logits(feat_s), y_s)

    # 8. Corrected unhinged target loss; c_t is a constant here
    #    [eq. 12]–[eq. 13]  [code: loss.py#L112]
    c_const = c_t.detach()
    loss_t = sum(c_const * (1 - prob_t), axis=1) * keep_t

    # 9. Discriminator minimises adversarial plus regulariser  [eq. 11]
    loss_d = mean(loss_adv_s + loss_adv_t) + mean(loss_reg)

    # 10. Classifier: source cross-entropy plus weighted target loss
    #     [eq. 14]
    loss_src = cross_entropy(C(feat_s), y_s)
    loss_c = loss_src + lam * mean(loss_t)

    # 11. Generator: as the classifier, but the adversarial term is
    #     reversed by the gradient reversal layer  [eq. 15]
    adv = mean(loss_adv_s + loss_adv_t)
    loss_g = loss_src + lam * mean(loss_t) - lam * adv
    return loss_c, loss_g, loss_d

# Schedules  [§ Methods] (Setup)

def lam_schedule(q):
    # lam(q) = 2 / (1 + exp(-10 q)) - 1, with q in [0, 1] as progress
    return 2.0 / (1.0 + exp(-10.0 * q)) - 1.0

def lr_schedule(eta_0, q):
    # eta = eta_0 / (1 + 10 q) ** 0.75, with the same progress q
    return eta_0 / (1.0 + 10.0 * q) ** 0.75

Hyperparameters

Values for the ResNet-50 experiments (Office-31, Office-Home, VisDA) unless noted. The paper states no batch size, training length or weight decay; those rows list the code's values and are marked as code-only.

Other training detail (code): ResNet-50 runs use separate forward passes for source and target batches, so batch-norm statistics are computed per domain train.py train.py.

Results

Author-reported accuracy (%) on the target domain. Setup: ResNet-50 (ImageNet pretrained) for Office-31, Office-Home and VisDA; digit networks for digits. Baselines are the paper's own. Digits results are the mean of three runs with variance § Experiments (Result). The number of runs is not stated for the other tables. Values with ± are copied as printed.

Office-31, ResNet-50 tab. 1

Office-Home, ResNet-50 tab. 2 (12 tasks; columns in the order the paper lists them: Ar, Cl, Pr, Rw domains as source→target)

VisDA-2017 tab. 3 (accuracy per class; columns C1 to C12 follow the paper's class order, and the paper prints abbreviated class names, one of them as "house". The table lists both ResNet-101 and ResNet-50 blocks; the text says ALDA reports both backbones)

Digits tab. 4 (three runs, mean ± variance)

Ablation on Office-31, ResNet-50 tab. 5

Headline claims: ALDA has the highest average on Office-31, Office-Home and the ResNet-50 VisDA block, and on the digit average at delta=0.6 § Experiments (Result). Against CDAN+E on Office-31, ALDA is higher on four of six tasks, ties on W→D and is lower on D→W tab. 1. Averages were recomputed from the printed columns; see Pitfalls for the mismatches.

Paper vs code

Official code: https://github.com/ZJULearning/ALDA, cloned at commit 09df8864fde9d25d25ad22e8c62f135d3d7596d5. The repository is linked from the paper and its abstract. It has no LICENSE file, so no licence can be recorded and the code is cited only, with no verbatim snippets. The paper's supplementary material is a separate PDF and was read for the proofs.

Discrepancies found, each with both citations:

• Base learning rate. The paper sets eta_0 = 0.01 § Methods (Setup). The code defaults to 0.001 train.py, uses 0.001 for Office-Home train.py and for most Office tasks, and 0.0003 for A→D and D→W train.py.
• Decay horizon. The paper's decay uses training progress q with alpha = 10 § Methods (Setup). The code decays by 0.001 per iteration train.py lr_schedule.py. These agree only if a run has 10,000 iterations. Office runs stop at 20,000 train.py, so the code's decay is faster than the paper's formula would give.
• Lambda ramp. The paper ramps lambda with progress q over the whole run § Methods (Setup). The code evaluates the ramp at the iteration index with max_iter 10,000, so it saturates at iteration 10,000 whatever the run length network.py train.py. The gradient-reversal coefficient inside the discriminator uses the same ramp network.py network.py.
• Digit losses. The paper says lambda is ramped in all experiments § Methods (Setup). The digit scripts weight both the adversarial and the corrected loss by 1, with no lambda train_svhnmnist.py train_uspsmnist.py.
• Digit learning rate. The paper says Adam with lr 1e-3 for digits § Methods (Setup). The code defaults to 2e-4 train_svhnmnist.py, uses 2e-4 for U→M and 1e-3 for M→U train_uspsmnist.py.
• Weight decay and momentum. The paper gives SGD momentum 0.9 only § Methods (Setup). The code uses Nesterov momentum and weight decay 5e-4 times a multiplier of 2, which becomes 1e-3 after the scheduler sets it train.py lr_schedule.py.
• Bottleneck and discriminator width. The paper mentions neither a bottleneck layer nor the discriminator width. The code uses a 512-dimensional bottleneck network.py and a 1024-wide discriminator for ResNet train.py, and a 500-wide discriminator for digits train_svhnmnist.py.
• Correction vector gradient. The paper does not say whether gradients reach the discriminator through the corrected target loss. The code treats the correction as a constant there loss.py.
• Constant in the unhinged loss. The paper's target loss is sum_k c_k (1 - p_k) eq. 13. The code computes the negative of sum_k c_k p_k loss.py. The two differ by a constant, because the correction vector sums to one by construction, so the gradients are the same.
• Pseudo-label form. The paper writes p(yhat = l | x) as a probability eq. 3. The code uses the one-hot predicted label loss.py.
• Threshold on the adversarial target term. The paper's target adversarial term eq. 8 has no mask, but the setup says samples below the threshold are ignored § Methods (Setup). The code masks the target adversarial term and the corrected loss by the threshold loss.py loss.py.
• Training length and checkpoint choice. The paper gives no training length or checkpoint rule. The code runs 10,000 iterations by default train.py, 20,000 for Office train.py and 10,000 for Office-Home train_home.sh. It evaluates every 500 iterations on the target test list and keeps the best checkpoint by that accuracy train.py. For these runs the test list is the target list train.py.
• Test-time averaging. The paper does not state a test protocol. The code averages softmax outputs over 10 crops train.py pre_process.py.
• Digit networks. The paper refers to the networks of [self-ensembling] § Methods (Setup). The code implements them as SVHN_EnsembNet and USPS_EnsembNet network.py network.py.
• VisDA. The paper reports VisDA results tab. 3. The released code raises an error for the visda dataset train.py, and the README says the VisDA code is still in progress. The VisDA numbers cannot be reproduced from this repository.

Runnability as shipped: the README targets Python 3.6 and PyTorch 1.0.0 [README]. requirements.txt pins numpy 1.14.6, and network.py uses np.float network.py, which was removed from NumPy in 1.24. The repository does not include the image datasets; they must be downloaded as the README describes.

Pitfalls

• Equation numbers. They were counted from the LaTeX source. The paper has no section numbers and its equations are numbered only in display order, so check them against the rendered PDF before relying on them.
• Shared discriminator outputs. One discriminator gives both the per-class noise (through a sigmoid) and the logits of the regulariser (through a softmax in the cross-entropy). The code applies the regulariser to the raw logits loss.py loss.py.
• Column sums. The sigmoid outputs are independent per class, so the noise vector is not normalised. Definition 1 makes each column of the confusion matrix sum to one by construction, and the code builds the matrix that way loss.py.
• Model selection on the target test set. The best checkpoint is picked by target test accuracy every 500 iterations train.py, and the target test list is the unlabelled target training list train.py. Reported numbers therefore depend on that selection. The paper does not describe it.
• Threshold chosen on the test table. The paper picks the digit threshold from the reported test accuracies § Experiments (Result) tab. 4. It does not describe a separate validation split.
• Small margins and a claim. The paper says ALDA "significantly" outperforms the state of the art § Experiments (Result). The Office-31 average is 88.7 against 87.7 for CDAN+E (1.0 point), and the Office-Home average is 66.6 against 65.8 (0.8 point). Office-31 D→W is 97.7, below CAN's 98.8 tab. 1. Runs-to-run variance is shown only for some rows, and the number of runs is not stated for Office.
• Averages that do not match their columns. Recomputing the averages from the printed columns gives 87.5 for the ALDA+ST ablation row (printed 87.8) tab. 5, and 82.8 for ADDA (printed 82.9) tab. 1. The other averages in these tables match to rounding.
• VisDA comparison. The VisDA text says ALDA performs better than other methods, but the ResNet-50 ALDA score (76.5) is compared with CBST on ResNet-101 (76.4) tab. 3 § Results (Image Results). The backbones differ.
• Unhinged loss and noise. The argument that the unhinged loss is robust to uniform noise is cited to other papers § Methods (Preliminaries). It is not re-derived here. The paper's cross-entropy variant averages 87.6 against 88.7 for ALDA tab. 5; the number of runs per ablation row is not stated.
• Theorems depend on assumptions. Theorem 1 assumes the two feature distributions have densities and that the max-min objective reaches its optimum; Theorem 2 further assumes an optimal labelling function on the feature space. Empirically the paper checks only the t-SNE figure fig. 3 and the supplementary weight plot [supp. p. 2].
• Adversarial training cost. Each step needs two forward passes of the generator (source and target) and one of the discriminator, plus a separate backward pass for the regulariser with the generator frozen train.py. Cost per step is not reported by the paper.
• Figures. No paper figure is reproduced here. The t-SNE plot fig. 3 and the framework and noise-correcting diagrams fig. 1 fig. 2 are described only.

Lineage

• Builds on: domain-adversarial learning with gradient reversal (DANN), used for the discriminator and the GRL § Methods (Noise-correcting Domain Discrimination); self-training with pseudo-labels (CBST) § Methods (Preliminaries); confusion-matrix correction for noisy labels, which the authors adapt to pseudo-labels § Introduction; unhinged loss for uniform noise § Methods (Preliminaries).
• Setup borrowed from: the learning-rate schedule and gradient-reversal form of DANN, and the lambda ramp and conditional discriminator setup of CDAN § Methods (Setup).
• Compared against: ResNet-50, DANN, ADDA, JAN, MADA, CBST, CAN, CDAN+E, MCS and TAT (Office and digits); MCD and CBST (VisDA and digits); Mean Teacher with a confidence threshold (digits) tab. 1–tab. 4.
• Superseded by: not stated.