--- kind: concept slug: domain-adaptation title: Domain adaptation summary: >- Reuse a model trained on a labeled source domain for a different target domain, usually without target labels. Variants adapt the data (translate or pseudo-label target samples), the loss, the features, the retriever or generator, the fine-tuning recipe, the merge or each test input. Each adds cost or fragility, and most gains are author-reported on narrow benchmarks. tags: [transfer, unsupervised, image-translation, segmentation, test-time, language-models, model-selection, self-supervision, toxicity, content-moderation, survey, optimal-transport, remote-sensing, cross-modal, federated, healthcare, fault-diagnosis, source-free, fact-checking, manifold-learning, image-classification, open-domain-qa, uav, materials-science, lidar, cooperative-perception, sim2real] papers: ["1611.02200", "1804.07265", "1807.07258", "1903.12212", "1904.03451", "1905.03198", "1907.09173", "1909.11825", "1909.13589", "2004.14974", "2005.07503", "2010.08240", "2101.10861", "2103.03097", "2103.03606", "2108.00946", "2109.15290", "2111.09832", "2202.10054", "2203.01054", "2204.10377", "2205.09904", "2208.03274", "2209.07522", "2210.02627", "2303.07601", "2303.09136", "2310.17389", "2311.00176", "2311.16079", "2402.10373", "2502.19316", "2606.04665"] related: [contrastive-learning, experience-replay, self-consistency, generative-adversarial-network, image-to-image-translation, semantic-segmentation, self-training, self-supervised-learning, masked-image-modeling, continual-learning, language-model-pretraining, model-merging, domain-generalization, transfer-learning, linear-probing, data-augmentation, semi-supervised-learning, latent-space-editing, vision-language-pretraining, retrieval-augmented-generation, dense-retrieval, test-time-training, low-rank-adaptation, optimal-transport, zero-shot-learning, metric-learning, importance-weighting, federated-learning, personalized-federated-learning, distributional-alignment, entropy-minimization, weight-decay, model-selection, automated-fact-checking, rationale-extraction, active-learning, toxicity-detection, object-detection, convolutional-neural-network, named-entity-recognition, conditional-random-field, collaborative-perception, sim-to-real-transfer] status: draft updated_at: 2026-11-01 --- ## Overview A classifier trained on one data distribution often degrades on another: a digit reader trained on street-number photos misreads clean handwritten digits, and a sentiment model trained on restaurant reviews misreads laptop reviews. Domain adaptation tries to carry the labeled knowledge across without labeling the new domain. The source domain has labels, the target domain may have none [arxiv:1611.02200 §1]. The cards share one idea: some signal links the two domains, and the method uses it. In the translation approach it is a fixed feature function `f` that should give the same output before and after the mapping [arxiv:1611.02200 §1]. In the segmentation approaches the signal is the unlabeled target image itself, through a loss on its predictions [arxiv:1909.13589 §3.2]. In the disentangling approach it is the image's layout, which is assumed to be shared across domains while texture is not [arxiv:1903.12212 §1]. In the language-model approaches it is unlabeled domain text, used for further pre-training [arxiv:2402.10373 §3.2] [arxiv:2311.00176 §2.2] [arxiv:2005.07503 §2] [arxiv:2311.16079 §4.1] [arxiv:2109.15290 §2.2]. In the self-supervision approach it is the unlabeled images themselves, through puzzles whose labels the method makes up [arxiv:1909.11825 §4]. In the aerial translation approach it is the unpaired target images that train the translator [arxiv:1905.03198 §4.1]. In the transport approach it is the match between source and target minibatches, scored by features and predicted labels [arxiv:2103.03606 §5.2]. In the federated approach it is each user's own data, whose feature statistics are aligned to the shared model's [arxiv:1907.09173 §3.4]. The fault-diagnosis and manifold cards use the unlabeled target features directly, through a maximum mean discrepancy (MMD) penalty on pseudo-labels [arxiv:1804.07265 §III-A] or on a kernel fitted to both domains [arxiv:1807.07258 §3.4]. The source-free card uses the unlabeled target images, with the source model's weights as an anchor and no source data [arxiv:2502.19316 §3.2]. A contrastive test-time card uses the same kind of input, the unlabeled target stream, and refines its pseudo labels in a memory queue [arxiv:2204.10377 §3.1]. The SciFact card is the exception. It needs no unlabeled target signal: it trains on a general fact-checking set and then on the labeled scientific set [arxiv:2004.14974 tab. 3]. In the content-moderation approach it is the unlabeled production traffic, which a Wasserstein critic compares with labeled public data on the encoder's embeddings [arxiv:2208.03274 §3.4]. In the vehicle-perception approach it is unlabeled real LiDAR from the target vehicles, which a domain-adversarial head compares with labeled simulated scans [arxiv:2303.07601 §4.3]. What differs between variants is where the adaptation happens: in the data, the loss, the features, the fine-tuning recipe, the merged weights, or at test time for each input. The thirty-three cards cover the approaches below. Their evidence is uneven. Some papers test on one benchmark pair, some report only internal data, two are surveys with no experiments of their own, one survey runs a single experiment, one dataset paper runs one Sim2Real adaptation benchmark, and one is a benchmark that compares training sources but proposes no adaptation method. Read the "When to use which" table before choosing a variant. Domain generalization is a neighbouring setting. A model sees several source domains and no data from the target, not even unlabeled, so it cannot adapt to the target directly [arxiv:2103.03097 §II.A] [§III.A]. The cards keep it separate; see the section below. ## Variants ### Data-based: translate source samples into the target domain The Domain Transfer Network (DTN) learns a map G from the source domain S to the target domain T. It uses unpaired samples from both domains and a fixed, pretrained feature function f, and it is trained so that f(x) and f(G(x)) stay close [arxiv:1611.02200 §1, §3]. G is built as g after f, so only the trainable part g decodes f's representation [arxiv:1611.02200 §4]. Training combines a three-way discriminator, a feature-constancy loss on source samples, an identity loss on target samples and a smoothness term [arxiv:1611.02200 eq. 3–7]. Adaptation then works as follows: G carries the labeled source set into T, the source labels are kept, and a classifier is fit on the translated samples. The paper uses a nearest-neighbour classifier for this step [arxiv:1611.02200 §2, §5.1]. On SVHN to MNIST, the author-reported accuracy is 84.44% when the train split of the source is transferred. The paper's comparison points are 73.85% for DANN, 59.32% for SA, and 79.72% when transferring the SVHN test split [arxiv:1611.02200 tab. 2]. Ablations on the same task show that removing the identity loss drops accuracy to 88.40% and removing the feature-constancy loss to 74.55%, with 90.66% for the full model [arxiv:1611.02200 tab. 1]. The paper also reports that the target set must contain every class. When class 3 is absent from T, accuracy on that class falls to 40.13% [arxiv:1611.02200 tab. 3]. Card: [Unsupervised Cross-Domain Image Generation](/papers/1611.02200). ### Data-based: pseudo-label target pairs with a slow, accurate model Augmented SBERT (AugSBERT) is a text version of the same idea, but the labels come from a model rather than a generator. A BERT cross-encoder is fine-tuned on a labeled source domain. It then scores unlabeled target-domain sentence pairs, and a fast bi-encoder is trained on those scores [arxiv:2010.08240 §3.2]. The scores are soft labels, not hard ones [arxiv:2010.08240 §3.1]. Which pairs are scored matters. Random pairs are mostly negatives [arxiv:2010.08240 §3.1]. BM25 gives the smallest silver sets and the best average score, which the paper credits to its computational efficiency [arxiv:2010.08240 §6.1]. On the domain-adaptation table, AugSBERT's AUC(0.05) for Quora → Sprint is 0.875, against 0.505 for SBERT trained on the source only. That is the "up to 37 points" headline [arxiv:2010.08240 tab. 6]. It is not a uniform gain. For Quora → AskUbuntu it reaches 0.602, against 0.715 for SBERT trained in-domain on the target [arxiv:2010.08240 tab. 6]. On Sprint as target, the paper's own Bi-LSTM baselines are level with or above it, the exception the paper itself notes [arxiv:2010.08240 §6.2]. The public example scripts train on the English STS Benchmark and label Quora pairs, so they do not reproduce the paper's setup [code: examples/sentence_transformer/training/data_augmentation/README.md#L85-L89]. Card: [Augmented SBERT](/papers/2010.08240). ### Data-based: cycle-consistent translation, then fine-tune the segmenter Benjdira et al. adapt a segmenter trained on Potsdam aerial tiles to Vaihingen tiles. A CycleGAN-style pair of translators, one for each direction, is trained on unpaired patches from both sites [arxiv:1905.03198 §4.1]. Its objective has an adversarial term per direction and a reconstruction term that asks each round trip to return the input [arxiv:1905.03198 eq. 8]. The source-to-target translator then restyles every labelled source image, and the source labels are kept [arxiv:1905.03198 §4.2]. The segmenter is fine-tuned on the restyled images, starting from its source weights [arxiv:1905.03198 §4.3]. Unlike the DTN pipeline above, no new classifier is fitted on the translated set. The source model is adapted in place. The source segmenter is a BiSeNet with a ResNet-101 feature extractor, trained with Adam at a learning rate of 0.0001 [arxiv:1905.03198 §5.2.1]. The translators use 512 by 512 patches, and the paper does not state a total loss weight or an identity term [arxiv:1905.03198 §5.2.2] [§4.3]. On the target, average accuracy rises from 0.35 before adaptation to 0.52 after, and IoU from 0.17 to 0.30 [arxiv:1905.03198 tab. 3]. The gain is uneven by class. Building accuracy goes from 0.23 to 0.71 and Tree from 0.06 to 0.51, while Low vegetation falls from 0.38 to 0.27 [arxiv:1905.03198 tab. 4]. The text says the other classes are unaffected [arxiv:1905.03198 §6], which the Low vegetation row does not support. The paper measures the target after each epoch and reports the gain at epoch 8 [arxiv:1905.03198 §5.2.4]. That choice uses target test labels, so the reported best-epoch numbers are optimistic. Results are for one source and one target pair with one segmenter. Card: [Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Images](/papers/1905.03198). ### Loss-based: maximum squares on unlabeled target pixels For segmentation with a labeled synthetic source (GTA5 or SYNTHIA) and an unlabeled real target (Cityscapes), the objective is the source cross-entropy plus a target term weighted by λ_T [arxiv:1909.13589 §3.1] [eq. 1]. The common target term is entropy minimization. Its gradient is largest on the pixels the model already predicts confidently, so those pixels dominate the update [arxiv:1909.13589 §1] [§3.2]. The paper replaces that term with the negative sum of squared target probabilities, normalised per image [arxiv:1909.13589 §3.2] [eq. 6]. It needs no discriminator network [arxiv:1909.13589 §1]. Two add-ons follow. Image-wise class weights use the predicted class counts of each target image, with a hyperparameter α [arxiv:1909.13589 §3.3] [eq. 13]. A second, low-level head is trained on confident guidance labels built from both heads' predictions, with threshold δ and weight λ_low [arxiv:1909.13589 §3.4] [§4.2]. On GTA5 to Cityscapes (mIoU over 19 classes, DeepLabv2 with ResNet-101), the author-reported numbers are 42.2 for their entropy-minimization reproduction, 44.3 for maximum squares, 45.2 with image-wise weights and 46.4 with the low-level head [arxiv:1909.13589 tab. 2] [tab. 4]. On SYNTHIA to Cityscapes, the full method reaches 41.4 over 16 classes, against 38.0 for entropy minimization [arxiv:1909.13589 tab. 3]. The paper also reports classification with ResNet-50 on Office-31: 85.6 average for maximum squares, 84.1 for entropy minimization, 82.2 for DANN and 76.1 for the source-only model [arxiv:1909.13589 tab. 1]. The paper calls those classification results uncompetitive with the state of the art [arxiv:1909.13589 §4.3]. Card: [Domain Adaptation for Semantic Segmentation with Maximum Squares Loss](/papers/1909.13589). ### Loss-based: label-free puzzles on both domains Sun et al. train one shared network on the labeled source task and on self-supervised tasks that use both domains. The tasks are rotation (4-way), vertical flip (binary) and patch location, and each head is a linear layer on the shared features [arxiv:1909.11825 §3] [§4]. The source labels are dropped in the puzzle losses, and the target images only feed those losses [arxiv:1909.11825 eq. 1]. The total loss is an unweighted sum of the source task and the puzzle losses [arxiv:1909.11825 eq. 2]. The paper uses no minimax or adversarial term [§4]. The puzzle heads are discarded at test time [§4]. The appendix trains one puzzle at a time, one gradient step per task on balanced batches of 64 source and 64 target images, then one supervised step on source [arxiv:1909.11825 app. B]. Per source batch, the cost is one update for each puzzle and one for the classifier. The paper reports best accuracy on four of seven digit and natural-image benchmarks. On MNIST to MNIST-M, rotation alone reaches 98.9 against 44.9 for source-only. On MNIST to USPS it reaches 96.5 against 94.7 [arxiv:1909.11825 tab. 2]. On CIFAR-10 to STL-10 the full set of tasks reaches 82.1, against 75.6 for source-only [arxiv:1909.11825 tab. 2]. The paper states that its method fails on the SVHN benchmarks, where rotation gives trivial solutions because the crop shows neighbouring digits [arxiv:1909.11825 §5.1]. On GTA5 to Cityscapes the mIoU is 28.9 against 25.3 for source-only, and 39.5 for CyCADA [arxiv:1909.11825 tab. 3]. The epoch is picked by a score that adds the normalised feature-mean distance on unlabeled data to the normalised source error [arxiv:1909.11825 §4.1] [eq. 3]. The public code selects the checkpoint on the test splits, not on validation data [code: utils/train.py#L38-L41]. Card: [Unsupervised Domain Adaptation through Self-Supervision](/papers/1909.11825). ### Loss-based: minibatch unbalanced optimal transport (JUMBOT) Fatras et al. align source and target minibatches with an optimal transport plan and train the network on the plan-weighted cost [arxiv:2103.03606 §5.2] [eq. 10]. Each pair of samples costs a squared feature distance plus the cross-entropy of the source label under the target prediction [arxiv:2103.03606 eq. 9]. Balanced transport must send all mass, so inside a small minibatch it pairs samples that do not correspond. The method replaces it with unbalanced transport, which charges a KL penalty on the marginals and adds entropic regularisation [arxiv:2103.03606 §2] [eq. 2]. The paper shows that the minibatch estimator stays unbiased and that its gradients behave under this change [arxiv:2103.03606 §4.1] [§4.3]. The paper reports state-of-the-art results with one pair of minibatches per step (k = 1) [arxiv:2103.03606 §5.2]. Source minibatches are class-stratified [arxiv:2103.03606 app. C]. The settings differ by task. For Office-Home the paper uses feature weight 0.01, label weight 0.5, transfer weight 1, τ = 0.5, ε = 0.01 and minibatch 65. For digits it uses τ = 1, ε = 0.1 and minibatch 500 [arxiv:2103.03606 app. C]. The same method covers partial domain adaptation, where the target has fewer classes [arxiv:2103.03606 §5.3]. On Office-Home DA, the end-of-training average is 70.0, against 66.6 for CDAN-E and 63.5 for DeepJDOT, both reproduced by the authors [arxiv:2103.03606 tab. 1]. On digits the average is 98.0, against 96.5 for ALDA [arxiv:2103.03606 tab. 2]. On VisDA the score is 72.5, against 70.5 for ALDA [arxiv:2103.03606 tab. 3]. The appendix reports Office-Home again with the best score over epochs, where JUMBOT gets 70.1 and CDAN-E 67.0 [arxiv:2103.03606 app. C, Table 4]. That protocol is not the one of the main table, so the two should not be mixed. Card: [Unbalanced minibatch Optimal Transport; applications to Domain Adaptation](/papers/2103.03606). ### Loss-based: MMD penalty on features, marginal and per class (JDA in a CNN) Han et al. call their model a deep transfer network (DTN). It is not the Domain Transfer Network above. A CNN is trained on labeled source data. Its loss then adds lambda times a joint distribution adaptation (JDA) term [arxiv:1804.07265 eq. 11]. The JDA term is the maximum mean discrepancy (MMD) between all source and target features, plus one MMD for each class between source samples of that class and target samples with that pseudo-label [arxiv:1804.07265 eq. 10]. The pseudo-labels come from the current network and are recomputed each round [arxiv:1804.07265 alg. 1]. The paper writes the marginal and class-wise terms with weight 1 each, so lambda is the only scale [arxiv:1804.07265 eq. 10]. The paper also tests the marginal-only version, called DTN with MDA. It drops the class-wise terms [arxiv:1804.07265 §III-D]. Across ten fault-diagnosis transfer tasks on three data sets, the average accuracy is 97.5 for DTN with JDA, 87.4 for DTN with MDA and 80.0 for the plain CNN [arxiv:1804.07265 tab. III]. These are the authors' numbers, on one fault-diagnosis setting. The source and target label spaces are assumed equal [arxiv:1804.07265 §III-A]. The lambda grid is listed in the card's hyperparameters, but the paper does not say which data chose it. Treat the figures as optimistic until that is clear [arxiv:1804.07265 §IV-A]. The paper names no code repository, and a search found no official code [arxiv:1804.07265 §IV-A]. Card: [Deep Transfer Network with Joint Distribution Adaptation](/papers/1804.07265). ### Feature-based: split structure from texture DISE (Domain-Invariant Structure Extraction) encodes each image into two codes. A structure code, shared by both domains, drives the segmentation classifier. A texture code, private to each domain, carries colour and surface look [arxiv:1903.12212 §3.1]. A decoder rebuilds images from both codes. Swapping the texture codes between a game image and a real one gives a translated image that keeps the game layout but has the real look [arxiv:1903.12212 §3.1]. Those translated images carry the source labels as pseudo-labels, because segmentation is assumed to depend only on structure [arxiv:1903.12212 §3.1] [§3.2]. The target labels are never used in the loss [arxiv:1903.12212 §3.2]. The model has five sub-networks, plus discriminators on the output space and on the translated images [arxiv:1903.12212 §3.3] [§3.2]. The author-reported result on GTA5 to Cityscapes is 45.4 mIoU with a DeepLab-v2 ResNet-101 base, against 42.4 for the output-space adaptation baseline in the same table [arxiv:1903.12212 tab. 2]. The loss weights are not stated in the paper [arxiv:1903.12212 eq. 1]. Card: [All about Structure](/papers/1903.12212). ### Feature-based: gradient-reversal domain loss across two modalities Dey et al. train two encoders, one for sketches and one for photos, into a shared embedding space [arxiv:1904.03451 §4.2]. A small classifier tries to tell which modality an embedding came from. A gradient-reversal layer sits in front of it, so the encoders learn embeddings the classifier cannot separate [arxiv:1904.03451 §4.3] [eq. 2]. The reversal weight is zero for the first few steps and then ramps up [arxiv:1904.03451 alg. 1]. The method is domain adaptation in the same adversarial form, but the two domains are two input types, not a labelled source and an unlabelled target. The test categories are unseen in training, so the setting is zero-shot retrieval [arxiv:1904.03451 §4.1]. In the paper's ablation, adding the domain term to the attention model moves mAP from 0.3207 to 0.3256 on Sketchy-Extended, from 0.0729 to 0.0845 on TU-Berlin-Extended and from 0.0456 to 0.0651 on QuickDraw-Extended [arxiv:1904.03451 tab. 3]. The gain is largest on the new QuickDraw benchmark. The full model, which adds the semantic loss, reaches 0.3691 on Sketchy-Extended [arxiv:1904.03451 tab. 3]. The TU-Berlin discussion says results move a lot across random splits [arxiv:1904.03451 §5.2]. The card records that the released code restarts the reversal schedule each epoch, while the paper ramps it over all iterations [arxiv:1904.03451 §4.3]. Card: [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](/papers/1904.03451). ### Feature-based: Wasserstein domain critic on production traffic Markov et al. adapt a content classifier from labeled public data to unlabeled production traffic. A transformer encoder feeds a classification head and a critic head. The critic maps each embedding to a real number, and the training objective adds the gap between the critic's mean output on the two domains to the classification loss [arxiv:2208.03274 §3.4]. This is Wasserstein domain-adversarial training (WDAT). The weight on the gap is λ = 0.01, and the critic weights are clipped to [-0.01, 0.01] to keep the critic Lipschitz [arxiv:2208.03274 §3.4]. The authors chose it over a classifier-based domain adversary, reporting more stable training and better production results. The paper does not state the critic's architecture beyond a domain critic head, and the optimizer and schedule are also not stated [arxiv:2208.03274 §4.1]. The effect is measured on a production validation set, with three seeds and no spread reported [arxiv:2208.03274 §4.4]. With public data only (PUB, about 90k labeled examples), WDAT raises average AUPRC in all eight categories, by 0.03 (S) to 0.29 (V2). The largest gains are self-harm, from .063 to .281, and V2, from .165 to .453 [arxiv:2208.03274 tab. 5]. When 69k curated synthetic examples are added (SYN), WDAT is higher in five categories. It is lower for H, from .551 to .476, which is a larger drop than the paper's "slightly hurt" wording suggests [arxiv:2208.03274 §4.4] [tab. 5]. When 60k labeled production examples are also added (MIX), WDAT is lower for S, H and V and higher for the other five, by 0.003 to 0.09 [arxiv:2208.03274 tab. 5]. So the gain is largest when labeled in-distribution data is scarce, and it is small or negative once that data is present. This matches the paper's own reading [arxiv:2208.03274 §4.4]. The paper's active-learning loop picks which production samples to label. That is a separate choice from the adversarial term, and it belongs to the active-learning concept. Its numbers are author-reported, and the production test set is not shared [arxiv:2208.03274 §4.2] [§4.3] [tab. 4]. Card: [A Holistic Approach to Undesired Content Detection in the Real World](/papers/2208.03274). ### Feature-based, Sim2Real: domain-adversarial training for cooperative LiDAR detection V2V4Real releases a real two-vehicle LiDAR dataset and defines a Sim2Real benchmark on it. The source is the labeled simulated OPV2V data and the target is the unlabeled real LiDAR of the training split [arxiv:2303.07601 §4.3]. The adapted model adds two discriminators to a cooperative detector, in the style of a domain-adversarial network [arxiv:2303.07601 §4.3]. One sits on the fused feature map and has two 3×3 convolutions. The other sits on the classification score map and has three linear layers with ReLU. Both use binary cross-entropy and a gradient-reversal layer [arxiv:2303.07601 app. C.3]. The paper does not state the discriminator loss weight or the reversal scalar [arxiv:2303.07601 app. C.3]. The benchmark reports AP@0.5 on the labeled real test split. The no-adaptation baseline trains on OPV2V and tests on real data. On this comparison, the domain-adversarial variant raises CoBEVT from 32.6 to 40.2 AP@0.5, a gain of 7.6 [arxiv:2303.07601 tab. 5]. Gains differ by fusion method: 0.9 for AttFuse, 13.7 for F-Cooper, 3.1 for V2VNet and 12.1 for V2X-ViT [arxiv:2303.07601 tab. 5]. The text reports an average gain of 7.46 points, while the table values average 7.48, so the rounded table values should not be quoted as the paper's average [arxiv:2303.07601 §5.4]. The authors also compare the adapted CoBEVT with a no-fusion model trained on the real data, 40.2 against 39.8 [arxiv:2303.07601 §5.4] [tab. 3]. That comparison uses one run per row with no spread reported, so it is a close call and not a clear win. The paper's scope is narrow. The benchmark uses LiDAR only and one merged vehicle class, and the source simulator's version and split are not given [arxiv:2303.07601 §4.1] [§4.3]. Its numbers are for cooperative detection only. The dataset is not openly licensed, so check its terms before reuse [arxiv:2303.07601 §3]. Card: [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](/papers/2303.07601). ### Feature-based: manifold alignment with a closed-form classifier (MEDA) Manifold Embedded Distribution Alignment (MEDA) works on fixed feature vectors, such as SURF or DeCAF6 features. It first maps both domains onto a geodesic flow kernel between their subspaces [arxiv:1807.07258 §3.3] [eq. 2]. Then it mixes a marginal MMD and a sum of per-class MMDs with a weight mu [arxiv:1807.07258 eq. 3]. The mu is estimated from A-distances on the unlabeled target, and it is recomputed each round [arxiv:1807.07258 §3.4] [eq. 5] [eq. 6]. The classifier is fitted in one closed-form linear solve, not by a training loop [arxiv:1807.07258 §3.5] [eq. 16]. The target labels start from a base classifier and are refined over T rounds [arxiv:1807.07258 alg. 1]. The paper sets T = 10 [arxiv:1807.07258 §4.3]. Author-reported accuracy over 28 tasks is 73.2, against 69.7 for JGSA, the best listed baseline [arxiv:1807.07258 §4.4]. The gain varies by feature set. On Office+Caltech10 with SURF it is 52.7 against 50.0 for JGSA [arxiv:1807.07258 tab. 2]. On USPS/MNIST and ImageNet/VOC it is 75.9 against 72.8 for ARTL [arxiv:1807.07258 tab. 3]. On Office+Caltech10 with DeCAF6 it is 92.8 against 90.7 for ARTL [arxiv:1807.07258 tab. 4]. Removing the manifold step lowers the accuracy in each of the six tasks of that table, and on C→A with SURF the average rises from 44.9 to 56.5 [arxiv:1807.07258 tab. 5]. The paper does not show a seed spread for the headline numbers. Its standard deviations are over mu, not over runs [arxiv:1807.07258 tab. 5]. Cost grows fast with data size. The method solves a system of size n plus m each round. The paper reports 2,931.7 seconds for ImageNet to VOC with 10,717 samples, against 2,648.8 for ARTL on the same task [arxiv:1807.07258 tab. 7]. It gives no memory figure. The paper's equations are not all the released code's choices. The MATLAB code normalises the mixed matrix, uses a different Laplacian, clips mu to [0, 1] and starts from 1-NN labels [arxiv:1807.07258 eq. 10] [eq. 14] [§3.4]. The paper does not state these choices, and the repository has no license, so the card quotes no code. Card: [Visual Domain Adaptation with Manifold Embedded Distribution Alignment](/papers/1807.07258). ### Test-time: adapt to each input Test-time training (TTT) with a masked autoencoder (MAE) takes a different route. There is no pool of target data. The paper assumes the shift is unknown in advance and that the test input is the only information about it [arxiv:2209.07522 §1] [§5]. For each test image the procedure resets the pretrained weights, runs 20 SGD steps on masked-patch reconstruction of that one image, predicts on the unmasked image, and then discards the adapted weights [arxiv:2209.07522 §3] [§4.1]. The main SGD step uses momentum 0.9 and a fixed learning rate of 5e-3 at batch 128, where the batch is 128 copies of the same image with independent masks [arxiv:2209.07522 §4.1]. On ImageNet-C at severity level 5, the author-reported accuracy improves over the fixed ViT-probing baseline on all 15 corruptions. For JPEG it goes from 51.2% to 63.0% [arxiv:2209.07522 tab. 2]. The paper also reports that the earlier rotation-based TTT hurts on rotation-invariant classes, which the masked-reconstruction task does not [arxiv:2209.07522 §2] [tab. 1]. The cost is paid at every prediction: 20 steps over 128 copies per image, and the paper gives no wall-clock time [arxiv:2209.07522 §4.1] [Limitations]. Card: [Test-Time Training with Masked Autoencoders](/papers/2209.07522). ### Generator adaptation from text, with no target images StyleGAN-NADA shifts a pretrained StyleGAN2 generator to a new domain from a pair of text prompts, such as "dog" to "cat" [arxiv:2108.00946 tab. 2]. A frozen copy of the generator supplies source images, and a trained copy is updated. Both share one frozen mapping network [arxiv:2108.00946 §4.1] [fig. 2]. The loss is one minus the cosine between the CLIP-space change from source to target image and the change from source to target text [arxiv:2108.00946 eq. 2]. A global CLIP loss on the target text alone is rejected, because it leads to adversarial pixel patterns and mode collapse [arxiv:2108.00946 §4.1] [app. B]. For larger shape changes, the trained layers are chosen at each iteration by a short latent optimisation, and only the top k layers are updated [arxiv:2108.00946 §4.2] [app. I]. The authors' default recipe uses Adam at lr 0.002 with batch size 2, about 300 iterations for texture changes (roughly three minutes on one V100), and 2000 iterations plus a latent mapper for animal changes [arxiv:2108.00946 app. I]. On FID with 10 target images, the Ojha et al. few-shot baseline goes from 45.13 to 41.22 with zero-shot pre-training, and with 100 images from 27.54 to 16.17 [arxiv:2108.00946 tab. 2]. The quality user study reports a preference for this method over most 5- and 10-shot baselines, without per-method percentages in the text [arxiv:2108.00946 §5.1] [fig. 12]. The paper says the method works best for style and fine detail, and may struggle with large geometry or scale changes [arxiv:2108.00946 §6]. Card: [StyleGAN-NADA](/papers/2108.00946). ### Source-free: a class-conditional generator on the source model alone 3C-GAN adapts a classifier when its source data is not available at all. The inputs are the source-trained classifier and unlabeled target images [arxiv:2502.19316 §1] [§3]. A class-conditional generator and a discriminator are trained against the target images. The generator must make images that look like the target and that the classifier assigns to the conditioning label [arxiv:2502.19316 §3.1] [eq. 2] [eq. 3]. The classifier is then trained on those labeled generated samples [arxiv:2502.19316 §3.1] [eq. 5]. Two penalties keep the classifier in place. A squared distance to the source weights, which becomes plain L2 weight decay when the source weights are zero, keeps it near the source model [arxiv:2502.19316 §3.2] [eq. 6]. A clustering term combines a conditional entropy on target images with a perturbation-based consistency term [arxiv:2502.19316 §3.3] [eq. 7] [eq. 8] [eq. 9]. The paper does not state the generator's semantic-loss weight λ_s, the perturbation radius, the batch size or the epoch count [arxiv:2502.19316 §4.1] [eq. 4] [alg. 1]. On digits, the author-reported accuracy is 99.4 for SVHN to MNIST, against 76.4 for the source-only model. For MNIST to USPS it is 97.3, against 92.4 [arxiv:2502.19316 tab. 1]. On Office-31 with ResNet50 the average is 89.6, against 76.1 for the source-only model [arxiv:2502.19316 tab. 2]. On VisDA17 with ResNet101 the class-mean accuracy is 81.6, against 52.4 for the source-only model [arxiv:2502.19316 tab. 3]. The SVHN to MNIST result ties the DIRT-T figure of 99.4 that the table lists as the best other method [arxiv:2502.19316 tab. 1]. In the digit ablation, removing the generator's classifier loss stops training from converging [arxiv:2502.19316 tab. 4] [§4.4]. The paper's own prose gives some figures that do not match its tables, so use the tables [arxiv:2502.19316 §4.2]. The paper names no code, and the card quotes none. Card: [Model Adaptation: Unsupervised Domain Adaptation without Source Data](/papers/2502.19316). ### Source-free, streaming: pseudo labels refined in a memory queue AdaContrast adapts a source-trained ResNet to an unlabeled target set, with no source images during adaptation [arxiv:2204.10377 §1]. The target model starts from the source weights, and a momentum copy follows it as a moving average [arxiv:2204.10377 §3.1] [eq. 2]. Each target mini-batch gives one weak view and two strong views of every image [arxiv:2204.10377 §3]. The pseudo label of the weak view is the argmax of the averaged probabilities of its N nearest entries in a queue of earlier features. The paper refines it at every step rather than once per epoch [arxiv:2204.10377 §3.1] [eq. 3] [eq. 4]. A contrastive loss pulls the two strong views together and drops negatives whose pseudo label matches the query's [arxiv:2204.10377 §3.2] [eq. 5] [eq. 6]. The total loss is a cross-entropy on the strong view, that contrastive term and a term on the mini-batch mean prediction, all with weight 1 and no tuning in any experiment [arxiv:2204.10377 §3.3] [eq. 10]. On VisDA-C train to validation, the author-reported per-class average is 86.8, against 83.0 for SHOT, a source-free baseline [arxiv:2204.10377 tab. 1]. On DomainNet-126 over seven shifts it is 67.8, against 67.1 for SHOT [arxiv:2204.10377 tab. 2]. The paper does not say how many runs each table averages, so the 0.7-point gap on DomainNet-126 is weak evidence [arxiv:2204.10377 §4]. The learning-rate sweep is more telling. At 10 times the base rate, SHOT falls to 72.1 on VisDA-C, while AdaContrast stays at 86.6 [arxiv:2204.10377 tab. 3]. Removing the contrastive term at that rate drops VisDA-C to 9.9, and removing the diversity term gives 84.3 against 86.6 [arxiv:2204.10377 tab. 4]. The queue size matters too, with 86.3 at 2048 entries against 84.4 at 512 [arxiv:2204.10377 §4.3]. The paper's text calls 78.7 the overall accuracy of its online mode, while table 1 lists it as a per-class average, so read that label from the table [arxiv:2204.10377 §4.2] [tab. 1]. The official code differs from the paper in several places. It uses 10 neighbours by default, not 11, and fills the queue from a deterministic transform rather than weak augmentations [arxiv:2204.10377 §4.3] [code: configs/learn/target.yaml#L18] [code: target.py#L46-L50]. It updates the queue before building the negative mask, which changes which negatives are used [code: target.py#L373-L381]. The knowledge-distillation and online modes are not in the repository [arxiv:2204.10377 §4.2]. The repository has no LICENSE file, so its terms are unstated and the card cites it by path only. Implement from the paper's equations, which the main loss matches [code: target.py#L398-L402]. Card: [Contrastive Test-Time Adaptation](/papers/2204.10377). ### Continued pre-training on domain text Continued pre-training starts from base weights and trains further on unlabeled domain text with the language-model loss. No task labels are used at this stage [arxiv:2402.10373 §3.2]. BioMistral continues pre-training Mistral 7B Instruct v0.1 on 3 billion tokens of open-access medical articles from PubMed Central [arxiv:2402.10373 §3.1] [§3.2]. It uses AdamW at a learning rate of 2e-5 with a cosine schedule [arxiv:2402.10373 §3.2]. The result is mixed. On the ten English few-shot tasks, the average is 50.3 for BioMistral against 51.2 for Mistral 7B Instruct. BioMistral is higher on 8 of the 10 tasks, and the average is pulled down by PubMedQA, 37.6 against 72.2 [arxiv:2402.10373 tab. 2]. The paper's best variant, a merge of the two models with SLERP, reaches 55.4 [arxiv:2402.10373 tab. 2]. Card: [BioMistral](/papers/2402.10373). ChipNeMo applies the same step to LLaMA2 at three sizes, on a mix of internal chip-design material and a sampled public share of about 9% of training tokens [arxiv:2311.00176 §2.2] [app. Training Data]. It adds about 9K tokens to the tokenizer first [arxiv:2311.00176 §2.1] [§3.1], uses a learning rate of 5e-6 for about 23,200 steps at batch 256 [arxiv:2311.00176 §2.2] [app. DAPT Hyperparameters], and runs alignment afterwards [arxiv:2311.00176 §2.3]. Its token counts disagree across the paper [arxiv:2311.00176 §2.2] [app. Training Data]. The authors report that DAPT on a chat-aligned model badly degraded alignment, so DAPT has to start from base weights [arxiv:2311.00176 §2.3]. Cost at 70B is 20,500 A100-hours [arxiv:2311.00176 §3.5]. On the authors' internal design-knowledge multiple-choice set, ChipNeMo-70B scores 76.6 against 58.4 for GPT-4. On the scripting set, GPT-4 is ahead, 77.4 against 73.9 [arxiv:2311.00176 app. Auto Evaluation Results]. These benchmarks are internal and not released [arxiv:2311.00176 §3.7]. Card: [ChipNeMo](/papers/2311.00176). COVID-Twitter-BERT (CT-BERT) continues BERT-Large, uncased with whole-word masking, on 22.5M cleaned COVID-19 tweets, using the masked-token and next-sentence objectives [arxiv:2005.07503 §2] [§3.1]. Sequences are at most 96 tokens, batches hold 1024 of them, and the paper stops at 500,000 steps [arxiv:2005.07503 §2] [§3.1]. The run took 120 hours on one TPU v3-8 [arxiv:2005.07503 §2]. Each downstream set is then fine-tuned with learning rate 2e-5 [arxiv:2005.07503 §2.1]. The authors say that recipe was tuned on BERT-Large and that CT-BERT may do better with its own tuning [arxiv:2005.07503 §2.1] [§4]. Author-reported mean F1 over five classification sets is 0.833, against 0.802 for BERT-Large [arxiv:2005.07503 tab. 2]. The paper's relative-gain measure is largest on the COVID-19 category set, at 25.88%, and smallest on SemEval, at 8.97% [arxiv:2005.07503 tab. 2] [§3.2]. SST-2 is not Twitter text and serves as a control, where the score moves from 0.937 to 0.944 [arxiv:2005.07503 §2.1] [tab. 2]. The released pretraining script uses warmup and linear decay, where the paper uses a constant rate, and it runs fewer steps than the paper [code: run_pretrain.py#L148-L160] [code: scripts/run_pretrain.sh#L15-L16]. Pick one setting and report which. MEDITRON continues Llama-2 at 7B and 70B on GAP-Replay, about 48 billion tokens of clinical guidelines, PubMed abstracts and PubMed Central papers. About 1% of the mix is general text from RedPajama, to limit forgetting [arxiv:2311.16079 §2] [tab. 1]. The 70B run uses peak learning rate 1.5e-4, global batch 512 and 2000 warmup steps, on 128 A100 80GB GPUs for 332 hours [arxiv:2311.16079 §3] [§4.1] [app. A]. The end learning rate is stated more than one way [arxiv:2311.16079 app. D.1]. Each benchmark then gets its own supervised finetune, and the paper compares a top-token answer, chain-of-thought and self-consistency over five sampled answers [arxiv:2311.16079 §4.2] [§4.3]. Few-shot averages rise from 60.8 for Llama-2-70B to 63.3 for MEDITRON-70B [arxiv:2311.16079 tab. 4]. After finetuning, self-consistency gives 72.0 against 69.2 for Llama-2-70B, and the top-token mode gives 69.0 against 67.2 [arxiv:2311.16079 tab. 5]. The data-mix choices were tested only with 7B models [arxiv:2311.16079 §7.2] [tab. 8]. The responsible-AI appendix tests truthfulness, risk and bias, and the authors advise testing in each use case before medical deployment [arxiv:2311.16079 app. E]. Card: [MEDITRON-70B](/papers/2311.16079) and [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](/papers/2005.07503). MatSciBERT continues SciBERT, not a general model, on about 285M words of materials-science full text and abstracts [arxiv:2109.15290 tab. 1] [§2.1]. It starts from SciBERT's uncased weights and uses a RoBERTa-style recipe with whole-word masking and no next-sentence loss [arxiv:2109.15290 §2.2]. The authors report about 10 days on two V100 GPUs [arxiv:2109.15290 §2.2]. The corpus itself is not released, so only the released checkpoint can be reused [arxiv:2109.15290 §2.1] [code: pretraining/run.sh#L4-L5]. On the authors' test sets, the BiLSTM-CRF head reaches 65.92 Macro-F1 on SOFC-Slot against 61.68 for SciBERT, and abstract classification F1 is 93.57 against 91.04 [arxiv:2109.15290 tab. 2] [tab. 6]. Gains are smaller on other tables. On Matscholar Macro-F1 the two models tie at 85.66 for the BiLSTM-CRF head, and relation classification is 91.26 against 91.04 Micro-F1 [arxiv:2109.15290 tab. 5] [tab. 7]. The baseline is SciBERT alone, and the MatSciBERT path normalises its text while the SciBERT path does not. Part of the gain may therefore come from preprocessing [code: ner/ner.py#L54-L62]. The numbers are means over three seeds with no spread shown [arxiv:2109.15290 §2.4.4] [tab. 2]. Card: [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](/papers/2109.15290). A legal-domain survey makes the same point from the evidence side. It lists domain pretraining and retrieval as ways to adapt LLMs to legal text. It says domain pretraining gives legal vocabulary but stale knowledge, and that retrieval can miss the right source or cite the wrong authority [arxiv:2303.09136 tab. 5]. It proposes no experiments of its own. Card: [A Short Survey of Viewing Large Language Models in Legal Aspect](/papers/2303.09136). ### Retriever and generator: joint training on the target domain In retrieval-augmented question answering, the retriever is a second model that the adaptation can also change. RAG-end2end trains the DPR question and passage encoders together with the BART generator on domain data. Every change to the passage encoder makes the stored passage vectors stale, so the authors re-encode the passages and rebuild the FAISS index in background processes while training continues. Training uses the answer loss, and an auxiliary task rebuilds a domain sentence from retrieved passages, marked with a control token [arxiv:2210.02627 §3.3] [§3.4] [eq. 3] [fig. 1]. The author-reported gains are exact match with reconstruction added (QA+R) of 3.66 to 8.32 on COVID-19, 8.62 to 14.08 on news and 14.21 to 25.95 on conversations, each against the public RAG checkpoint fine-tuned on the same QA pairs [arxiv:2210.02627 tab. 1]. Initialising RAG from a domain-adapted DPR, without training RAG end to end, gives part of that gain [arxiv:2210.02627 tab. 4]. The cost is the re-encoding and re-indexing, which the paper gives as several GPU hours and several CPU hours on a knowledge base of millions of passages [arxiv:2210.02627 §3.3]. The paper reports one run per setting, with no seeds or intervals [§4.2]. Card: [Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering](/papers/2210.02627). ### Head first, then fine-tune all layers (LP-FT) Fine-tuning all parameters from a pretrained network usually beats linear probing on the training distribution. On a large shift it can do worse. The authors trace this to feature distortion: with a random head, the early gradient moves the features in the directions the training data touches, which hurts the directions out-of-distribution data uses [arxiv:2202.10054 §3.2] [§3.2.1]. Linear probing then fine-tuning (LP-FT) first fits the head with the features frozen, then fine-tunes everything from that head [arxiv:2202.10054 §4.2]. Averaged over 10 shifts, the author-reported numbers are 85.1 ID and 59.3 OOD for fine-tuning, 82.9 and 66.2 for linear probing, and 85.7 and 68.9 for LP-FT [arxiv:2202.10054 tab. 1] [tab. 2]. The gain depends on the features and the shift. With an earlier MoCo-v1 checkpoint, fine-tuning beats linear probing OOD on Living-17 (71% against 66%), and on CIFAR-10 against CIFAR-10.1 fine-tuning is also better OOD [arxiv:2202.10054 §4.3]. The paper calls LP-FT a first step, not a state-of-the-art method [arxiv:2202.10054 §1]. On ImageNet, LP and LP-FT run without augmentation while FT uses it, and LP-FT's ID accuracy ties FT at 81.7 [arxiv:2202.10054 app. B] [tab. 1]. Card: [Fine-Tuning can Distort Pretrained Features](/papers/2202.10054). ### Federated personalisation: align the head to each user's data FedHealth has two stages. A cloud CNN is built from encrypted updates sent by the users [arxiv:1907.09173 §3.3]. Each user then freezes the convolution and pooling layers and fine-tunes the fully connected layers on its own data [arxiv:1907.09173 §3.4] [§4.2]. The user objective adds η times a CORAL term, which compares the covariance matrices of the two feature sets [arxiv:1907.09173 eq. 5] [eq. 6]. The paper fixes η at 0.01 [arxiv:1907.09173 §4.2]. The text does not say which tensor or batch each covariance is estimated from [arxiv:1907.09173 §3.4] [eq. 5]. On five isolated UCI Smartphone subjects, the author-reported mean accuracy is 99.4%, against 94.1% for the cloud model with no transfer. That is a gap of 5.3 points. Per subject the gap runs from 3.9 to 7.4 points [arxiv:1907.09173 tab. 2]. The text says transfer beats no transfer by 4% on average, and that plain fine-tuning and MMD alignment also help. Their per-variant values are only in [fig. 4] [arxiv:1907.09173 §4.4]. The encryption scheme and the server's aggregation rule are not stated, so the federated part cannot be rebuilt from the text [arxiv:1907.09173 §3.3]. The experiments section does not describe running the encrypted steps, so whether the reported numbers include them is not stated [arxiv:1907.09173 §4.2]. The result rests on five test subjects, five runs and no spread [arxiv:1907.09173 tab. 2]. Card: [FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare](/papers/1907.09173). ### Merge a model with a donor model Fisher merging averages the parameters of models that share an initialization, with each parameter weighted by a diagonal Fisher information value rather than by a plain mean [arxiv:2111.09832 §2.3] [§2.4]. For domain adaptation, the paper merges a domain-adapted pre-trained checkpoint with a task fine-tuned checkpoint, with RoBERTa-base [arxiv:2111.09832 §3.4]. On ChemProt the merge scores 83.1 ± 0.4 micro-F1, against 82.7 ± 0.3 for the unmerged model and 82.5 ± 0.1 for a model fine-tuned from the domain checkpoint [arxiv:2111.09832 tab. 1]. On ACL-ARC, the Fisher merge reaches 73.2 ± 1.7 and isotropic merging 72.5 ± 2.3. On SciERC, isotropic merging is the best row at 81.7 ± 0.5, and the Fisher merge is 81.3 ± 0.5 [arxiv:2111.09832 tab. 1]. The authors estimate the merge at about 6 times fewer FLOPs than fine-tuning for an RTE checkpoint [arxiv:2111.09832 §3.3]. That estimate counts the Fisher computation, and it is not a wall-clock time [arxiv:2111.09832 §3.3]. With an MNLI donor on RTE, a sequential gradient-trained model reaches 77.4 ± 1.6, against 73.2 ± 5.1 for the Fisher merge [arxiv:2111.09832 tab. A1] [tab. A3]. So merging does not beat gradient-based transfer in every setting. Card: [Merging Models with Fisher-Weighted Averaging](/papers/2111.09832). ### Choose a model without target labels This is model selection, not training. Deep Embedded Validation (DEV) scores candidate UDA models using labeled source validation data, reweighted to look like the target [arxiv:2606.04665 §4.2] [§4.3]. The weights come from a domain classifier trained on source-training and unlabeled target features, computed in the model's own feature space [arxiv:2606.04665 eq. 7] [alg. 1]. A control variate reduces the variance of the estimate [arxiv:2606.04665 §4.4] [eq. 9] [eq. 10]. On VisDA with MCD, the mean target accuracy of the selected model is 72.42 with DEV, 66.36 with source risk and 54.72 with IWCV. Target-label selection gives 72.55 [arxiv:2606.04665 tab. 2]. On the other three tables, DEV is within 0.13 to 0.41 points of target-label selection [arxiv:2606.04665 tab. 2–5]. These gaps come from one number per method with no run count or spread, so they may be within noise [arxiv:2606.04665 §5]. The IWCV rows use a classifier-based density ratio on pre-trained features, not the Gaussian fit of the original method [arxiv:2606.04665 §2.2] [§5]. The paper gives no bound on the classifier's error, so expect bias in practice [arxiv:2606.04665 §4.3]. Card: [Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation](/papers/2606.04665). ### Domain generalization: no target data at all Domain generalization (DG) trains on several source domains and tests on an unseen one, with no target data [arxiv:2103.03097 §II.A]. The survey groups methods into data manipulation, representation learning and learning strategies [arxiv:2103.03097 §IV]. Its benchmark uses DeepDG with ResNet-18. On PACS, ERM averages 81.2, and CORAL and RSC both average 82.0. On Office-Home, ERM averages 62.9 and RSC 63.7 [arxiv:2103.03097 §VII.C]. Each cell is one run, with no seed count, so differences of 0.5 or less are within what one seed can move [arxiv:2103.03097 §VII.C]. Card: [Generalizing to Unseen Domains](/papers/2103.03097). ### Two-stage supervised transfer: general fact-checking, then science The SciFact paper trains its label model in two stages. It starts from FEVER, a general fact-checking set, and then continues on the 809 labeled scientific training claims [arxiv:2004.14974 tab. 2] [§5]. Both stages use labeled data. The paper does not frame the step as domain adaptation, and it is listed here because the card carries that concept. On the dev set, label accuracy is 81.9 after FEVER and SciFact, against 75.7 for SciFact alone and 67.6 for FEVER alone [arxiv:2004.14974 tab. 3]. For rationale selection the order reverses: F1 is 69.7 after FEVER and SciFact, and 72.1 with SciFact alone [arxiv:2004.14974 tab. 3]. So the two-stage model helps the label step and hurts the rationale step on the authors' numbers. The FEVER stage needs the FEVER data and takes about 700 minutes on one P100 [arxiv:2004.14974 app. A.4]. On the open test setting, a FEVER-only system that was not trained on SciFact reaches 36.4 for abstract label with rationale, and the SciFact-trained VeriSci reaches 46.5 [arxiv:2004.14974 tab. 4]. The zero-shot FEVER threshold was tuned on SciFact dev, so that row is not free of target data [arxiv:2004.14974 app. A.1]. Card: [Fact or Fiction: Verifying Scientific Claims](/papers/2004.14974). ### Surveys, benchmarks and taxonomies Plested et al. survey deep transfer learning for image classification. Their main advice is to decide how far the weights may move from the source, not whether to transfer at all [arxiv:2205.09904 §7] [§9]. The taxonomy has two axes: source-target similarity, which is continuous, and target size, with small around 10,000 labeled examples and large a million or more [arxiv:2205.09904 §3.7] [fig. 5]. The survey ranks source types, from a very similar labeled source much larger than the target down to self-training on the target when no related source exists [arxiv:2205.09904 §4.2]. Its penalties toward the pretrained weights, such as L2-SP, and its rules for layers and learning rates are given in prose and are not quantified [arxiv:2205.09904 §6.2] [§7]. The only experiment is a Big Transfer extension to less similar targets. On Cars, the mean moves from 81.8 to 87.6 with the less-similar learning rate, but the column meanings need checking against the Big Transfer setup [arxiv:2205.09904 tab. 3]. The claim that viewing transfer as a continuum can remove negative transfer is the authors' proposal and is not tested in the paper [arxiv:2205.09904 §9]. Card: [Deep transfer learning for image classification: a survey](/papers/2205.09904). A review of deep learning on UAV imagery lists domain adaptation among its open directions. It names GAN-based translation between domains and category-aware adaptation, but evaluates neither [arxiv:2101.10861 §5.3]. Its figures are the authors' own tallies of 232 studies, and they do not measure adaptation [arxiv:2101.10861 §3]. Card: [A Review on Deep Learning in UAV Remote Sensing](/papers/2101.10861). The aspect-based sentiment survey has no experiments. Its only domain-adaptation content is a short taxonomy of cross-domain methods for ABSA [arxiv:2203.01054 §6.1]. Card: [A Survey on Aspect-Based Sentiment Analysis](/papers/2203.01054). ToxicChat is a benchmark, not an adaptation method. It shows the shift that adaptation has to close. Public toxicity tools score poorly on real chatbot prompts: OpenAI Moderation reaches F1 20.6 and Perspective at threshold 0.7 reaches 5.3 [arxiv:2310.17389 tab. 1]. Among RoBERTa-base models trained on other toxicity datasets, the best F1 on the ToxicChat test half is 51.6, and a model trained on the ToxicChat training half reaches 72.1 [arxiv:2310.17389 tab. 2]. That last result is in-domain supervised training, not adaptation. The paper does not test any transfer method. Card: [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](/papers/2310.17389). ## When to use which | Situation | Use | Why | |---|---|---| | Images, labeled source, unlabeled target, and a pretrained feature extractor you can keep frozen | Data-based translation (DTN) [arxiv:1611.02200 §4] | The generator is built on f, which stays fixed. The paper reports the gain on SVHN to MNIST [arxiv:1611.02200 tab. 2]. | | Text pairs, labeled source, unlabeled target, and budget to run a slow model over candidate target pairs | Pseudo-label with a cross-encoder (AugSBERT) [arxiv:2010.08240 §3.2] | Use BM25-chosen pairs, not random ones [arxiv:2010.08240 §6.1]. Expect a gap to the in-domain upper bound on most rows [arxiv:2010.08240 tab. 6]. | | Dense prediction (segmentation) with a labeled synthetic source and an unlabeled real target, and an existing entropy-minimization pipeline | Swap the target loss for maximum squares [arxiv:1909.13589 §3.2] | The change is in the loss only, with no discriminator network [arxiv:1909.13589 §1]. Gains over entropy minimization are reported on the same backbone [arxiv:1909.13589 tab. 2]. | | Segmentation where the appearance gap is large and you can afford extra sub-networks and discriminators | Feature split and texture swap (DISE) [arxiv:1903.12212 §3.1] [§3.3] | Translated images inherit source labels, so the target gets extra supervision [arxiv:1903.12212 §3.2]. The costs are the sub-networks and the unstated loss weights [arxiv:1903.12212 eq. 1]. | | Shift that is unknown in advance, only one input at a time, and the compute per prediction is available | Per-input test-time training (TTT-MAE) [arxiv:2209.07522 §3] [§4.1] | No target set is needed. The cost is 20 steps over 128 copies per prediction [arxiv:2209.07522 §4.1]. | | Image generator, no target images, a text pair describing the change, and checkpoints whose licences you can clear | StyleGAN-NADA [arxiv:2108.00946 §4.1] | Zero-shot pre-training lowers FID for each few-shot baseline in the paper's table [arxiv:2108.00946 tab. 2]. Check each checkpoint and dataset licence before commercial use; the paper's own tables list non-commercial terms for some [arxiv:2108.00946 tab. 4] [tab. 5]. | | Pretrained network with good features and a large shift, where fine-tuning does worse OOD than probing | LP-FT [arxiv:2202.10054 §4.2] | Gains need good pretrained features and a large shift. With weaker features or a small shift, fine-tuning is better OOD [arxiv:2202.10054 §4.3]. | | Language model, large domain shift, a corpus of unlabeled domain text, and budget for continued pre-training | Continued pre-training from base weights [arxiv:2402.10373 §3.2] [arxiv:2311.00176 §2.2] | The average on English few-shot tasks did not beat the base model, but the model won on most tasks [arxiv:2402.10373 tab. 2]. LoRA was clearly worse than full pre-training on in-domain tasks [arxiv:2311.00176 §3.4]. | | Several parties hold private data of one kind that cannot be pooled, a shared model is wanted, and each user can fine-tune its own fully connected layers | Federated pre-training, then freeze the early layers and fine-tune the head with a CORAL term (FedHealth) [arxiv:1907.09173 §3.4] [eq. 6] | The gain is reported on five subjects, 99.4 against 94.1 mean accuracy [arxiv:1907.09173 tab. 2]. Encryption and aggregation are underspecified, so a rebuild needs its own choices [arxiv:1907.09173 §3.3]. | | Question answering over a retrieval store of domain passages, the off-the-shelf retriever is weak on that domain, and budget to re-encode and re-index the store during training | Joint retriever and generator training with a background index refresh (RAG-end2end) [arxiv:2210.02627 §3.3] | Author-reported exact match gains over the public RAG checkpoint in three domains [arxiv:2210.02627 tab. 1]. If you cannot afford the refresh, the adapted-DPR setting gives part of the gain [arxiv:2210.02627 tab. 4]. Results are single runs [§4.2]. | | Two or more models from the same start, and no training budget for a new run | Fisher merge [arxiv:2111.09832 §2.4] | Gains over plain averaging are small and within one standard deviation on some tasks [arxiv:2111.09832 tab. 1]. Gradient-based transfer can still win [arxiv:2111.09832 tab. A1]. | | Encoder of BERT size, a large unlabeled corpus from one domain (about 22.5M short texts), and a budget for about 120 hours on one TPU v3-8 | Continued pre-training of the encoder on the domain corpus, then the usual fine-tune (CT-BERT) [arxiv:2005.07503 §2] [§2.1] | Author-reported gains over BERT-Large on five classification sets, largest on the in-domain set [arxiv:2005.07503 tab. 2]. The authors say the fine-tune recipe was tuned on the baseline and may be improved for CT-BERT [arxiv:2005.07503 §4]. | | Encoder in a specialised technical field with its own vocabulary, a scientific base model to start from (SciBERT), and a task set of NER, relation or classification | Continued pre-training of the scientific encoder on the field's text, then fine-tune per task (MatSciBERT) [arxiv:2109.15290 §2.2] [§2.4] | Author-reported gains over SciBERT on most NER and classification cells [arxiv:2109.15290 tab. 2] [tab. 6]. Ties occur, and the baseline's preprocessing differs [code: ner/ner.py#L54-L62]. The pretraining corpus is not released, so reuse the checkpoint if its licence allows [arxiv:2109.15290 §2.1]. | | Open general LLM you can continue training, a medical or other specialised corpus of about 48B tokens, and a budget for a multi-node run | Continued pre-training with a small replay share, then per-benchmark finetuning and an inference mode (MEDITRON) [arxiv:2311.16079 §4.1] [§4.3] | The 70B run took 332 hours on 128 GPUs [arxiv:2311.16079 app. A]. Self-consistency gives the best author-reported average, 72.0 against 69.2 for Llama-2-70B [arxiv:2311.16079 tab. 5]. The data-mix ablations were run at 7B only [arxiv:2311.16079 §7.2]. | | Source model only, no source data during adaptation, an unlabeled target stream, and memory for a queue of target features | Source-free contrastive adaptation with queue-refined pseudo labels (AdaContrast) [arxiv:2204.10377 §3.1] [§3.2] | Author-reported gains over SHOT on VisDA-C and DomainNet-126 [arxiv:2204.10377 tab. 1] [tab. 2]. Queue size changes the result, 86.3 at 2048 entries against 84.4 at 512 [arxiv:2204.10377 §4.3]. The paper does not report how many runs its tables average [§4]. | | Choosing among UDA models with no target labels | DEV [arxiv:2606.04665 §4.4] | Gaps to target-label selection are small on the reported tables [arxiv:2606.04665 tab. 2–5]. Fix the classifier's random split before comparing candidates [arxiv:2606.04665 §5]. | | Segmentation across an aerial sensor or resolution change, with unlabeled target tiles and budget for a GAN plus a fine-tuning run | CycleGAN-style translation, then fine-tune [arxiv:1905.03198 §4.2] | The gain is reported on one source and target pair. Building and Tree gain most, and Low vegetation drops [arxiv:1905.03198 tab. 4]. The reported epoch was chosen on target test data [arxiv:1905.03198 §5.2.4]. | | Classification with labeled source and unlabeled target, where a per-step minibatch alignment is affordable and source batches can be class-balanced | JUMBOT, minibatch unbalanced transport [arxiv:2103.03606 §5.2] | The paper reports gains on Office-Home, digits and VisDA over its baselines [arxiv:2103.03606 tab. 1–3]. Use the appendix learning rate, since the released code differs [arxiv:2103.03606 app. C]. | | Two input types, such as sketch and photo, whose embeddings must align, and test categories unseen in training | Gradient-reversal domain loss on the embeddings [arxiv:1904.03451 §4.3] | The domain term adds mAP in the paper's ablation, most on QuickDraw-Extended [arxiv:1904.03451 tab. 3]. The ablation rows are cumulative, so the gain is measured on top of attention only. | | Several labeled source domains, none of them the target, and no target data at all | Domain generalization methods [arxiv:2103.03097 §IV] | The benchmark gains over ERM are small [arxiv:2103.03097 §VII.C]. | | Target domain with classes missing from the target data | Collect target samples for every class first | Dropping a class from T cut accuracy on that class to 40.13% in the paper's test [arxiv:1611.02200 tab. 3]. | | Training cost, data size, or GPU budget for the translation, loss or feature variants | Not stated | The cards give the setup of each method, but no run time for the DTN, DISE or maximum squares adaptation. Use the cards' numbers only as the authors' settings [arxiv:1611.02200 §4, §5] [arxiv:1909.13589 §4.2] [arxiv:1903.12212 §4.2]. | | Labeled source and unlabeled target, a CNN you can retrain, and a feature-level penalty that fits in the loss | MMD penalty with per-class terms on pseudo-labels (JDA) [arxiv:1804.07265 eq. 10] [alg. 1] | The average gain on ten fault-diagnosis tasks is reported against the plain CNN [arxiv:1804.07265 tab. III]. The per-class terms depend on the network's own pseudo-labels, and the λ selection data is not stated [§IV-A]. | | Only the source-trained model is available, no source data, unlabeled target images, and a GPU budget for a conditional GAN | Source-free generation with a weight anchor (3C-GAN) [arxiv:2502.19316 §3.1] [§3.2] | Author-reported gains over the source-only model on digits, Office-31 and VisDA17 [arxiv:2502.19316 tab. 1–3]. The generator's semantic-loss weight and the perturbation radius are not stated [§4.1], so expect to tune them. | | Fixed feature vectors such as SURF or DeCAF6, no target labels, and a closed-form solve per round is affordable | MEDA, with manifold features and a mu-weighted MMD [arxiv:1807.07258 §3.5] [alg. 1] | Gains over JGSA on the 28-task average are reported [arxiv:1807.07258 §4.4]. Cost is high at the largest size in the paper, 10,717 samples, where it is slower than ARTL [arxiv:1807.07258 tab. 7]. Implement the paper's equations, not the released code's choices [arxiv:1807.07258 eq. 10]. | | Labeled general data and labeled target data, no unlabeled target data, and a choice per sub-task | Two-stage training: general data first, then the target (FEVER then SciFact) [arxiv:2004.14974 tab. 3] | Dev label accuracy rises from 75.7 to 81.9, but rationale F1 falls from 72.1 to 69.7 [arxiv:2004.14974 tab. 3]. Choose the stage per sub-task and check each on dev. | | Image classification with a pretrained source and a small labeled target, where the question is how much to transfer | Choose the source type, then the depth and learning rate by similarity and target size (survey taxonomy) [arxiv:2205.09904 §4.2] [§7] | The rules are the survey's and mostly summarise cited work. Its one experiment is a Big Transfer extension to less similar targets [arxiv:2205.09904 tab. 3]. Check the rules on your own validation split. | | Labeled public source, unlabeled production target, and a training loop where a second head and a clipped critic are affordable | Wasserstein domain-adversarial training (WDAT) [arxiv:2208.03274 §3.4] | Author-reported gains over public-only training are large for rare categories [arxiv:2208.03274 tab. 5]. With labeled in-distribution data the gains are small or mixed, so test on your own validation set. | | Simulated labeled LiDAR for a cooperative detector, unlabeled real scans from the target vehicles, and a second discriminator head affordable | Domain-adversarial training on the fused features and score map (V2V4Real Sim2Real benchmark) [arxiv:2303.07601 §4.3] [app. C.3] | Author-reported gain of 7.6 points for CoBEVT, 32.6 to 40.2 AP@0.5, with gains from 0.9 to 13.7 across fusion methods [arxiv:2303.07601 tab. 5]. Loss weights are not stated, and the dataset is not openly licensed [arxiv:2303.07601 §3]. | ## Implementation sketch Sketches follow, one for each family of variants with a distinct training loop. Learning rates, batch sizes and iteration counts are not always stated in the cards, so expect to tune them. Translation-based adaptation, as the DTN paper describes it [arxiv:1611.02200 §4]. ```pseudocode # Data-based adaptation by translation (DTN-style) [arxiv:1611.02200 §4] # Inputs: labeled source xs, unlabeled target xt, frozen feature function f # g: trainable decoder on top of f; D: three-way discriminator def train_step(xs, xt, alpha, beta, gamma): # Encode both domains with frozen f [§4] fs, ft = f(xs), f(xt) # Decode into the target domain: G = g o f [§4] gen_s, gen_t = g(fs), g(ft) # Discriminator: classes for generated-from-source, # generated-from-target and real target [eq. 3] update(D, discriminator_loss(gen_s, gen_t, xt)) # Generator: both generated sets should look real [eq. 4] # Feature constancy on source, identity on target [eq. 5, eq. 6] # Optional smoothness term, weight gamma [eq. 7] update(g, adversarial_loss + alpha * constancy + beta * identity + gamma * smoothness) # After training: translate the labeled source, keep its labels, # and fit a classifier on the translated samples [§5.1] translated = g(f(source_images)) classifier = fit(translated, source_labels) ``` Loss-based adaptation with a maximum squares target term [arxiv:1909.13589 eq. 1] [eq. 6]. ```pseudocode # Source supervision plus a maximum-squares target term # [arxiv:1909.13589 eq. 1] [eq. 6] # Inputs: labeled source (xs, ys), unlabeled target xt; lambda_t (paper: 0.1) [§4.2] def adapt_loss(xs, ys, xt, lambda_t): # Supervised cross-entropy on the source batch [eq. 2] loss_src = cross_entropy(model(xs), ys) # Target probabilities on unlabeled pixels [§3.2] p_t = softmax(model(xt)) # (B, C, H, W) # Negative sum of squared probabilities, normalised per image [eq. 6] loss_tgt = -sum(p_t ** 2) / (2 * num_pixels) return loss_src + lambda_t * loss_tgt # Optional image-wise weights from the predicted class counts [eq. 13] # Each class term is divided by 2 * N_c^alpha * N^(1 - alpha), alpha = 0.2 [§4.2] ``` Pseudo-labelling of target pairs, as AugSBERT describes it [arxiv:2010.08240 §3.2]. ```pseudocode # Pseudo-labelling a target domain with a cross-encoder [arxiv:2010.08240 §3.2] # Inputs: labeled source pairs; unlabeled target candidate pairs # Output: bi-encoder trained on the target-domain soft labels def adapt_by_pseudo_labels(source_pairs, target_candidates): # Fit the accurate but slow cross-encoder on labeled source pairs [§3.2] cross_encoder = train_cross_encoder(source_pairs) # Score each candidate; the score is the soft label [§3.1] # Candidates should come from BM25 neighbours, not random draws [§6.1] silver = [(a, b, cross_encoder.score(a, b)) for a, b in target_candidates] # Fit the fast bi-encoder on the soft-labeled target pairs [§3.2] return train_bi_encoder(silver) ``` Minibatch transport with a label-aware cost, as JUMBOT describes it [arxiv:2103.03606 eq. 9] [eq. 10]. ```pseudocode # One JUMBOT training step, k = 1 [arxiv:2103.03606 §5.2] [eq. 9] [eq. 10] # Inputs: class-stratified source batch (xs, ys) and unlabeled target batch xt, # both of size m; eta1, eta2, eta3 weights; tau, eps for the unbalanced plan def jumbot_step(xs, ys, xt, eta1, eta2, eta3, tau, eps): # Embed both batches with the feature extractor [§5.2] fs, ft = g(xs), g(xt) # (m, d) # Class probabilities for the target batch [§5.2] prob_t = softmax(f(ft)) # (m, C) # Pair cost: squared feature distance plus source-label cross-entropy [eq. 9] cost_feat = squared_distances(fs, ft) # (m, m) cost_label = -log(prob_t[:, ys]).T # (m, m): source i, target j cost = eta1 * cost_feat + eta2 * cost_label # Unbalanced entropic plan on the pair; the plan carries no gradient [§2] [app. B] plan = unbalanced_sinkhorn(uniform(m), uniform(m), cost.detach(), eps, tau) # Supervised loss plus the plan-weighted transfer term [eq. 10] return cross_entropy(f(fs), ys) + eta3 * sum(plan * cost) ``` Adversarial domain critic on a shared encoder, as the moderation paper describes it [arxiv:2208.03274 §3.4]. ```pseudocode # One WDAT step; lambda = 0.01; critic weights clipped after each update # [arxiv:2208.03274 §3.4] # Inputs: labeled public batch (xs, ys), unlabeled production batch xt def wdat_step(xs, ys, xt, lam=0.01): # Shared encoder embeds both domains [§3.4] hs, ht = encoder(xs), encoder(xt) # Classification loss on the labeled source batch [§3.4] class_loss = classification_loss(head(hs), ys) # Absolute gap between mean critic scores of the two domains [§3.4] gap = abs(mean(critic(hs)) - mean(critic(ht))) # Encoder and head minimise this; the critic is trained on the same gap # in the opposite direction, then its weights are clipped [§3.4] return class_loss + lam * gap ``` MMD penalty on pseudo-labelled target features, as the deep transfer network describes it [arxiv:1804.07265 eq. 10] [eq. 11]. ```pseudocode # Source supervision plus a JDA penalty on pseudo-labelled target features # [arxiv:1804.07265 alg. 1] [eq. 10] [eq. 11] # Inputs: labeled source (xs, ys), unlabeled target xt; lambda (grid in the paper) def jda_train(xs, ys, xt, lam, rounds): # Base network on labeled source data [alg. 1 line 2] net = fit_cross_entropy(net, xs, ys) # Initial pseudo-labels for the target [alg. 1 line 3] yt_hat = argmax(net(xt)) for _ in range(rounds): # paper: until the labels stop changing [alg. 1] fs, ft = net.features(xs), net.features(xt) # Marginal MMD over all features [eq. 10] penalty = mmd(fs, ft) # One class-wise MMD per class, source vs pseudo-labelled target [eq. 9] # An empty class on either side is not defined in the paper [§III-A] for c in classes: penalty += mmd(fs[ys == c], ft[yt_hat == c]) # Cross-entropy plus lambda times the penalty [eq. 11] net = sgd_step(net, cross_entropy(net(xs), ys) + lam * penalty) # Re-predict the target pseudo-labels [alg. 1] yt_hat = argmax(net(xt)) return net ``` Queue-refined pseudo labels with a contrastive term, as AdaContrast describes them [arxiv:2204.10377 alg. 1]. ```pseudocode # One AdaContrast step on an unlabeled target mini-batch [arxiv:2204.10377 alg. 1] # Inputs: target images x; target model from source weights; momentum copy; # queue_w (features, probabilities, M entries); queue_s (keys, pseudo labels) def adapt_step(x, target, momentum, queue_w, queue_s, N, tau): # Weak view refines the pseudo label from the queue [§3.1] [eq. 3-4] x_weak = weak_augment(x) f_weak = target.feature(x_weak) nearest = top_n_cosine(f_weak, queue_w.feats, N) pseudo = argmax(mean(queue_w.probs[nearest], axis=0)) # Strong query from the target model, key from the momentum model [§3.2] q, logits = target.feature_and_logits(strong_augment(x)) k = normalize(momentum.feature(strong_augment(x))) # Contrastive loss; same-pseudo-label negatives are masked [eq. 5-6] loss_ctr = info_nce(normalize(q), k, queue_s, pseudo, tau) # Cross-entropy against the refined label, plus a mean-prediction term [eq. 7-9] loss_ce = cross_entropy(logits, pseudo) p_bar = mean(softmax(logits), axis=0) loss_div = sum(p_bar * log(p_bar)) # Equal weights, no tuning [eq. 10] update(target, loss_ce + loss_ctr + loss_div) # Momentum copy follows slowly; queues take the new entries [eq. 1-2] momentum.ema_update(target, m) queue_w.push(*momentum(x_weak)) # (feature, probabilities) queue_s.push(k, pseudo) ``` Notes on where the variants differ: - The translation sketch has two networks to train and one frozen extractor. The loss sketch has one network and changes only the target term. The pseudo-labelling sketch has no target-side training of the teacher at all. - The image-wise weight divides by the count of each predicted class in the image. That count can be zero, so the weight needs a guard the paper does not state [arxiv:1909.13589 eq. 13]. - The maximum squares normalisation in eq. 6 is what the paper writes. Check the card's Pitfalls before matching a λ_T value from another implementation [arxiv:1909.13589 §4.2]. - The JUMBOT sketch is the only one with an optimal transport plan inside the step. Its gradient flows through the cost, not the plan [arxiv:2103.03606 app. B]. The paper's uniform vectors carry the mass m_α, so the unit-mass weights in the sketch match it only when that mass is 1 [arxiv:2103.03606 §4.1]. - The WDAT sketch adds a critic and weight clipping to a shared encoder. The paper does not give the critic's architecture, so the critic is a placeholder [arxiv:2208.03274 §3.4]. Its objective is not numbered, so cite §3.4. - The aerial translation pipeline keeps the translation step of the DTN sketch but trains the translator in both directions. It then fine-tunes the source segmenter instead of fitting a new classifier [arxiv:1905.03198 §4.1] [§4.3]. - The AdaContrast sketch adds to the queue after the loss, as the paper's equations imply. The released code adds the batch to the queue before it builds the negative mask, so copying the code changes which negatives are used [code: target.py#L373-L381]. The paper gives no pseudo-label threshold, so every target sample reaches the loss, and the diversity term is the only guard against collapse [arxiv:2204.10377 §3.3]. ## Contradictions Where two cards report numbers on a similar benchmark, the protocols usually differ. The following comparisons are worth knowing before you read the tables side by side. - **Segmentation on GTA5 to Cityscapes.** DISE reports 45.4 mIoU and calls it the highest in its table [arxiv:1903.12212 tab. 2] [§4.2]. The maximum squares card reports 46.4 for its full method with the same DeepLab-v2 ResNet-101 base [arxiv:1909.13589 tab. 2]. Neither table contains the other method, so the two numbers are not a head-to-head comparison. - **Entropy-minimization baseline.** The maximum squares card reports 42.2 for its own entropy-minimization reproduction and 42.3 for ADVENT's reported value [arxiv:1909.13589 tab. 2]. Both are author-reported, and the card marks the first as the authors' reproduction. - **Scope of adaptation.** The test-time card adapts to each input using only that input [arxiv:2209.07522 §1], while the translation, loss and feature cards use the whole unlabeled target set [arxiv:1611.02200 §3] [arxiv:1909.13589 §3.1] [arxiv:1903.12212 §3.1]. The numbers on these settings are not comparable. - **Fisher merging vs isotropic merging.** The Fisher merge beats isotropic merging on ChemProt and ACL-ARC, but not on SciERC [arxiv:2111.09832 tab. 1]. The paper reports both results. Do not generalise either direction. - **Fine-tuning vs probing on OOD data.** Linear probing beats fine-tuning OOD on 8 of the 10 shifts [arxiv:2202.10054 §4.1] [tab. 2]. The same paper reports fine-tuning ahead OOD on CIFAR-10 against CIFAR-10.1 and with a weaker MoCo-v1 checkpoint [arxiv:2202.10054 §4.3]. Both are the authors' results, and the condition differs. - **IWCV baseline.** The DEV paper's IWCV rows use a classifier-based density ratio on pre-trained features, not the Gaussian fit it describes for the original method [arxiv:2606.04665 §2.2] [§5]. Compare these numbers only with other IWCV results that use the same ratio estimator. - **Domain generalization.** The DG survey's text says RSC is best on both PACS and Office-Home, but its own table has RSC and CORAL tied at 82.0 on PACS [arxiv:2103.03097 §VII.C]. This page follows the table. - **RAG-end2end numbers within one paper.** Table 1 gives F1 12.20 and Table 4 gives 12.12 for the same model, RAG-original-QA+R on COVID-19 [arxiv:2210.02627 tab. 1] [tab. 4]. The text also gives an EM gain from reconstruction on news of 8.39 points, where Table 1 gives 5.69 [arxiv:2210.02627 §4.4]. Use the table values and check any quoted difference against them. - **Classification.** The Office-31 results come from a ResNet-50 classifier, while the DTN results are on digits [arxiv:1909.13589 tab. 1] [arxiv:1611.02200 tab. 2]. The two sets of gains are not on the same task. - **Reporting protocol on Office-Home.** The JUMBOT main table gives the end-of-training average, 70.0 [arxiv:2103.03606 tab. 1]. The appendix gives best-over-epochs averages instead, with JUMBOT at 70.1 and the other methods at different values, for example CDAN-E at 67.0 against 66.6 in the main table [arxiv:2103.03606 app. C, Table 4]. Compare numbers within one protocol. - **Epoch choice on target test data.** The aerial paper reports the gain at the epoch chosen on target test labels [arxiv:1905.03198 §5.2.4]. The JUMBOT appendix also reports best test accuracy over the run [arxiv:2103.03606 app. C]. Numbers chosen this way are not comparable to results with a held-out target split, and neither paper describes one. - **USPS and MNIST under two protocols.** The MEDA paper works on 256-feature vectors and reports USPS to MNIST at 72.1 and MNIST to USPS at 89.5 [arxiv:1807.07258 tab. 3]. Its table lists 1,800 USPS and 2,000 MNIST samples, while its text describes the full splits [arxiv:1807.07258 tab. 1] [§4.1]. The source-free paper works on images with a DIRT-T-style classifier and reports MNIST to USPS at 97.3 and USPS to MNIST at 99.3 [arxiv:2502.19316 tab. 1]. The inputs and the sample counts differ, so the pairs of numbers are not comparable. - **Source-only on VisDA.** The source-free GAN paper reports 52.4 for the source-only model on VisDA17 with ResNet-101 [arxiv:2502.19316 tab. 3]. The contrastive test-time paper reports 43.8 for source-only on VisDA-C with ResNet-101 [arxiv:2204.10377 tab. 1]. Both name the VisDA family and the backbone, but the two cards do not show the same split or averaging, so the baselines and the adapted scores should not be compared across the papers. ## Papers - [Unsupervised Cross-Domain Image Generation](/papers/1611.02200): DTN, which translates unpaired source samples into the target domain through a frozen feature function, and reports the translated set as domain adaptation for digits. - [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](/papers/1904.03451): aligns sketch and photo embeddings with a gradient-reversal domain loss, alongside a triplet loss and a semantic loss, for zero-shot retrieval. - [Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Images](/papers/1905.03198): restyles labelled Potsdam tiles to the Vaihingen look with a cycle-consistent translator, then fine-tunes the segmenter on them. - [All about Structure](/papers/1903.12212): DISE, which splits images into a shared structure code and a domain-specific texture code, and swaps textures to carry source labels into the target domain. - [Domain Adaptation for Semantic Segmentation with Maximum Squares Loss](/papers/1909.13589): replaces the entropy target term with a maximum squares term, adds image-wise class weights and a low-level head, for segmentation and Office-31 classification. - [Augmented SBERT](/papers/2010.08240): pseudo-labels sampled sentence pairs with a cross-encoder to train a bi-encoder, with pair sampling as the main design choice and a domain-adaptation setting on four datasets. - [StyleGAN-NADA](/papers/2108.00946): shifts a StyleGAN2 generator to a text-defined domain with no target images, using a directional CLIP loss and adaptive layer selection. - [Unbalanced minibatch Optimal Transport; applications to Domain Adaptation](/papers/2103.03606): replaces balanced minibatch transport with unbalanced transport and proves the estimator's properties; JUMBOT uses it for domain adaptation with a label-aware cost. - [Merging Models with Fisher-Weighted Averaging](/papers/2111.09832): averages model parameters with a diagonal Fisher weight, and reports merges with donor and domain-adapted checkpoints on GLUE and three scientific NLP tasks. - [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](/papers/2202.10054): explains when full fine-tuning does worse OOD than linear probing, and proposes LP-FT as a two-step recipe. - [Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering](/papers/2210.02627): RAG-end2end, which trains the DPR encoders and the BART generator jointly on domain text, refreshes the passage index in the background, and adds a sentence-reconstruction task. - [Test-Time Training with Masked Autoencoders](/papers/2209.07522): adapts a masked autoencoder to each test image by reconstruction, then resets the weights, and reports gains on ImageNet-C. - [BioMistral](/papers/2402.10373): continued pre-training of a 7B language model on medical text, with evaluation on medical question answering and merges with the base model. - [ChipNeMo](/papers/2311.00176): domain-adaptive pre-training of LLaMA2 on chip-design text, with tokenizer extension, alignment and a fine-tuned retriever for RAG. - [A Review on Deep Learning in UAV Remote Sensing](/papers/2101.10861): a literature review of deep learning on UAV imagery that codes 232 studies and lists domain adaptation among its open directions. - [A Short Survey of Viewing Large Language Models in Legal Aspect](/papers/2303.09136): a critical survey of LLMs in legal AI that organises adaptation, retrieval and agent mechanisms by the evidence they need; it proposes no experiments. - [Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation](/papers/2606.04665): DEV, a label-free estimate of target error for choosing among UDA models, using a feature-space density ratio and a control variate. - [FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare](/papers/1907.09173): builds a cloud CNN from encrypted federated updates, then fine-tunes each user's fully connected layers with a CORAL alignment term. - [Generalizing to Unseen Domains: A Survey on Domain Generalization](/papers/2103.03097): survey of domain generalization with a taxonomy of methods and a DeepDG benchmark on PACS and Office-Home. - [A Survey on Aspect-Based Sentiment Analysis](/papers/2203.01054): a survey of ABSA with a short section on cross-domain transfer, naming feature-based, data-based and combined methods. - [Deep Transfer Network with Joint Distribution Adaptation](/papers/1804.07265): a CNN trained on labeled fault data, with a marginal and class-wise MMD penalty on pseudo-labelled target features, evaluated on ten fault-diagnosis transfer tasks. - [Visual Domain Adaptation with Manifold Embedded Distribution Alignment](/papers/1807.07258): MEDA, which adds a geodesic flow kernel and a mu-weighted marginal and class-wise MMD, with a closed-form classifier, on fixed image features. - [Fact or Fiction: Verifying Scientific Claims](/papers/2004.14974): the SciFact benchmark and the VeriSci baseline, whose label model is trained first on FEVER and then on scientific claims. - [Model Adaptation: Unsupervised Domain Adaptation without Source Data](/papers/2502.19316): 3C-GAN, which adapts a source-trained classifier with a class-conditional generator, a weight anchor and a clustering term, without source data. - [Deep transfer learning for image classification: a survey](/papers/2205.09904): a survey that sorts transfer by source-target similarity and target size, ranks source types, and reports one Big Transfer extension to less similar targets. - [A Holistic Approach to Undesired Content Detection in the Real World](/papers/2208.03274): a moderation classifier adapted from public data to production traffic with Wasserstein domain-adversarial training, alongside active learning and synthetic data. - [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](/papers/2005.07503): continued pretraining of BERT-Large on COVID-19 tweets, followed by fine-tuning on five classification sets, with a released script that differs from the paper's schedule. - [Contrastive Test-Time Adaptation](/papers/2204.10377): AdaContrast, a source-free method that refines pseudo labels in a memory queue and trains a contrastive loss with same-label negatives removed. - [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](/papers/2311.16079): continued pretraining of Llama-2 at 7B and 70B on a medical corpus with a replay share, then per-benchmark finetuning and three inference modes. - [Unsupervised Domain Adaptation through Self-Supervision](/papers/1909.11825): trains one shared network on the labeled source task and on rotation, flip and patch puzzles over both domains, with no adversarial term. - [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](/papers/2310.17389): a benchmark showing that public toxicity tools transfer poorly to chatbot prompts; it proposes no adaptation method. - [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](/papers/2109.15290): continued pretraining of SciBERT on materials-science text, then task fine-tuning for NER, relation and abstract classification. - [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](/papers/2303.07601): a real two-vehicle LiDAR dataset with cooperative detection, tracking and a Sim2Real domain-adversarial benchmark.