Visual Domain Adaptation with Manifold Embedded Distribution Alignment

In plain words

A classifier trained on one collection of labelled images often has to work on a different collection, and the new collection has no labels. The two collections look different, so a classifier that works on one drifts on the other. MEDA tackles this in two steps. First it reshapes the features with a geometric trick: it describes each collection by its main directions, treats those as points on a curved space, and follows a smooth path between them. This removes part of the distortion in the raw features. Second, it pulls the two collections together, both overall and class by class.

The novel part is how much weight each kind of pulling gets. The weight is estimated from the unlabelled data itself, because the new collection has no labels to tune it on. The classifier is then fitted in one closed-form step rather than by an iterative training loop. The authors report better accuracy than earlier traditional and deep methods on image benchmarks.

TL;DR

MEDA is an unsupervised domain adaptation method (no target labels) for image features. It works in three parts: a geodesic flow kernel for manifold features §3.3, a dynamic weight mu between marginal and class-conditional MMD §3.4, and a closed-form kernel solution eq. 16. The authors report 73.2% average accuracy over 28 tasks against 69.7% for JGSA §4.4. Use it when source labels exist, target labels do not, and features are fixed vectors (SURF or DeCAF6 in the paper).

Problem

Source and target domains share the feature and label spaces, but the marginal P(x) and the class-conditional Q(y|x) both differ. The goal is a classifier f for the unlabelled target domain, learned from labelled source data §3.1.

Two challenges are named. (1) Degenerated feature transformation: alignment in the original feature space is hampered by feature distortion, and subspace learning alone does not remove the divergence §1. (2) Unevaluated alignment: earlier methods weight marginal and conditional alignment equally, although their relative importance differs by task §1 §3.4.

Method

Prose overview. The approach has three parts, combined in one objective eq. 1 eq. 7 §3.2:

1. Manifold feature learning §3.3. Take the PCA subspaces of source and target and use the geodesic flow kernel G between them (GFK, Gong et al. 2012) eq. 2. Transform features as z = sqrt(G) x. The sqrt is computed with the Denman-Beavers iteration §3.3.
2. Laplacian regularisation §3.5. Build a symmetric p-nearest-neighbour graph with cosine similarity, where two points are joined if either is among the other's p nearest neighbours eq. 13. The regulariser is R_f = tr(beta^T K L K beta) with L = D - W eq. 14.
3. Dynamic distribution alignment §3.4. The MMD term is (1 - mu) times a marginal term plus mu times a sum of per-class terms eq. 3. Each MMD is computed with a projected MMD on f eq. 4. Class-conditional terms use the target's soft labels, which start from a base classifier trained on the source and are then refined by the iteration §3.4 alg. 1.
4. Adaptive factor mu §3.4. Estimate mu from A-distances: d_A = 2(1 - 2 epsilon(h)) for a linear classifier h discriminating two sets eq. 5, and mu-hat = 1 - d_M / (d_M + sum_c d_c) eq. 6. The estimate is recomputed every iteration.
5. Closed-form classifier §3.5. With the representer theorem eq. 8 and the square loss eq. 7, the coefficients beta come from a single linear system eq. 16. No separate classifier is trained (the paper contrasts this with JGSA and CORAL) §3.5.

Algorithm 1 gives the loop. It returns f after the iteration, and the paper says to repeat "until convergence" alg. 1. The experiments use T = 10 §4.3.

# MEDA: Manifold Embedded Distribution Alignment  [alg. 1]
# Input:  source_x (n, D), source_y (n,) labels 1..C, target_x (m, D)
# Output: scores for every sample (n + m, C)

def meda(source_x, source_y, target_x, dim, lam, eta, rho, p, T):
    n = len(source_y)
    m = len(target_x)
    # Manifold features from the geodesic flow kernel  [§3.3] [eq. 2]
    feats = geodesic_flow_features(source_x, target_x, dim)    # (n + m, dim)
    # Laplacian of the p-NN cosine graph  [§3.5] [eq. 13] [eq. 14]
    laplacian = graph_laplacian(feats, p)                      # (n + m, n + m)
    # RBF kernel, bandwidth = variance of the inputs  [§4.3]
    kernel = rbf_kernel(feats)                                 # (n + m, n + m)
    # Source indicator A; one-hot labels, zero rows for the target  [§3.5]
    source_mask = diagonal_indicator(n, m)                     # (n + m, n + m)
    label_matrix = one_hot_with_zero_target_rows(source_y, m)  # (n + m, C)
    # Soft labels for the target from a base classifier  [§3.4]
    target_scores = base_classifier(feats, source_y)           # (m, C)
    for iteration in range(T):    # the paper says "until convergence"  [alg. 1]
        # Adaptive factor mu from A-distances  [§3.4] [eq. 5] [eq. 6]
        mu = estimate_mu(feats, source_y, target_scores)
        # MMD matrix: marginal and per-class terms, mixed by mu  [§3.4]
        # [eq. 10] [eq. 11] [eq. 12]
        mmd = (1 - mu) * marginal_mmd(feats) + mu * conditional_mmd(
            feats, target_scores
        )
        # Coefficients from the closed-form solution  [eq. 16]
        beta = solve(
            (source_mask + lam * mmd + rho * laplacian) @ kernel
            + eta * identity,
            source_mask @ label_matrix,
        )
        # Scores for all samples; target rows are the new soft labels  [alg. 1]
        scores = kernel @ beta
        target_scores = scores[n:]    # the target rows follow the n source rows
    return scores

Key equations. The dynamic alignment and the adaptive factor eq. 3 eq. 6:

\overline{D_f}(\mathcal{D}_s,\mathcal{D}_t) = (1-\mu)\,D_f(P_s,P_t) + \mu \sum_{c=1}^{C} D^{(c)}_f(Q_s,Q_t) eq. 3

\hat{\mu} \approx 1 - \frac{d_M}{d_M + \sum_{c=1}^{C} d_c} eq. 6

The closed-form coefficients, with M = (1 - mu) M_0 + mu sum_c M_c eq. 10:

\boldsymbol{\beta}^\star= \big((\mathbf{A} + \lambda \mathbf{M} + \rho \mathbf{L}) \mathbf{K} + \eta \mathbf{I}\big)^{-1} \mathbf{A} \mathbf{Y}^T eq. 16

Hyperparameters

Values the paper states. Anything not in the paper is marked "not stated".

Results

Author-reported. Setup: accuracy on the target domain (fraction correct) §4.3. Office+Caltech10 with 12 tasks and SURF (800 features) or DeCAF6 (4,096 features) §4.1 tab. 2 tab. 4. USPS+MNIST and ImageNet+VOC with 4 tasks §4.1 tab. 3. Baselines are 12 traditional methods (counting 1NN, SVM and PCA separately) and 5 deep methods §4.2. Hardware for timing: Intel Core i7-4790, 24 GB RAM §4.7.

Headline numbers:

The text reports a 3.5-point average gain over JGSA and an 11.6% average error reduction, and says MEDA is best in 21 of 28 tasks §4.4. The per-table numbers above check against the text: 28-task mean = (12 × 52.7 + 4 × 75.9 + 12 × 92.8) / 28 = 73.2 tab. 2 tab. 3 tab. 4.

Full results. Table 2 (SURF, accuracy %) tab. 2; bold = best in the table:

Table 3 (USPS+MNIST and ImageNet+VOC, accuracy %) tab. 3; "-" = not available in the paper:

Table 4 (Office+Caltech10, DeCAF6, accuracy %) tab. 4; "-" = not available. The table is split into three blocks only for layout: 4a and 4b hold the traditional methods, 4c the deep methods and MEDA.

Table 4a, traditional methods (part 1):

Table 4b, traditional methods (part 2):

Table 4c, deep methods and MEDA (part 3):

Table 5, effect of manifold feature learning (mean (std) accuracy, %, over mu in {0, 0.1, ..., 1}) tab. 5:

Table 6, accuracy at the grid-search optimum mu_opt versus the estimate mu-hat tab. 6:

Table 7, running time (s) tab. 7:

Dataset statistics as the paper lists them tab. 1. Note the conflict with the text in the Pitfalls section.

No figures are included, because figure support is not yet in the build; they are described and cited instead. Fig. 1 shows two target domains against one source domain, which motivates the adaptive factor fig. 1. Fig. 2 is the main-idea diagram fig. 2. Fig. 3 plots accuracy against mu on the original and manifold spaces, with the best baseline as a dashed line fig. 3. Fig. 4 is a component ablation (SRM, DA, Laplacian) fig. 4. Fig. 5 shows sensitivity to d, p and lambda and the convergence curve fig. 5. The paper says the ablation supports each component §4.5.3 fig. 4.

Paper vs code

Code: the authors' repository wenchieh/MEDA at commit b74e0765d23516da3b178b35910d393e4c7f49b8. It is an official repository: its owner is Wenjie Feng, the paper's second author (GitHub profile name matches). The paper's project page is transferlearning.xyz, which I could not open (certificate error), so the link is by owner match, not by a link in the paper. Searched: the paper's URL, the GitHub user search for "MEDA" (found this repo), and the authors' transferlearning repository (its README does not mention MEDA). The repository has no LICENSE file and GitHub reports no license, so no code is quoted; all code is cited by path and line. The repository contains MATLAB (matlab/) and a partial Python port (MEDA.py). The MATLAB version is the main reference. The README says the Python port is for reference only and does not implement the Laplacian readme.md.

Discrepancies, each with both citations:

1. Laplacian normalisation. Paper: L = D - W, unnormalised §3.5 eq. 14. Code: symmetric normalised L = I - D^{-1/2} W D^{-1/2} MEDA.m.
2. Neighbour set includes the point itself. Paper: p-nearest neighbours §3.5 eq. 13. Code: takes k + 1 = p + 1 neighbours, including the point itself lapgraph.m lapgraph.m, then symmetrises by max(W, W') lapgraph.m. Self-connections are kept by default lapgraph.m.
3. Marginal term scaled by C. Paper: M_0 entries are 1/n², 1/m², and -1/(mn) eq. 11, with no factor C. Code: multiplies the marginal matrix by the number of classes C MEDA.m MEDA.py. This shifts the effective balance between marginal and conditional terms by a factor of C, and so changes what mu means.
4. Frobenius normalisation of M. Paper: M = (1 - mu) M_0 + mu sum_c M_c, unnormalised §3.5 eq. 10. Code: divides the mixed M by its Frobenius norm MEDA.m MEDA.py. This is not in the paper and makes lambda's effective scale data-dependent.
5. Adaptive factor uses the mean, not the sum. Paper: mu-hat = 1 - d_M / (d_M + sum_c d_c) eq. 6. Code: averages the per-class A-distances, then mu = mean(d_c) / (mean(d_c) + d_M) estimate_mu.m estimate_mu.m MEDA.py. The two agree only if the sum and the mean coincide. Code also clips mu to [0, 1] and sets mu = 0 for values at or below 1e-3 estimate_mu.m MEDA.py, which the paper does not state.
6. RBF bandwidth. Paper: bandwidth = variance of inputs §4.3. Code (MATLAB): sigma is the square root of the mean column norm, computed after every column is unit-normalised, which makes sigma = 1 and K = exp(-D / 2) MEDA.m MEDA.m MEDA.m. Code (Python): gamma = 1 with K = exp(-gamma D), i.e. exp(-D), a different kernel MEDA.py MEDA.py. The "sigma = 1" result follows from the code's own normalisation; the code does not state it.
7. Soft labels become hard labels. Paper: the base classifier gives soft labels, refined as f(Z_t) at each iteration §3.4 alg. 1. Code: uses hard labels from 1-NN, then the arg-max of the scores at each iteration MEDA.m MEDA.m MEDA.m.
8. Base classifier. Paper: "a base classifier trained on D_s", not named §3.4. Code: 1-NN MEDA.m.
9. Linear classifier for the A-distance. Paper: "a linear classifier" §3.4. Code: MATLAB fitclinear with SVM estimate_mu.m; Python LinearSVC MEDA.py. Both fit and score on the same samples, so the error is in-sample estimate_mu.m MEDA.py.
10. Feature normalisation. Paper: not stated. Code: each column of the stacked features is L2-normalised MEDA.m. The demo also divides each row by its sum and z-scores the features demo_office_caltech_surf.m demo_office_caltech_surf.m.
11. Subspace dimension in GFK. Paper: d-dimensional PCA subspaces for both domains §3.3. Code: the source basis is built from the full pca(Xs) plus its null space, and only the target uses the first d columns GFK_Map.m.
12. Square root of G. Paper: Denman-Beavers iteration §3.3. Code: MATLAB matrix power G^0.5 GFK_Map.m. The results should agree, but the algorithm is different.
13. Label matrix shape. Paper: beta in R^{(n+m)×1} with Y a row of labels §3.5. Code: one-hot (n+m) × C matrix, so beta has one column per class MEDA.m MEDA.m. The paper's scoring rule is not stated for C > 2; the code uses arg-max MEDA.m.
14. Stopping rule. Paper: "until convergence" alg. 1. Code: fixed T iterations MEDA.m. The code's default is T = 10, the same as the paper's setting §4.3 MEDA.m.
15. Which accuracy is returned. Paper: not stated. Code: returns the accuracy after the last iteration MEDA.m MEDA.m.
16. Defaults differ from the paper. The MATLAB defaults set lambda = 1.0 MEDA.m. The demo sets the paper's values demo_office_caltech_surf.m. The Python defaults are dim = 30 and lamb = 1 MEDA.py, and its demo uses lamb = 10 MEDA.py.
17. Python port bugs. The Laplacian is set to 0 MEDA.py, as the README says readme.md. Row 0 of the label matrix is zeroed in every column except class 1, regardless of that sample's class MEDA.py. Do not copy this line.

The README reports ResNet-50 tables and claims MEDA is the best traditional method on Office-31 and Office-Home readme.md. These numbers are not in the paper, so they are not used here.

Pitfalls

• The kernel bandwidth is not defined by a formula. The paper says "variance of inputs" §4.3. The MATLAB code effectively uses sigma = 1 on unit-norm columns, and the Python code uses a different kernel (see Paper vs code, items 6 and 17). Pick one and test it; results may depend on it.
• mu can fall outside [0, 1]. The A-distance is 2(1 - 2 epsilon), so it can be negative when the in-sample error exceeds 0.5 eq. 5. The denominator d_M + sum_c d_c can approach zero eq. 6. The paper states neither clipping nor a floor; the code clips to [0, 1] and uses a small floor (1e-3) estimate_mu.m estimate_mu.m.
• The A-distance is computed in-sample. Both the code and the formula train and score on the same samples estimate_mu.m. The paper does not say whether a held-out split is used.
• Class labels must be integers 1..C. The code loops over 1..C MEDA.m MEDA.m and the A-distance loops over the same range estimate_mu.m. Other label encodings will silently break the class terms. The paper does not state an encoding.
• The linear system is large. Each iteration solves an (n+m) × (n+m) system eq. 16 alg. 1. For the 10,717-sample ImageNet→VOC-size task, one dense double-precision matrix is about 0.9 GB (10,717² × 8 bytes). Several such matrices are formed (K, M, L). Reported time: 2,931.7 s for I→V on the authors' machine tab. 7 §4.7. This memory figure is a derived estimate, not a number from the paper.
• The first iteration depends on the base classifier. The paper does not name it, and the code uses 1-NN MEDA.m. Results may depend on this choice.
• The paper's sample counts conflict with its text. Table 1 lists 1,800 USPS and 2,000 MNIST samples tab. 1. §4.1 describes the full splits of 7,291/2,007 and 60,000/10,000 §4.1. Table 7 uses 3,800 samples for M → U tab. 7, which matches Table 1. The subsampling is not explained.
• Feature preprocessing is not stated. The paper uses extracted SURF and DeCAF6 features §4.3 but does not say how they were normalised. The demo's z-scoring is the only example demo_office_caltech_surf.m.
• The sensitivity claims are the authors' sweeps. Robustness to d, p, lambda, eta and rho is shown on "randomly selected tasks" §4.6. The full sweeps for rho and eta are in the supplementary file, which is not in the source archive §4.6.2.
• Office-31 results are in the supplementary file. The paper says the Office-31 numbers and the parameter-setting method are in a supplementary file §4.4 §4.3. That file was not read, and the card does not include Office-31 numbers.
• Seeds are not stated. The only spread the paper reports is the standard deviation in Table 5, which is over mu, not over random seeds tab. 5.
• The code is not a fully faithful reproduction of the paper. Use the paper's equations as the specification, and the code as a reference for the implementation choices it makes (items 1 to 7 in Paper vs code).

Lineage

• Builds on: the geodesic flow kernel (Gong et al., 2012) §3.3 for manifold features, and joint distribution adaptation (JDA, Long et al., 2013) §2 §3.4 for the marginal and conditional MMD terms.
• Uses the projected MMD of Quanz et al. (2009) §3.4. The Laplacian regulariser is from Belkin et al. (2006) §3.5.
• Compared against: twelve traditional baselines (three generic ones, plus nine domain-adaptation methods) and five deep baselines, one of them a plain convolutional network §4.2. Table 2 to Table 4 give the full comparison.
• Related: subspace and distribution alignment methods discussed in §2 (SA, SDA, CORAL, SCA, DIP, DMM).