Group-aware Label Transfer for Domain Adaptive Person Re-identification

In plain words

Person re-identification asks whether two photos, taken by different cameras, show the same person. A model trained on one set of cameras often does worse on another set. Unsupervised domain adaptation tries to close that gap without labels in the new setting: the model groups the new images by clustering, treats the groups as labels, and trains on them. Those groups are often wrong, and the usual fix is a slow, separate pass over the whole dataset between training rounds.

This paper fixes the groups while training. A balancing step borrowed from optimal transport reassigns each image so the groups stay roughly equal in size. The groups are built at several granularities at once, and they act as implicit attribute groups rather than labelled attributes. The reassigned groups feed back into training, and the improved features give better groups in the next step.

TL;DR

GLT (Group-aware Label Transfer) trains a target-domain person re-ID model on pseudo labels that are refined during training rather than in a separate offline pass §3.2. The refinement solves an entropic optimal-transport problem with a uniform-group prior, the Sinkhorn-Knopp algorithm, and runs it for several group counts at once §3.2 eq. 6 eq. 9. Author-reported Rank-1 is 92.2 for DukeMTMC to Market-1501 and 82.0 for Market-1501 to DukeMTMC tab. 1. The released code omits the memory-bank contrastive term and differs in several hyperparameters, so read "Paper vs code" before reproducing the numbers.

Problem

Unsupervised domain adaptive (UDA) re-ID adapts a model trained on a labelled source domain to an unlabelled target domain, with no target annotations §3.1. Clustering-based UDA methods alternate between clustering the target features and fine-tuning on the cluster labels §2 §3.1. The labels from clustering are noisy because of the domain gap and the clustering algorithm §1 §3.1. The label refinement in clustering is also offline: it needs a pass over the whole dataset, so it cannot correct labels during training §1. The target set has no identity labels, and the true number of identities is unknown, so a single cluster count k is hard to choose §1 §3.2.

Method

Overview. GLT pretrains the encoder on labelled source data §3.1 eq. 1. It clusters the target features into groups to get pseudo labels §3.1. Then, in each epoch, it trains the encoder on the current labels and refines the labels with a Sinkhorn-based label transfer §3.2 alg. 1. For several group counts k_1 to k_M, each group has its own classification head, and all heads share the encoder §3.2 eq. 8. A target memory bank with a weighted contrastive loss adds a term over target instances §3.3 eq. 10. The total objective is a weighted sum of three terms eq. 11.

Steps:

1. Source pretraining: minimise a cross-entropy term and a triplet term on labelled source images §3.1 eq. 1. The triplet term uses the hardest positive and negative cosine similarity in the batch §3.1.
2. Initial target labels: extract target features and cluster them into K groups, one clustering per k_m §3.1 §3.2 alg. 1.
3. Group probabilities: map each feature to K prototype vectors, then take a softmax over the similarities with a temperature τ §3.2 eq. 2.
4. Label transfer: the refined labels Q are the solution of a transport problem with uniform marginals, which forces the groups to have similar sizes §3.2 eq. 3 eq. 4. The entropic regularisation uses a parameter λ §3.2 eq. 5. The solution has the scaling form in eq. 6, computed with the iteration in eq. 7.
5. Multi-group loss: sum the group cross-entropy over the M groups §3.2 eq. 8. The group-wise transport problems are written in eq. 9.
6. Memory bank: keep a queue of target features, compare each mini-batch feature with the queue by cosine similarity, and apply a weighted contrastive loss §3.3 eq. 10. The paper says the queue size is a free hyperparameter §3.3.
7. Joint training: minimise the weighted total loss over source and target batches §4.2 eq. 11. The paper says the target pseudo labels are updated before each epoch §4.2.

# Group-aware label transfer, training loop  [alg. 1] [§3.2]
# Input:  labelled source (x_s, y_s); unlabelled target x_t;
#         group sizes k_1..k_M; Sinkhorn lambda
# Output: encoder theta; refined labels per group

def glt_train(source, target, group_sizes, lam, epochs):
    # Pretrain the encoder on labelled source data  [§4.2] [eq. 1]
    theta = pretrain_on_source(source)

    # Initial pseudo labels: cluster target features for each k  [§3.2]
    labels = [kmeans(encode(theta, target), k) for k in group_sizes]

    for epoch in range(epochs):
        # Joint batches of source and target; labels fixed in the epoch  [§4.2]
        for batch in joint_batches(source, target, labels):
            # Weighted sum of the three losses  [eq. 11]
            loss = weighted_total_loss(theta, batch)
            theta = adam_step(theta, loss)

        # Refine each group's labels by label transfer  [§3.2] [eq. 6]
        for m, k in enumerate(group_sizes):
            # Group probabilities from the current encoder  [§3.2]
            probs = group_probabilities(theta, target, k)
            # Sinkhorn scaling returns one label per image  [eq. 7]
            labels[m] = sinkhorn_labels(probs, lam)
    return theta, labels

# Sinkhorn label transfer for one group  [eq. 6] [eq. 7] [§3.2]
# Input:  probs (K, N) group probabilities P; lam (lambda)
# Output: one refined label per target image, (N,)

def sinkhorn_labels(probs, lam, tol=1e-2):
    # Raise the probabilities to the power lambda  [eq. 6]
    powered = probs ** lam                     # (K, N)
    # Column marginal 1/N, row marginal 1/K  [code: sinkhornknopp.py#L119-L133]
    cols = ones(N) / N
    # Alternate row and column scalings until the column vector settles
    # [eq. 7] [code: sinkhornknopp.py#L140-L146]
    while error > tol:
        rows = (1 / K) / (powered @ cols)      # (K,)
        new_cols = (1 / N) / (powered.T @ rows)  # (N,)
        error = sum(abs(cols / new_cols - 1))  # checked every 10 steps
        cols = new_cols
    # Refined plan diag(rows) P^lambda diag(cols)  [eq. 6]
    plan = rows[:, None] * powered * cols[None, :]
    # Each image takes its most likely group  [code: sinkhornknopp.py#L153-L155]
    return argmax(plan, axis=0)                # (N,)

Key equations, as numbered in the compiled paper. Equation numbers follow the order of the displayed equations, because the source reuses one label for two equations (see Pitfalls).

\mathcal L_{id} = -\frac{1}{m_t}\sum_{i=1}^{m_t}\log p(\tilde y_i\mid x_i), \qquad \mathcal L_{tri} = \frac{1}{m_t}\sum_{i=1}^{m_t}\max\left(0, s^n_i - s^p_i + m\right) eq. 1

\boldsymbol Q = \operatorname{diag}(\alpha)\,\boldsymbol P^{\lambda}\,\operatorname{diag}(\beta), \qquad \boldsymbol P = \boldsymbol C^{\top}\boldsymbol F eq. 6

\forall y:\ \alpha_y \leftarrow \left[\boldsymbol P^{\lambda}\beta\right]_y^{-1}, \qquad \forall i:\ \beta_i \leftarrow \left[\alpha^{\top}\boldsymbol P^{\lambda}\right]_i^{-1} eq. 7

\mathcal L_G = -\frac{1}{N}\sum_{m=1}^{M}\sum_{i=1}^{N} q_{k_m}(\tilde y_i\mid x_i)\,\log p_{k_m}(\tilde y_i\mid x_i) eq. 8

\sum_{m=1}^{M}\min_{\boldsymbol Q_{k_m}\in H_{k_m}}\Big(\langle \boldsymbol Q_{k_m}, -\log \boldsymbol P_{k_m}\rangle + \frac{1}{\lambda}\,\mathrm H(\boldsymbol Q_{k_m})\Big), \quad H_{k_m} = \{\boldsymbol Q\ge 0 \mid \boldsymbol Q\mathbf 1 = \boldsymbol r,\ \boldsymbol Q^{\top}\mathbf 1 = \boldsymbol c\} eq. 9

\mathcal L_{WCL} = \log\Big[1 + \sum_{j=1}^{L}\exp\big(\gamma\alpha^n_j(s^n_j - m)\big)\cdot\sum_{i=1}^{K}\exp\big(-\gamma\alpha^p_k(s^p_k - 1 + m)\big)\Big], \quad \alpha^n_j = [m + s^n_j]_+,\ \alpha^p_k = [1 + m - s^p_k]_+ eq. 10

\mathcal L_{target} = \lambda_{tri}\mathcal L_{TRI} + \lambda_g\mathcal L_G + \lambda_{wcl}\mathcal L_{WCL} eq. 11

Symbols as defined in the paper: m_t is the target mini-batch size; s^p and s^n are the hardest positive and negative cosine similarities; F stacks the N target features; C stacks the K prototypes; r and c are the row and column marginals; H is the entropy; γ is a scale factor, and L and K count the negative and positive pairs in the memory bank for one sample §3.1 §3.2 §3.3.

Algorithm 1 gives the same structure. The label-refinement step sits inside the inner loop over iterations, not only between epochs alg. 1.

Hyperparameters

Results

Author-reported. Setup: DukeMTMC-reID, Market-1501 and MSMT17 §4.1. The paper's text does not name the backbone; the repository README says ResNet-50 README.md and four GPUs (Training section) README.md. The baseline column is the strongest UDA method in the paper's table for each setting. The supervised row is the paper's BOT reference tab. 1. Source pretrain is the paper's own starting point tab. 1.

Source: tab. 1. The supervised row is printed with the same values in both MSMT17 columns.

Full GLT metrics from tab. 1:

Ablation on DukeMTMC → Market-1501 tab. 2. Column key: TMB = target memory bank; G = labels from several group counts on the baseline; LT = plain label transfer; GLT = the group-aware variant.

Loss ablation tab. 3. Mean average precision and top-1 for each variant.

The mAP margin over the next-best listed method is 3.5 points on DukeMTMC → Market-1501 and 3.1 points on Market-1501 → DukeMTMC §4.3. Both values match the MEB-Net row of tab. 1.

Paper vs code

Repository: https://github.com/zkcys001/UDAStrongBaseline, cloned at 9841a280a095b37be03fc6a44b1baabfc62b83b8. The paper's abstract footnote links this repository. Its GitHub handle matches the first author's email address in the paper's author block. No LICENSE file is present, so the code is cited but not quoted. The README names a different method (MLT, NeurIPS 2020) and says the method will be updated, so treat the code as a snapshot.

• Training loss. Paper: three terms, including the weighted contrastive loss over a memory bank eq. 11 §3.3. Code: the step is cross-entropy plus a soft triplet loss only; the contrastive call is commented out trainers.py trainers.py. The memory bank is built but never used in the loss sbs_trainkmeans.py.
• Multi-group loss. Paper: cross-entropy summed over M group heads eq. 8 §3.2. Code: the cross-entropy uses only head 0, labelled with the first group's labels trainers.py. The second group's labels feed the triplet term, selected by --choice_c 1 GLT_kmeans_duke2market.sh trainers.py. The third group's (800) labels appear in no loss term, as read from this code.
• Group sizes. Paper: 500, 1000, 1500, 2000 for K-means §4.2. Code: 600, 700, 800 in the default and in the GLT script sbs_trainkmeans.py GLT_kmeans_duke2market.sh.
• Triplet margin. Paper: m = 0.3 §4.2, but m = 0 in §3.1 §3.1. Code: margin 0.0 trainers.py, with a soft-target triplet over the hardest pair, not the hinge of eq. 1 triplet.py.
• Label smoothing. Paper: none mentioned. Code: the cross-entropy is built with epsilon set to False, which is zero, so no smoothing is applied trainers.py crossentropy.py.
• Timing of label refinement. Paper: Algorithm 1 refines inside the inner iteration loop alg. 1; §4.2 says per epoch §4.2. Code: once per epoch, after training sbs_trainkmeans.py.
• Sinkhorn λ. Paper: "high", no value §3.2. Code: 25, unused arguments commented sbs_trainkmeans.py.
• Row marginal. Paper: eq. 9 writes the row marginal as (1/N)·1 eq. 9. Code: 1/K over the K groups sinkhornknopp.py.
• Convergence test. Paper: Algorithm 1 loops while the L1 change of α is below 0.1 as typeset alg. 1. Code: loops while the column-ratio error exceeds 1e-2, checked every 10 steps sinkhornknopp.py.
• Hard labels. Paper: the matrix Q is used as the pseudo label §3.2. Code: the argmax over groups of the plan is the hard label sinkhornknopp.py.
• Model used for labels. Paper: group probabilities come from the current model §3.2 alg. 1. Code: the Sinkhorn step and the initial and final clustering run on the EMA copy of the model sbs_trainkmeans.py sbs_trainkmeans.py. The EMA is not described in the paper.
• Precision. Code: the Sinkhorn step uses float64 on the CPU sbs_trainkmeans.py. Paper: not stated.
• Domain-specific BatchNorm. Code: every BatchNorm layer is replaced by separate source and target branches sbs_trainkmeans.py dsbn.py. Paper: not mentioned.
• Batch composition. Paper: 64 source images from 4 identities, 16 each, and 64 target images from 4 pseudo identities §4.2. Code: the sampler gives 16 identities with 4 images each, per the help text sbs_trainkmeans.py sampler.py.
• Source pretraining. Paper: 30 epochs, LR ÷10 every 10 epochs §4.2. Code: 80 epochs, milestones 40 and 70 pretrain_dukemtmc.sh. Margin 0.0 pretrain_dukemtmc.sh.
• Joint-training schedule. Paper: no schedule for joint training §4.2. Code: 300 iterations per epoch regardless of --iters sbs_trainkmeans.py; LR multipliers of 0.1 and 0.01 sbs_trainkmeans.py; rounds of 40 epochs repeated 11 times, the last with 80 GLT_kmeans_duke2market.sh sbs_trainkmeans.py. The paper does not describe these rounds.
• Last epoch. Code: at the final epoch the loop refits the clusters and then exits before training or evaluating that epoch sbs_trainkmeans.py.
• Clustering details. Paper: K-means, no initialisation given §4.2. Code: target features are L2-normalised sbs_trainkmeans.py; the first round uses faiss K-means on GPU with 300 iterations sbs_trainkmeans.py; later rounds use sklearn K-means with the seed sbs_trainkmeans.py.
• DBSCAN. Paper: eps values in a set §4.2, with DBSCAN results in the supplement. Code: fixed eps = 0.6 in its DBSCAN branch, and no DBSCAN GLT script sbs_trainkmeans.py. The supplement is not in the repository.
• Reported numbers. Paper: GLT Duke → Market top-1 92.2 tab. 1. README: 92.7 for the same method and setting README.md. The mAP matches at 79.5.
• Other settings. The repository has a GLT script only for DukeMTMC → Market-1501 GLT_kmeans_duke2market.sh. No script for Market-1501 → DukeMTMC or MSMT17 is present.

No discrepancies were found for the input size, the augmentation list, or the optimizer family.

Pitfalls

• Repeated symbol m. In §3.1 m is 0, in §4.2 it is 0.3 for the triplet margin, and eq. 10 uses m as the contrastive margin §3.1 §4.2 eq. 10. The code uses 0.0 for the triplet. Take the margin from the code if you follow the code, and from §4.2 if you follow the paper.
• Eq. 10 index mismatch. The sum over i uses α^p_k and s^p_k, with k instead of i eq. 10. Read it as a sum over the positive pairs. The released code does not implement this loss.
• Marginal naming in Algorithm 1 and eq. 7. The text sets α_0 = c and β_0 = w eq. 7. The dimensions of c and w do not match the rows and columns of P. The code uses 1/K for rows and 1/N for columns, and this card follows the code sinkhornknopp.py.
• Sinkhorn exponent. With λ = 25 sbs_trainkmeans.py, raising a probability to the power 25 underflows float64 once p falls below about 1e-12. A zero row or column then makes the scaling divide by zero. This is arithmetic from the code, not a run. The code uses NaN-aware sums and argmax sinkhornknopp.py sinkhornknopp.py.
• Equal-size prior. The uniform marginals force groups of similar size eq. 4. If the true number of identities differs from k, the groups must split or merge real identities. The compiled paper has no sensitivity analysis for k or λ.
• Gains in §4.4 do not match Table 2. The text gives +11.3% rank-1 and +3.0% mAP for group-aware label transfer §4.4. The table rows Baseline + G and Baseline + G + GLT differ by +6.7 rank-1 and +15.7 mAP tab. 2. The paper does not say which rows it compares.
• Source-pretrain numbers differ between tables. Duke → Market source pretrain is 33.9 / 65.3 in Table 1 and 38.5 / 65.7 in Table 2 tab. 1 tab. 2.
• Table 3 caption. The caption says "w/o WCL denotes using the weighted contrastive loss" tab. 3. It seems to mean the variant without it. The caption cites a different key from the text (he2019momentum rather than sun2020circle).
• Duplicate equation label. The source uses the label eq:2 for both the prototype loss and L_G, so LaTeX cross-references can resolve to the wrong equation. This card uses positional numbers.
• Missing values. The paper does not state the weight decay, dropout, iterations per epoch, joint-training epochs, backbone name, EMA, or the WCL scale and margins §4.2 §3.3. The code fills most of these; see Paper vs code.
• Compute cost. The paper says each Sinkhorn update costs O(MNK) and is "relatively quick" §3.2. The README reports about 35 hours for GLT with K-means on four GPUs README.md. The code also repeats 11 outer rounds, so the total cost is larger than one run of 40 epochs (inferred from the script).
• GPU clustering. The first round uses faiss K-means with gpu=True sbs_trainkmeans.py. The README installs faiss-gpu 1.6.3 README.md (installation section).
• Unlicensed code. The repository has no LICENSE, so its code is cited, not quoted. The pseudocode in this card is written from scratch.

Lineage

• Builds on: the clustering-based UDA pipeline, which alternates pseudo labels and fine-tuning §2. The paper's Table 1 compares against PUL, SSG, PAST, MMT, NRMT, ECN, MEB-Net and others tab. 1; their arXiv ids are not on this wiki page.
• Builds on: Sinkhorn-Knopp scaling for optimal transport, cited as Cuturi (2013) §3.2 and self-labelling, cited as Asano et al. (2019) §3.2. Neither has a card here.
• Builds on: group-based description of people, cited as Kim et al. (2020) §3.2. No card here.
• Builds on: the target memory bank for contrastive learning, a queue of past features cited as He et al. (2019) §3.3, and the weighted contrastive loss, cited as Sun et al. (2020) §3.3. No cards here.
• Same authors: the uncertainty-based DBSCAN method 2012.08733 is listed next to GLT in the repository README README.md.
• Superseded by: not known from content in this wiki.