wikillm

Paper · 2022-11-07 · cs.CV

Body Part-Based Representation Learning for Occluded Person Re-Identification

BPBreID learns a pixel-wise body-part classifier, supervised by coarse parsing masks from PifPaf poses, and pools one feature per part. GiLt applies the identity loss to holistic features only and a part-averaged triplet loss to part features only. Inference compares mutually visible parts only. Author-reported Occluded-Duke: 66.7 rank-1 / 54.1 mAP with ResNet-50, 75.1 / 62.5 with HRNet-W32.

Part-based featuresMetric learningLabel smoothingperson-re-identificationocclusionbody-partsmetric-learning

arXivPDFCodeMarkdown15 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Re-identification means recognising the same person across camera images. It gets hard when other people or objects hide part of the person. This paper's model splits the person into body parts, such as head, torso, legs and feet, and learns a map that says which pixels belong to which part. It turns each part into its own feature vector. Two people are then compared only on the parts that both images can see.

Training is the tricky part. One body part is often not distinctive enough to tell two people apart, so the usual identity objective does not fit part features. The paper's fix is to apply the identity loss to whole-person features and a triplet loss to the part features only. The authors report gains on both occluded and ordinary person re-identification benchmarks.

TL;DR

BPBreID is a person re-identification model that learns K body-part attention maps from coarse parsing masks and pools one feature per part §3.1 §3.2. GiLt gives the identity classification objective to whole-person embeddings and a part-averaged metric objective to the part embeddings alone §3.3. At inference it compares a pair only over mutually visible parts §3.4. Author-reported, the HRNet-W32 model beats every listed method on Occluded-Duke at rank-1 and mAP; the ResNet-50 model does not beat PFD there tab. 1.

Problem

Global re-identification features are built from the whole image, so they pick up occluders and other pedestrians, and they cannot align body parts between two partly hidden images §1. Part-based methods fix alignment, but two things go wrong when training them with identity labels: single parts are often not discriminative, and standard identity and triplet losses assume that global appearance separates identities §1. Also, re-identification datasets have no body-part annotation, and off-the-shelf pose or parsing tools transfer poorly to them §1.

Method

The model has two modules and one training objective §3. The body part attention module §3.1 applies a 1×1 convolution and a softmax over K+1 classes, background plus K parts, to the backbone feature map G eq. 1. It is trained with a label-smoothed cross-entropy against coarse parsing labels §3.1.3 eq. 2. Those labels come from PifPaf pose outputs that are merged into K regions §3.1.2 § Supplementary materials (Human Parsing Labels Generation with PifPaf). The global-local module §3.2 then pools G with the part maps eq. 3. It produces a foreground embedding (the maximum of the part maps), K part embeddings, a global average embedding and a concatenation of the part embeddings §3.2.1. It also gives each part a binary visibility score, which the paper computes as 1 when the part map exceeds a threshold of 0.4 §3.2.2 eq. 4. The paper sets the visibility of the holistic embeddings to 1 §3.2.2.

Training minimises a weighted sum of the part attention loss and GiLt eq. 5. GiLt applies the identity loss to the global, foreground and concatenated embeddings and a part-averaged triplet loss to the K part embeddings only §3.3.1 eq. 6. The identity loss uses label smoothing and the BNNeck trick §3.3.1. The part-averaged triplet loss averages the Euclidean distance over the K part pairs of two samples eq. 7 and then applies the batch-hard triplet with a margin §3.3.2 eq. 8. Visibility is not used in training §3.2.2.

At inference the query-gallery distance uses only the foreground embedding and the K part embeddings §3.4. Each pair is weighted by the product of the two visibility scores, and pairs with no mutually visible embedding get an infinite distance §3.4 eq. 9. The global and concatenated embeddings are not used at inference §3.4. The paper illustrates the model with K=4 for head, torso, legs and feet fig. 2.

The training loop below follows the paper's equations. Values not stated in the paper are marked in the Paper vs code section.

Pseudocode
# Training step of BPBreID  [§3.3] [eq. 5]
# Input:  images (N, 3, 384, 128); part_masks (N, K+1, H, W); pids (N,)
# Output: scalar loss

def training_step(images, part_masks, pids):
    # Backbone feature map G with C channels  [§3.2]
    feature_map = backbone(images)              # (N, C, Hf, Wf)

    # Per-pixel scores over K+1 classes, class 0 is background  [§3.1.1] [eq. 1]
    pixel_scores = pixel_classifier(feature_map)  # (N, K+1, Hf, Wf)
    part_maps = softmax(pixel_scores, dim=1)      # (N, K+1, Hf, Wf)

    # Part attention loss against the coarse parsing labels  [§3.1.3] [eq. 2]
    target = resize_and_argmax(part_masks, feature_map.shape)
    loss_pa = attention_loss(pixel_scores, target)

    # Foreground map: pixelwise max over the K part maps  [§3.2.1]
    foreground_map = max(part_maps[:, 1:], dim=1)

    # Weighted pooling of G per part and for the foreground [§3.2.1] [eq. 3]
    part_feats = weighted_pool(feature_map, part_maps[:, 1:])     # (N, K, C)
    foreground_feat = weighted_pool(feature_map, foreground_map)  # (N, C)

    # Global average pooling and concatenation of the part features  [§3.2.1]
    global_feat = average_pool(feature_map)                       # (N, C)
    concat_feat = flatten(part_feats)                             # (N, K*C)

    # Identity loss on the three holistic embeddings, with BNNeck and label
    # smoothing  [§3.3.1] [eq. 6]
    loss_id = sum(
        id_loss(f, pids)
        for f in [global_feat, foreground_feat, concat_feat]
    )

    # Part-averaged triplet loss on the K part embeddings only, margin 0.3
    # [§3.3.2] [eq. 7] [eq. 8] [§4.2]
    loss_tri = part_averaged_triplet(part_feats, pids, margin=0.3)

    # Overall objective with lambda_pa = 0.35  [§3.3] [eq. 5]
    return 0.35 * loss_pa + loss_id + loss_tri
Pseudocode
# Part-averaged triplet loss  [§3.3.2] [eq. 7] [eq. 8]
# Input:  parts (N, K, D); pids (N,); margin alpha
# Output: scalar loss

def part_averaged_triplet(parts, pids, margin):
    # Distance between samples i and j: mean of the K part distances  [eq. 7]
    dist = mean_over_parts(euclidean(parts[:, None], parts[None, :]))
    # Hardest positive and hardest negative for each anchor  [§3.3.2]
    d_pos = max(dist where pids match and i != j)
    d_neg = min(dist where pids differ)
    # Hinge with margin alpha  [eq. 8]
    return mean(relu(d_pos - d_neg + margin))
Pseudocode
# Query-gallery distance with mutual visibility  [§3.4] [eq. 9]
# Used embeddings: foreground and the K part embeddings only  [§3.4]
# Visibility per part: v_i = 1 if max over pixels of M_i > 0.4  [§3.2.2] [eq. 4]

def pair_distance(query, gallery):
    # Weight v_q * v_g per used embedding  [§3.4] [eq. 9]
    weight = query.visible * gallery.visible
    # Euclidean distance per used embedding  [§3.4] [eq. 7]
    dist = euclidean(query.embeddings, gallery.embeddings)
    # No mutually visible embedding means an infinite distance  [§3.4]
    if sum(weight) == 0:
        return infinity
    # Visibility-weighted mean over the used embeddings  [eq. 9]
    return sum(weight * dist) / sum(weight)

Key equations, as printed in the paper:

S = \text{softmax}(G P^{T}), \quad P \in R^{(K+1) \times C} eq. 1

L_{pa} = -\sum_{k=0}^{K}\sum_{h=0}^{H-1}\sum_{w=0}^{W-1} q_{k}\,\log S_{k}(h,w), \quad q_k = \begin{cases} 1-\frac{N-1}{N}\varepsilon & Y(h,w)=k \\ \frac{\varepsilon}{N} & \text{otherwise} \end{cases} eq. 2

f_i = \frac{\sum_{h,w} G(h,w)\, M_i(h,w)}{\sum_{h,w} M_i(h,w)}, \quad i \in \{\text{f},1,\dots,K\} eq. 3

v_i = \begin{cases} 1 & \max_{h,w} M_i(h,w) > \lambda_v \\ 0 & \text{otherwise} \end{cases} eq. 4

L = \lambda_{pa} L_{pa} + L_{GiLt} eq. 5

L_{GiLt} = L_{id} + L_{tri} = \sum_{i \in \{g,f,c\}} L_{CE}(f_i) + L_{tri}^{parts}(f_1,\dots,f_K) eq. 6

d_{parts}^{ij} = \frac{1}{K}\sum_{k=1}^{K} dist_{eucl}(f_k^i, f_k^j) eq. 7

L_{tri}^{parts} = \left[ d_{parts}^{ap} - d_{parts}^{an} + \alpha \right]_{+} eq. 8

dist_{total}^{qg} = \frac{\sum_{i \in \{f,1,\dots,K\}} v_i^{q} v_i^{g}\, dist_{eucl}(f_i^{q}, f_i^{g})}{\sum_{i \in \{f,1,\dots,K\}} v_i^{q} v_i^{g}} eq. 9

Symbols follow the paper: G is the backbone map, M_i the attention map of part i, N the batch size in eq. 2 as printed (see Pitfalls), lambda_v = 0.4 and lambda_pa = 0.35.

Hyperparameters

NameValueWhereNotes
BackboneResNet-50 (RN) with last stride 1; HRNet-W32 (HR) and ResNet-50-ibn (RI) also used§4.2ImageNet pre-training §4.2; code default last stride 1 default_config.py
Input size256×128 for RN; 384×128 for HR and RI§4.2Occluded-Duke train config uses 384×128 bpbreid_occ_duke_train.yaml
K (body parts)5 for holistic datasets; 8 for occluded datasets§4.2K=8 grouping: head, torso, arms, legs, feet § Supplementary materials (Study on K, the number of body parts)
AugmentationRandom crop with 10 px padding; random erasing with probability 0.5§4.2Random flip is not used in the paper text
Batch64 samples: 16 identities × 4 images§4.2Code sampler uses 4 instances per identity default_config.py
Epochs120§4.2Code default default_config.py
OptimizerAdam§4.2Code default default_config.py
Learning rateLinear warm-up from 3.5e-5 to 3.5e-4 over 10 epochs; multiplied by 0.1 at epochs 40 and 70§4.2Code peak 3.5e-4 default_config.py; warm-up start differs, see Paper vs code
Weight decaynot stated§4.2Code default 5e-4 default_config.py
Label smoothing epsilon0.1§4.2Also the default of the code's cross-entropy cross_entropy_loss.py
Triplet margin alpha0.3§4.2Code default GiLt_loss.py
lambda_pa (part attention weight)0.35§3.3Code value bpbreid_occ_duke_train.yaml
lambda_v (visibility threshold)0.4§3.2.2Code uses a different rule, see Paper vs code
lambda_t (label threshold)0.5§ Supplementary materials (Human Parsing Labels Generation with PifPaf)Code background threshold 0.5 default_config.py
Visibility during trainingnot used§3.2.2Code default off default_config.py
Test embeddingsForeground and K part embeddings; no global or concatenated§3.4Code: test_embeddings bn_foreg and parts bpbreid_occ_duke_test.yaml
HardwareOne NVIDIA Quadro RTX8000 GPU§4.2Not stated for the other settings
EvaluationSingle query, no re-ranking; CMC rank-1 and mAP§4.1Occluded-ReID has no train set, so Market-1501 is used for training §4.1

Results

Author-reported. The setup is the single-query protocol without re-ranking §4.1. Baseline is the BoT training scheme with a ResNet-50 backbone §4.4.1. The table below lists the paper's main comparison for the three BPBreID variants against BoT tab. 1. The full table, with all baselines, is at the end of this section.

DatasetMetricBoT, RN-50BPBreID, RN-50BPBreID, HRNet-W32Source
Occluded-Dukerank-151.466.775.1tab. 1
Occluded-DukemAP44.754.162.5tab. 1
Market-1501rank-194.595.195.7tab. 1
Market-1501mAP85.987.089.4tab. 1
DukeMTMC-reIDrank-186.489.692.4tab. 1
DukeMTMC-reIDmAP76.478.384.2tab. 1
Occluded-ReIDrank-158.476.982.9tab. 1
Occluded-ReIDmAP52.368.675.2tab. 1
P-DukeMTMCrank-187.091.093.0tab. 1
P-DukeMTMCmAP74.977.883.2tab. 1

The abstract's margins of 0.7 mAP and 5.6 rank-1 on Occluded-Duke §1 equal the HRNet-W32 row minus the PFD row of the comparison table (62.5 − 61.8 and 75.1 − 69.5). This is our arithmetic on the table, and the HRNet-W32 model is the one that gives those margins. The ResNet-50 model is below PFD on that dataset tab. 1.

Ablations on Occluded-Duke with ResNet-50 §4.4.1 tab. 2:

Variantrank-1mAP
BoT baseline51.444.7
BPBreID66.754.1
w/o learnable attention (fixed PifPaf weights)51.639.2
w/o visibility scores52.645.3
w/o part-averaged triplet loss (one triplet per part)64.851.7

Embeddings on Occluded-Duke with ResNet-50, with the paper's text giving K=6 for this study §4.4.3 tab. 3:

Embeddingrank-1mAP
Global f_g60.247.5
Foreground f_f64.149.7
Concatenated f_c64.450.3
Part 1, head47.124.7
Part 2, torso52.031.7
Part 3, left arm55.734.4
Part 4, right arm56.734.1
Part 5, legs22.313.3
Part 6, feet16.19.0
All parts f_1..f_665.952.2
Foreground and all parts66.152.5

Loss combinations on Occluded-Duke with ResNet-50 §4.4.2 tab. 4. The Identity loss and Triplet loss columns list the embeddings each loss is applied to: g, f and c are the holistic embeddings, p the part embeddings. "Triplet on p" is the part-averaged triplet loss. The last two columns are the PCB architecture under the same loss placement:

RowIdentity loss onTriplet loss onBPBreID rank-1BPBreID mAPPCB column rank-1PCB column mAP
GiLt (ours)g, f, cp (part-averaged)66.754.154.646.3
PCBpnone57.243.251.240.8
1pg, f, c52.943.250.240.9
2g, f, c, pg, f, c, p59.548.251.142.8
3g, f, c, pnone61.549.552.144.8
4noneg, f, c, p53.941.945.537.6
5g, f, c, pp61.849.451.043.5
6g, fp65.551.452.943.5
7g, cp56.541.9not reportednot reported
8f, cp64.052.956.246.2
9g, f, cc, p66.253.355.946.1
10g, f, cf, p63.652.2not reportednot reported
11g, f, cg, p64.052.454.845.9
12g, f, cc65.352.954.445.7

Number of parts K on Occluded-Duke with ResNet-50 § Supplementary materials (Study on K, the number of body parts):

Krank-1mAPGrouping for the parsing labels
258.349.0top region (head, trunk, arms); bottom region (legs, feet)
363.052.0head; central region (trunk, arms); bottom region (legs, feet)
464.352.9head; trunk; arms; bottom region (legs, feet)
565.053.3head; trunk; arms; legs; feet
666.152.5head; trunk; right arm and left arm as two parts; legs; feet
866.754.1head; trunk; each arm; each leg; each foot
1166.552.9head; upper and lower trunk; upper and lower arm on each side; each leg; each foot

Full comparison tab. 1. Symbols: dagger for global methods, double dagger for part-based methods. The paper's bold and rank marks are omitted. The three BPBreID rows are the paper's own models; the baselines are the paper's reported numbers:

MethodMarket R-1Market mAPDuke R-1Duke mAPOcc-Duke R-1Occ-Duke mAPOcc-ReID R-1Occ-ReID mAPP-Duke R-1P-Duke mAP
BoT (dagger)94.585.986.476.451.444.758.452.387.074.9
SGAM (double dagger)91.467.383.567.355.135.3----
PGFA (double dagger)91.276.882.665.551.437.3--44.223.1
MHSA (double dagger)94.684.087.373.159.744.8--70.741.1
VGTri (double dagger)----62.246.381.071.0--
OAMN (dagger)93.279.886.372.662.646.1----
HG (dagger)95.686.187.177.561.450.5----
BPBreID RN-50 (double dagger)95.187.089.678.366.754.176.968.691.077.8
PVPM (double dagger)------66.859.585.169.9
HOReID (double dagger)94.284.986.975.655.143.880.370.2--
ISP (double dagger)95.388.689.680.062.852.3----
PAT (double dagger)95.488.088.878.264.553.681.672.1--
PGFL (double dagger)95.387.289.679.563.054.180.770.381.164.2
HPNet (double dagger)------87.377.4--
SSGR (dagger)96.189.391.181.369.057.278.572.9--
FED (dagger)95.086.389.478.068.156.486.379.3--
LDS (dagger)95.890.391.582.564.355.7--91.982.9
PFD (double dagger)95.589.791.283.269.561.881.583.0--
BPBreID RI (double dagger)95.788.491.781.371.357.577.070.991.379.2
BPBreID HR-W32 (double dagger)95.789.492.484.275.162.582.975.293.083.2

The paper says it outperforms all part-based methods on Market-1501 except PFD, which uses a much heavier architecture: a ViT backbone and an HRNet-W48 branch for pose estimation §4.3. Global methods such as SSGR and LDS score higher than the ResNet-50 model on Market-1501 tab. 1. On Occluded-ReID the paper notes that the strongest methods use an external pose model at inference or occlusion augmentation, and BPBreID uses neither §4.3.

Paper vs code

Code: official repository github.com/VlSomers/bpbreid at commit a2dc430, read by hand. The paper also names github.com/VlSomers/person-reid in its experiments footnote §4.2; that repository was not checked.

  • Test-time visibility. The paper sets v_i = 1 when the part map's maximum exceeds 0.4 §3.2.2 eq. 4. The code, with default settings, marks a part visible when at least one pixel's argmax over the K+1 classes is that part bpbreid.py default_config.py. The engine uses these scores at test time when mask_filtering_testing is set, which the Occluded-Duke test config does part_based_engine.py bpbreid_occ_duke_test.yaml.
  • Holistic visibility. The paper sets the foreground and concatenated visibility to 1 §3.2.2. The code takes the maximum of the part visibilities for both bpbreid.py.
  • Distance weight. The paper weights each pair by v_q × v_g §3.4 eq. 9. The code uses sqrt(v_q × v_g) inside the masked mean distance.py. The two are the same for binary visibility only.
  • Pairs with no mutually visible part. The paper gives an infinite distance §3.4. The code sets them to the largest value in the matrix plus one distance.py. This is effectively infinite for ranking.
  • Test features. The paper does not state which embeddings are batch-normalised at test time. The code uses the batch-normalised foreground embedding and the un-normalised part embeddings part_based_engine.py bpbreid.py. The code also L2-normalises features before the distance by default default_config.py part_based_engine.py. The paper states neither.
  • Attention-loss normalisation. The paper's eq. 2 is a sum over k, h and w eq. 2. The code uses PyTorch cross-entropy, which averages over all pixels in the batch body_part_attention_loss.py body_part_attention_loss.py.
  • Label smoothing N. The paper calls N the batch size in eq. 2 §3.1.3. The code smooths over K+1 classes body_part_attention_loss.py.
  • Warm-up. The paper starts the warm-up at 3.5e-5 §4.2. The code scales the peak learning rate by 0.01 at the start, which gives 3.5e-6 with the default warmup_factor lr_scheduler.py. The scheduler is stepped by update_lr engine.py. The call site was not traced, so the per-epoch step is an inference.
  • Holistic part grouping, K=5. The paper's K=5 grouping is head, torso, arms, legs and feet § Supplementary materials (Study on K). That matches the code's five transform pifpaf_mask_transform.py. The holistic configs use five_v, a different five-part vertical split pifpaf_mask_transform.py bpbreid_market1501_train.yaml.
  • Occluded-Duke test config. The train config uses eight (K=8) bpbreid_occ_duke_train.yaml. The test config uses five_v bpbreid_occ_duke_test.yaml. Part count comes from the preprocess setting [code: torchreid/data/masks_transforms/init.py#L55-L64], so as written the test config would build K=5 parts. This was read, not run.
  • Part-label resolution. The paper defines Y at input resolution §3.1.2. The code resizes the masks to the feature map bilinearly and then takes the argmax part_based_engine.py.
  • Loss weights. λ_pa = 0.35 and the triplet margin 0.3 match bpbreid_occ_duke_train.yaml GiLt_loss.py.
  • Training and the paper's equations. The GiLt weights in the config match the paper's identity-on-holistic, triplet-on-parts placement bpbreid_occ_duke_train.yaml GiLt_loss.py.

Pitfalls

  • Label smoothing N. Eq. 2 is the standard smoothing formula with N as the number of classes, K+1, not the batch size §3.1.3. Use K+1, as the code does.
  • Attention loss scale. Eq. 2 sums over pixels, but the code averages. The 0.35 weight is the code's value, so match the averaging.
  • Visibility rule. Implement either the paper's 0.4 threshold or the code's argmax rule and state which one. The two are different functions of the same attention maps.
  • Test embeddings. Batch-norm on the foreground embedding and L2 normalisation change distances. The paper does not state either choice.
  • Warm-up start. The paper's start (3.5e-5) and the code's default (3.5e-6) differ by ten times.
  • Part grouping. The paper's K=5 and the holistic configs' five_v are different groupings. The paper's K=6 embedding study §4.4.3 also differs from its K=8 headline results §4.4.1.
  • Occluded-Duke test config. It builds K=5 as written, while the checkpoints are K=8. Check the preprocess setting before evaluating.
  • Headline margins. The 0.7 mAP and 5.6 rank-1 margins §1 use the HRNet-W32 model. The ResNet-50 model is below PFD on Occluded-Duke tab. 1.
  • Label generation. The paper's uncommented text does not give the PifPaf weights used for labels §3.1.2. The repository contains torchreid/scripts/get_labels.py and requirements_labels.txt; they were not reviewed for this card.
  • Numerical guards. The code adds epsilon 1e-16 to zero distances before the square root part_averaged_triplet_loss.py part_averaged_triplet_loss.py. The weighted average pooling clamps the mask sum at 1e-6 bpbreid.py, so an empty part mask gives a near-zero embedding. The paper does not mention either.
  • Compute. The best results use HRNet-W32 at 384×128 §4.2 tab. 1. Training ran for 120 epochs on one GPU §4.2. The paper gives no timing.
  • Licence of the code. The repository is under the Hippocratic License 3.0, which is not an OSI licence and carries ethical-use terms. This card quotes no code, only paths and lines. Whether the repository is acceptable as code for a commercially used page is a decision for the human reviewer.

Lineage

  • Builds on: the BoT training scheme and ResNet-50 baseline arXiv:1903.07071 §4.2 §4.4.1; the batch-hard triplet loss arXiv:1703.07737 §3.3.2; PifPaf pose estimation arXiv:1903.06593 for the parsing labels §3.1.2.
  • Compared against: PCB arXiv:1711.09349 tab. 4; and the methods in tab. 1 named PGFA, HOReID, PAT, ISP, PVPM, PFD, FED, LDS, OPReID (SSGR), HPNet, VGTri and HG.

Source and license

Adapted from “Body Part-Based Representation Learning for Occluded Person Re-Identification” by Vladimir Somers, Christophe De Vleeschouwer, Alexandre Alahi (arXiv:2211.03679), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/VlSomers/bpbreid at commit a2dc43042847 (Hippocratic License 3.0 (ethical-use license, not OSI; no code quoted)). Any quoted code is shown under that license.