Gait recognition

Overview

Gait recognition identifies a person from the way they walk, usually from video taken at some distance and without their cooperation 2102.09546 §1.1. Clothing, carried objects, viewpoint, self-occlusion, backgrounds and lighting all degrade it 2102.09546 §1. The shared core is a sequence of frames per person, an encoder that turns it into features, and a metric or classification loss that pulls features of one person together 2102.09546 §4.2 2204.02569 §3.3. The variants differ in what the encoder receives and how it handles time.

Variants

Silhouette templates and set pooling

The input is binary silhouettes. They are either averaged into one template, such as the gait energy image, or pooled over frames as a set. A CNN then learns features, often split into horizontal parts 2102.09546 §4.1 2102.09546 §4.2 2102.09546 §4.3. This is the dominant design: the survey counts silhouettes in over 81% of the reviewed methods and templates in about 70% 2102.09546 §5.1. GaitSet, a set-based member of this family, reports an 84.2% average on CASIA-B and 83.40% rank-1 on OU-MVLP in the survey's tables 2102.09546 tab. 4 2102.09546 tab. 6. The 3D variant below reuses its horizontal pyramid pooling 2204.02569 §3.2. Survey card: Deep Gait Recognition: A Survey.

3D body-model alignment

Each frame also carries a 3D body-model vector: pose, shape and camera values. A small network maps that vector to a transform, which is applied to the appearance feature map. The aim is to normalise viewpoint and body shape 2204.02569 §3.2. The model is trained with a triplet loss and a cross-entropy loss, weighted 1.0 and 0.1 2204.02569 §3.3 2204.02569 §5.1.3. On the authors' Gait3D benchmark, the author-reported rank-1 is 53.20% at 88×128, against 42.60% for GaitSet and 47.70% for the same network without the 3D branch 2204.02569 tab. 2. The transform works in feature space; the meshes are not rotated, and the viewpoint claim is the authors' interpretation 2204.02569 §1 2204.02569 §3.2. The per-frame body vectors must be recovered from video first; the paper uses ROMP for this 2204.02569 §4.2.3. Card: Gait Recognition in the Wild (SMPLGait).

Skeleton input

The input is joint positions from a depth sensor or a pose estimator. They feed sequence volumes, recurrent networks or graph networks 2102.09546 §4.1 2102.09546 §4.4.3 2102.09546 §4.4.8. The survey says skeletons are more robust to viewpoint and clothing, but depend on accurate joint detection and add computation 2102.09546 §4.1. Skeletons appear in about 13% of the reviewed methods 2102.09546 §5.1. On Gait3D, the author-reported rank-1 at 88×128 is 0.24% for PoseGait and 6.25% for GaitGraph 2204.02569 tab. 2. No skeleton method has a card on this wiki yet.

Gait as a training-only signal for image re-identification

This variant targets person re-identification from one image, when clothes change between sightings. A gait stream takes one silhouette, predicts a short silhouette sequence and produces a gait feature. Two embedding losses tie that feature to the image feature, and the gait stream is dropped at test time 2103.15537 §3.1 2103.15537 §3.3. Author-reported gains with an OSNet backbone: on LTCC cloth-changing, rank-1 rises from 23.43% to 28.11% and mAP from 10.56% to 13.17% 2103.15537 tab. 8. On PRCC's cross-clothes setting, rank-1 rises from 28.70% to 37.55% 2103.15537 tab. 8. Card: GI-ReID. The card records that the released repository holds only the image baseline, so the gait stream must be built from the paper.

When to use which

Caveats for any choice:

• Numbers from different papers use different splits, preprocessing and training budgets. The survey says its tables are not a fair ranking 2102.09546 §5.2. Compare within one table and one protocol.
• The CASIA-B rule excludes probes whose angle equals a reference angle, but does not say which reference a probe is matched to 2102.09546 §3.2.
• No card reports training time or memory for any variant.
• Gait3D is not public. The authors' repository asks for non-commercial use, so its data cannot be used commercially 2204.02569 §6.

Implementation sketch

A common core for the silhouette family. The steps follow the cited papers; the paper does not name the temporal operator in the 3D variant, and it does not state the triplet margin or the cross-entropy scale.

# Input: binary silhouettes of one walk, (T, H, W)
# Step 1: per-frame features from a small CNN  [arxiv:2204.02569 §3.2]
frames = cnn(silhouettes)                       # (T, C, h, w)
# Step 2: pool over frames. The paper says set pooling and does not name
# the operator. A template such as GEI is the mean over frames instead.
# [arxiv:2204.02569 §3.2] [arxiv:2102.09546 §4.2]
pooled = max_over_time(frames)                  # (C, h, w)
# Step 3: horizontal pyramid pooling into 31 parts  [arxiv:2204.02569 §3.2]
parts = horizontal_pyramid(pooled, [16, 8, 4, 2, 1])   # (C, 31)
# Step 4: one embedding per part, then the two losses  [arxiv:2204.02569 §3.3]
emb = per_part_fc(parts)                        # (31, d)
loss = 1.0 * triplet(emb, labels) + 0.1 * cross_entropy(logits, labels)

To add the 3D branch, map each frame's body vector to a square matrix, add the identity, and multiply the feature map by it 2204.02569 §3.2 2204.02569 eq. 3. To use gait only during training, add the gait stream and its two consistency losses, and drop them at inference 2103.15537 §3.3.

Contradictions

• Skeletons and viewpoint. The survey says skeletons are more robust to viewpoint and clothing 2102.09546 §4.1. On Gait3D, the skeleton methods PoseGait and GaitGraph score far below silhouette methods, with 0.24% and 6.25% rank-1 against 42.60% for GaitSet 2204.02569 tab. 2. The datasets and protocols differ, and neither source explains the gap. Not resolved here.
• Silhouettes and clothing. The survey says silhouettes focus on gait rather than clothing 2102.09546 §4.1. GI-ReID reports a silhouette set method (GaitSet) at 2.14% mAP and 7.22% rank-1 on LTCC cloth-changing, below its image baseline 2103.15537 tab. 5. The settings differ, a re-identification gallery against a gait benchmark, so this is not a direct test of the survey's claim.

Papers

• Deep Gait Recognition: A Survey: taxonomy of body, temporal, feature and architecture choices, with datasets and reported results up to January 2021.
• GI-ReID: gait prediction as a training-only regulariser for cloth-changing image re-identification.
• Gait Recognition in the Wild (SMPLGait): silhouette CNN aligned by per-frame 3D body parameters, with the Gait3D benchmark.