--- kind: concept slug: gait-recognition title: Gait recognition summary: >- Identifying people from how they walk, usually from distant video. Most methods turn a silhouette sequence into a temporal template or set and learn an embedding. Variants add 3D body models for viewpoint, use skeletons, or use gait only during training to help image re-identification. Reported results depend strongly on the dataset. tags: [biometrics, computer-vision] papers: ["2102.09546", "2103.15537", "2204.02569"] related: [metric-learning, human-mesh-recovery, parametric-body-model, pose, distributional-alignment] status: draft updated_at: 2026-11-08 --- ## Overview Gait recognition identifies a person from the way they walk, usually from video taken at some distance and without their cooperation [arxiv:2102.09546 §1.1]. Clothing, carried objects, viewpoint, self-occlusion, backgrounds and lighting all degrade it [arxiv:2102.09546 §1]. The shared core is a sequence of frames per person, an encoder that turns it into features, and a metric or classification loss that pulls features of one person together [arxiv:2102.09546 §4.2] [arxiv:2204.02569 §3.3]. The variants differ in what the encoder receives and how it handles time. ## Variants ### Silhouette templates and set pooling The input is binary silhouettes. They are either averaged into one template, such as the gait energy image, or pooled over frames as a set. A CNN then learns features, often split into horizontal parts [arxiv:2102.09546 §4.1] [arxiv:2102.09546 §4.2] [arxiv:2102.09546 §4.3]. This is the dominant design: the survey counts silhouettes in over 81% of the reviewed methods and templates in about 70% [arxiv:2102.09546 §5.1]. GaitSet, a set-based member of this family, reports an 84.2% average on CASIA-B and 83.40% rank-1 on OU-MVLP in the survey's tables [arxiv:2102.09546 tab. 4] [arxiv:2102.09546 tab. 6]. The 3D variant below reuses its horizontal pyramid pooling [arxiv:2204.02569 §3.2]. Survey card: [Deep Gait Recognition: A Survey](/papers/2102.09546). ### 3D body-model alignment Each frame also carries a 3D body-model vector: pose, shape and camera values. A small network maps that vector to a transform, which is applied to the appearance feature map. The aim is to normalise viewpoint and body shape [arxiv:2204.02569 §3.2]. The model is trained with a triplet loss and a cross-entropy loss, weighted 1.0 and 0.1 [arxiv:2204.02569 §3.3] [arxiv:2204.02569 §5.1.3]. On the authors' Gait3D benchmark, the author-reported rank-1 is 53.20% at 88×128, against 42.60% for GaitSet and 47.70% for the same network without the 3D branch [arxiv:2204.02569 tab. 2]. The transform works in feature space; the meshes are not rotated, and the viewpoint claim is the authors' interpretation [arxiv:2204.02569 §1] [arxiv:2204.02569 §3.2]. The per-frame body vectors must be recovered from video first; the paper uses ROMP for this [arxiv:2204.02569 §4.2.3]. Card: [Gait Recognition in the Wild (SMPLGait)](/papers/2204.02569). ### Skeleton input The input is joint positions from a depth sensor or a pose estimator. They feed sequence volumes, recurrent networks or graph networks [arxiv:2102.09546 §4.1] [arxiv:2102.09546 §4.4.3] [arxiv:2102.09546 §4.4.8]. The survey says skeletons are more robust to viewpoint and clothing, but depend on accurate joint detection and add computation [arxiv:2102.09546 §4.1]. Skeletons appear in about 13% of the reviewed methods [arxiv:2102.09546 §5.1]. On Gait3D, the author-reported rank-1 at 88×128 is 0.24% for PoseGait and 6.25% for GaitGraph [arxiv:2204.02569 tab. 2]. No skeleton method has a card on this wiki yet. ### Gait as a training-only signal for image re-identification This variant targets person re-identification from one image, when clothes change between sightings. A gait stream takes one silhouette, predicts a short silhouette sequence and produces a gait feature. Two embedding losses tie that feature to the image feature, and the gait stream is dropped at test time [arxiv:2103.15537 §3.1] [arxiv:2103.15537 §3.3]. Author-reported gains with an OSNet backbone: on LTCC cloth-changing, rank-1 rises from 23.43% to 28.11% and mAP from 10.56% to 13.17% [arxiv:2103.15537 tab. 8]. On PRCC's cross-clothes setting, rank-1 rises from 28.70% to 37.55% [arxiv:2103.15537 tab. 8]. Card: [GI-ReID](/papers/2103.15537). The card records that the released repository holds only the image baseline, so the gait stream must be built from the paper. ## When to use which | Situation | Use | Why | |---|---|---| | Video, comparison with the literature, and standard benchmarks | Silhouette templates or set pooling | Most common design; CASIA-B, OU-ISIR and OU-MVLP protocols are defined [arxiv:2102.09546 §3.2] | | Many viewpoints in unconstrained scenes, and body-model recovery is affordable | 3D body-model alignment | Author-reported +5.5 rank-1 from the 3D branch on Gait3D [arxiv:2204.02569 tab. 2]; the dataset is licensed for research only [arxiv:2204.02569 §6] | | One image at test time, and clothes change between sightings | Gait as a training-only signal | Test-time cost is the ReID backbone alone [arxiv:2103.15537 §3.3]; reported gains on LTCC and PRCC [arxiv:2103.15537 tab. 8] | | Reliable joint detection, and severe viewpoint change | Skeleton input | Survey argues for robustness to viewpoint and clothing [arxiv:2102.09546 §4.1], but Gait3D results for skeletons are low [arxiv:2204.02569 tab. 2]; test on your own data first | Caveats for any choice: - Numbers from different papers use different splits, preprocessing and training budgets. The survey says its tables are not a fair ranking [arxiv:2102.09546 §5.2]. Compare within one table and one protocol. - The CASIA-B rule excludes probes whose angle equals a reference angle, but does not say which reference a probe is matched to [arxiv:2102.09546 §3.2]. - No card reports training time or memory for any variant. - Gait3D is not public. The authors' repository asks for non-commercial use, so its data cannot be used commercially [arxiv:2204.02569 §6]. ## Implementation sketch A common core for the silhouette family. The steps follow the cited papers; the paper does not name the temporal operator in the 3D variant, and it does not state the triplet margin or the cross-entropy scale. ```pseudocode # Input: binary silhouettes of one walk, (T, H, W) # Step 1: per-frame features from a small CNN [arxiv:2204.02569 §3.2] frames = cnn(silhouettes) # (T, C, h, w) # Step 2: pool over frames. The paper says set pooling and does not name # the operator. A template such as GEI is the mean over frames instead. # [arxiv:2204.02569 §3.2] [arxiv:2102.09546 §4.2] pooled = max_over_time(frames) # (C, h, w) # Step 3: horizontal pyramid pooling into 31 parts [arxiv:2204.02569 §3.2] parts = horizontal_pyramid(pooled, [16, 8, 4, 2, 1]) # (C, 31) # Step 4: one embedding per part, then the two losses [arxiv:2204.02569 §3.3] emb = per_part_fc(parts) # (31, d) loss = 1.0 * triplet(emb, labels) + 0.1 * cross_entropy(logits, labels) ``` To add the 3D branch, map each frame's body vector to a square matrix, add the identity, and multiply the feature map by it [arxiv:2204.02569 §3.2] [arxiv:2204.02569 eq. 3]. To use gait only during training, add the gait stream and its two consistency losses, and drop them at inference [arxiv:2103.15537 §3.3]. ## Contradictions - **Skeletons and viewpoint.** The survey says skeletons are more robust to viewpoint and clothing [arxiv:2102.09546 §4.1]. On Gait3D, the skeleton methods PoseGait and GaitGraph score far below silhouette methods, with 0.24% and 6.25% rank-1 against 42.60% for GaitSet [arxiv:2204.02569 tab. 2]. The datasets and protocols differ, and neither source explains the gap. Not resolved here. - **Silhouettes and clothing.** The survey says silhouettes focus on gait rather than clothing [arxiv:2102.09546 §4.1]. GI-ReID reports a silhouette set method (GaitSet) at 2.14% mAP and 7.22% rank-1 on LTCC cloth-changing, below its image baseline [arxiv:2103.15537 tab. 5]. The settings differ, a re-identification gallery against a gait benchmark, so this is not a direct test of the survey's claim. ## Papers - [Deep Gait Recognition: A Survey](/papers/2102.09546): taxonomy of body, temporal, feature and architecture choices, with datasets and reported results up to January 2021. - [GI-ReID](/papers/2103.15537): gait prediction as a training-only regulariser for cloth-changing image re-identification. - [Gait Recognition in the Wild (SMPLGait)](/papers/2204.02569): silhouette CNN aligned by per-frame 3D body parameters, with the Gait3D benchmark.