--- kind: concept slug: parametric-body-model title: Parametric body model summary: >- A human body described per frame by a short parameter set, used as a network input or as an estimator output. SMPLGait feeds an 85-value SMPL vector to a gait network; NARF uses bone frames from a joint tree to condition a neural radiance field; FrankMocap outputs SMPL-X parameters. Trade-off: NARF needs known poses, and the FrankMocap and NARF code carry non-commercial licences. Author-reported. tags: [3d-human-body, gait-recognition, biometrics, human-mesh-recovery, smpl-x, neural-rendering] papers: ["2204.02569", "2108.06428", "2104.03110"] related: [metric-learning, human-mesh-recovery, neural-radiance-field] status: draft updated_at: 2026-11-06 --- ## Overview An image-based model sees appearance, not body structure. A silhouette or a frame does not state the body's shape, the pose over time or the viewing angle [arxiv:2204.02569 §1]. A compact 3D body description per frame adds that structure. The three papers so far use it in different ways. In SMPLGait it is an input that conditions a recognition network [arxiv:2204.02569 §3.2]. In FrankMocap it is the output that the estimator must produce [arxiv:2108.06428 §3.1]. In NARF it is an input in a different form: a joint tree gives each bone a rigid frame, and a radiance field is evaluated in those frames [arxiv:2104.03110 §3.2] [§3.3]. In the two estimation papers the parameters are per frame and come from images or video, not from a 3D scan [arxiv:2204.02569 §1] [arxiv:2108.06428 §1]. NARF takes its bone poses from SMPL parameters in a dataset, and joint pose estimation is listed as future work [arxiv:2104.03110 §4] [§6]. ## Variants Three variants have cards. They differ in role: one feeds a parameter vector into a downstream network, one produces parameters, and one turns bone poses into a renderable model. ### SMPL vector driving a spatial transformer Each frame gives 85 values: 24 joints times 3 pose values, 10 shape values and 3 camera values [arxiv:2204.02569 §3.2]. A three-layer network maps the vector to a transform, which is reshaped into a square matrix and applied to the silhouette feature map as (I + G) [arxiv:2204.02569 §3.2]. The mechanism is a learned viewpoint and shape alignment in feature space, not a rotation of the mesh [arxiv:2204.02569 §3.2]. The authors report 53.20% Rank-1 on Gait3D at 88x128, against 42.60% for GaitSet [arxiv:2204.02569 tab. 2]. Card: [SMPLGait paper card](/papers/2204.02569). ### SMPL-X parameters from separate part regressors FrankMocap estimates the face, each hand and the body with separate SMPL-X-based regressors, all from one RGB image [arxiv:2108.06428 §3]. The SMPL-X vector has 21 body joints with 3 axis-angle values each, 15 joints per hand, one jaw joint, a shape vector and ten expression values [arxiv:2108.06428 §3.1]. The outputs are then merged into one whole-body mesh in one of three ways [arxiv:2108.06428 §3.5]: - **Copy-paste.** Body shape and camera come from the body module, finger poses from the hand modules, and each wrist is set from the hand's global orientation converted through the body's kinematic chain [arxiv:2108.06428 eq. 8]. - **Wrist-correction network.** A six-layer MLP adjusts the arm after copy-paste, so that the wrist lands where the hand module saw it, without gradient steps [arxiv:2108.06428 §3.5] [arxiv:2108.06428 app. A2.2]. - **Per-image optimisation.** All SMPL-X parameters are fitted to 2D keypoints, the hand meshes and a shape prior, in two stages [arxiv:2108.06428 eq. 9]. Author-reported EHF All V2V errors are 69.9 for copy-paste, 69.3 for the wrist network and 63.5 for optimisation, against 81.9 for ExPose and 146.2 for SMPLify-X [arxiv:2108.06428 tab. 4]. The table gives 0.15 s per frame for the first two merges and 2 s for optimisation; the hardware is not stated [arxiv:2108.06428 tab. 4]. Card: [FrankMocap paper card](/papers/2108.06428). ### Bone frames inside a neural radiance field NARF has no parameter vector. The kinematic model is a tree of joints joined by bones of fixed length. Forward kinematics gives each bone a rigid pose (R, t) [arxiv:2104.03110 §3.2]. A global point x is mapped into a bone's frame as R^-1 (x - t), and the view direction as R^-1 d [arxiv:2104.03110 §3.3]. The recommended variant, NARF_D, has one small occupancy network per bone working in that bone's frame. A softmax over bones gives each point a probability per bone. Each bone's encoded inputs are scaled by that probability and fed to one NeRF [arxiv:2104.03110 §3.4]. The other two variants are NARF_P, one NeRF per part, and NARF_H, one NeRF on all parts concatenated [arxiv:2104.03110 §3.4]. On the authors' synthetic renders of human bodies, NARF_D has the best mask, PSNR and SSIM in all four test settings [arxiv:2104.03110 §4]. Its novel pose, novel view PSNR is 27.24 and its mask error 123.8, against 26.50 and 125.8 for NARF_P and 23.98 and 374.6 for NARF_H [arxiv:2104.03110 §4]. The cost table gives NARF_D 205M FLOPs and 382K memory per ray, close to P-NeRF at 156M and 356K, and far below NARF_P at 2140M and 6544K [arxiv:2104.03110 tab. 1]. Card: [NARF paper card](/papers/2104.03110). ## When to use which | Situation | Use | Why | |---|---|---| | Only silhouettes or 2D poses, no mesh estimator | An appearance-only baseline | The 3D branch needs per-frame SMPL parameters, which the paper obtains with ROMP on its own data [arxiv:2204.02569 §4.2.3] | | Recognising walkers, and per-frame mesh recovery from video is available | SMPL vector driving a spatial transformer | Authors report +5.5 Rank-1 at 88x128 and +3.4 at 44x64 over the same network without the 3D branch [arxiv:2204.02569 tab. 2] [arxiv:2204.02569 §5.1.5] | | Sequences are long or unevenly sampled (gait) | Fix the training length, around 30 frames | Authors found accuracy peaked near 30 training frames and fell for longer sequences [arxiv:2204.02569 §5.2] [arxiv:2204.02569 fig. 5] | | Need to reuse the gait dataset commercially | Not this paper's dataset | Only derived data is released; raw frames are not [arxiv:2204.02569 §4.2.3]. Access terms are not stated in the paper | | Need a whole-body mesh (body, hands and face) from one RGB image | Per-part SMPL-X regressors with a merge | The modular design aims to use per-part state of the art without whole-body 3D training data [arxiv:2108.06428 §1] [arxiv:2108.06428 §5] | | Live video, latency matters | Copy-paste merge | Cheapest merge, at 0.15 s per frame in the authors' table. Its All V2V is 69.9, against 63.5 for optimisation [arxiv:2108.06428 tab. 4] | | Offline, whole-mesh accuracy matters more than speed | Per-image optimisation | Lowest All V2V (63.5) and All PA-V2V (54.7) on EHF, at 2 s per frame [arxiv:2108.06428 tab. 4] | | Need the face output to be reproducible | Check before relying on it | The face module is RingNet, and the released code has no face module [arxiv:2108.06428 §3.4] [FrankMocap card](/papers/2108.06428) | | Product or other commercial use of the FrankMocap code | Re-implement from the paper, do not use the released code | The paper is CC0, but the code repository is non-commercial [FrankMocap card](/papers/2108.06428). Several details are not stated in the paper | | Need to render a posed person from new poses and new camera views, with known bone poses and no template mesh | NARF_D | Best mask, PSNR and SSIM in all four author-reported settings on synthetic renders [arxiv:2104.03110 §4]. On real images the authors report lower quality and attribute it to rigid parts [arxiv:2104.03110 app. "Experiment on Real Human Images"] | | Need part labels together with the render | NARF_D | Its per-point argmax over bone probabilities gives a segmentation. NARF_H cannot give one, because every location depends on all parts [arxiv:2104.03110 §3.4] | | Compute per ray is limited | NARF_D or NARF_H, not NARF_P | NARF_P needs 2140M FLOPs per ray, against 205M for NARF_D and 197M for NARF_H [arxiv:2104.03110 tab. 1] | | Many bodies with varying appearance, one model | NARF_D with the autoencoder latent | Among NARF variants, NARF_D has the lowest mask error in all four settings of the autoencoder table. A 2D CNN baseline in the same table beats it on mask in the first two settings and on PSNR in all four [arxiv:2104.03110 app. "Ablation Studies for AutoEncoder"] | | Only images, no known joint poses | NARF does not apply as published | NARF needs per-image bone poses and bone lengths, and joint pose estimation is future work [arxiv:2104.03110 §4] [§6]. Estimating the pose first, for example with a per-frame mesh method, is not tested in the paper | | Commercial use of the NARF code | Re-implement from the paper, do not use the released code | The paper is CC BY 4.0, but the repository mixes MIT code with NVIDIA's non-commercial licence, and the core model imports from the non-commercial folder [NARF paper card](/papers/2104.03110). Several details, such as samples per ray and network widths, are not stated in the paper | ## Implementation sketch The sketch follows the paper's description of the training step for SMPLGait [arxiv:2204.02569 §3.2] [arxiv:2204.02569 §3.3]. Dimensions are per batch of identities; the paper uses 30 frames per sequence [arxiv:2204.02569 §5.1.3]. ```pseudocode # Input: silhouettes (B, S, H, W); smpl vectors (B, S, 85); labels (B,) def training_step(silhouettes, smpl, labels): # 1. Per-frame appearance feature maps from a six-layer CNN [§3.2] [eq. 1] maps = silhouette_cnn(silhouettes) # (B, S, C, h, w) # 2. Per-frame transform vector from the 85 SMPL values [§3.2] [eq. 2] vectors = smpl_transform_net(smpl) # (B, S, n*n) # 3. Reshape to a square matrix G and add identity [§3.2] G = reshape_square(vectors) + identity # (B, S, n, n) # 4. Zero-pad each map to square, then multiply by (I + G) [eq. 3] aligned = pad_to_square(maps) @ G[:, :, None] # 5. Pool over frames, then horizontal pyramid pooling [§3.2] seq = horizontal_pyramid(max_over_frames(aligned)) # 6. Triplet loss and cross-entropy, weighted 1.0 and 0.1 [§3.3] [eq. 4] return 1.0 * triplet(seq, labels) + 0.1 * cross_entropy(seq, labels) ``` The paper does not give the pooling operator for frames beyond the name "Set Pooling" [arxiv:2204.02569 §3.2]. The sketch uses a max, which is what the released code does; see Contradictions. The FrankMocap variant reverses the interface: the estimator outputs the vector, and the merge assembles it from parts. The wrist step needs the body's forward kinematics to convert the hand orientation to a local rotation, and the paper does not state the frame in which that orientation is given [arxiv:2108.06428 §3.5]. The NARF variant has a different core. There is no parameter vector; the per-bone frames are the input. The snippet covers only the step that maps points and directions into each bone's frame [arxiv:2104.03110 §3.3]. ```pseudocode # Map points and directions into each bone's frame [§3.2] [§3.3] # bones: list of (R, t) from forward kinematics over the joint tree def to_bone_frames(x, d, bones): # R is a rotation, so its inverse is its transpose [§3.3] local_x = [R.T @ (x - t) for R, t in bones] # each (3,) local_d = [R.T @ d for R, _ in bones] # each (3,) return local_x, local_d ``` ## Contradictions - **Input resolution.** Headline results are at 88x128 [arxiv:2204.02569 tab. 2]. The released configs for this method use 64-pixel silhouettes, which gives 44x64 input, and no 128-pixel config is in the repository [code: configs/smplgait/smplgait_gait3d.yaml#L22-L24]. - **Training length.** The paper trains for 1,200 epochs with learning-rate steps at epochs 200 and 600 [arxiv:2204.02569 §5.1.1]. The released config uses 180,000 iterations with steps at 30,000 and 90,000 [code: configs/smplgait/smplgait_gait3d.yaml#L76-L81]. The number of iterations per epoch is not stated, so the two cannot be converted. - **Feature grid.** The paper sets the grid size to max(h, w) [arxiv:2204.02569 app. A.1]. The released model hard-codes a 16 x 16 grid [code: opengait/modeling/models/smplgait.py#L27-L29]. - **Input order.** The paper lists pose, shape and camera [arxiv:2204.02569 §3.2]. The preprocessing code concatenates camera, pose and shape, which keeps the same length of 85 [code: datasets/Gait3D/pretreatment_smpl.py#L42-L45]. - **Released configs.** Two configs in the same repository differ: one uses cosine similarity with Adam, the other Euclidean distance with SGD [code: configs/smplgait/smplgait.yaml#L21]. The paper states cosine and Adam [arxiv:2204.02569 §3.3] [arxiv:2204.02569 §5.1.1]. Which one produced the reported numbers is not stated. - **Merge ranking.** The paper says the optimisation merge is best in general, and that the wrist network beats copy-paste [arxiv:2108.06428 §4.3]. Both hold for the All rows of tab. 4. They do not hold for every part: copy-paste has the lowest Body V2V (83.7, against 83.9 for the wrist network and 93.1 for optimisation), and ExPose has the lowest Face V2V (19.5) [arxiv:2108.06428 tab. 4]. - **Face output.** The paper describes its face output as comparable to other methods [arxiv:2108.06428 §4.3]. In tab. 4 its Face V2V is 30.7 for copy-paste and the wrist network, 20.4 for optimisation and 19.5 for ExPose [arxiv:2108.06428 tab. 4]. - **Vertex count.** The FrankMocap text gives 10,745 SMPL-X vertices, while its matrix dimension gives 10475 [arxiv:2108.06428 §3.1]. The paper does not say which is right. - **NARF autoencoder text versus table.** The appendix says NARF_D outperforms the other methods on all metrics and settings. In its own autoencoder table the CNN baseline is best in 8 of 12 metric cells [arxiv:2104.03110 app. "Ablation Studies for AutoEncoder"]. The paper does not say which statement the authors intended, so the table's numbers are the ones to rely on. Interpretation of the transform as viewpoint normalisation is the authors' own reading [arxiv:2204.02569 §1] [arxiv:2204.02569 §3.2]. ## Papers - [SMPLGait paper card](/papers/2204.02569): SMPLGait, a silhouette CNN with a 3D spatial transformer driven by SMPL parameters, and the Gait3D benchmark. - [FrankMocap paper card](/papers/2108.06428): FrankMocap, monocular whole-body capture from separate SMPL-X regressors and three merge options. - [NARF paper card](/papers/2104.03110): NARF, a neural radiance field over bone frames of an articulated body, with a soft part assignment (NARF_D).