Feed-forward reconstruction

Overview

A feed-forward reconstructor is trained once on many objects, scenes or faces. At test time it reads the input images and outputs a 3D representation in a single pass, so nothing is fitted to the new input. Most variants train by comparing renders of that representation with real views 2312.13150 §3.2 2312.13150 §3.3 2012.02190 §3 2012.02190 eq. 2 2306.16928 §3.3 2403.14627 §3.3.

The representation can be a set of Gaussian primitives, drawn by a fast splatting renderer 2312.13150 §3.2 2312.13150 §3.3. MVSplat also outputs Gaussians, but for two views it places their centres from a predicted depth 2403.14627 §3.1 2403.14627 §3.2. For driving logs it is a 4D set: Gaussians for two frames, with vehicle Gaussians moving at a constant velocity 2603.07552 §3.1 2603.07552 §3.3. Forge4D does the same for people in uncalibrated video: per-frame Gaussians, plus a motion field that moves them to times between frames 2509.24209 §3.2 2509.24209 §3.4. iLRM also outputs Gaussians, one per pixel of each viewpoint slot, with each position set by the pixel's ray and a predicted depth 2507.23277 §3.2 2507.23277 app. A. FLARE places Gaussian centres on global point maps, which it predicts from camera-conditioned tokens 2502.12138 §3.3. It can also be a radiance field, queried along each camera ray and composited with volume rendering 2012.02190 §4 2012.02190 eq. 1. Another option is a signed distance field, from which a mesh is extracted with marching cubes 2306.16928 §3.3. Another is a voxel occupancy grid, which is compared with the input through silhouettes rather than through colour 1612.00814 §3.1 1612.00814 §3.2. For faces, the output is a UV position map: each cell stores the x, y and z of one face point, so the network's output is itself the reconstruction and nothing is rendered 1803.07835 §3.1 1803.07835 §3.2. GeFF takes a third route: a set of sparse latent points, one per input frame, is decoded at any 3D position into a signed distance and a feature that is aligned with text. The features are volume-rendered along rays 2403.07563 §3.1 2403.07563 §3.2.

The problems this family addresses:

• Per-scene NeRF needs many posed images and a long optimisation for each scene. With one or two images and no image features, it generalises poorly 2012.02190 §1.
• Implicit, NeRF-style reconstructors are slow to render, up to 2 s for one 128 × 128 image 2312.13150 §2.
• Explicit voxel methods are fast but scale poorly with resolution, and they assume absolute viewpoints 2312.13150 §2.
• Triplane reconstructors need hundreds of GPUs for several days to reach large datasets 2312.13150 §2.
• Gaussian splatting is fast, but it normally needs many images per scene and had not been used in a learned, single-image setting 2312.13150 §2. See Gaussian splatting.
• Feed-forward Gaussian methods that predict depth as a distribution over candidates give noisy geometry. The closest prior method, pixelSplat, needs an extra fine-tuning stage with depth regularisation to improve it 2403.14627 §1 2403.14627 §4.2.
• Earlier 3D face methods often fit a 3D Morphable Model to each image, or warp a template, which limits the shape to the model's space and is slow 1803.07835 §1 1803.07835 §2.
• Per-shape optimisation with a 2D diffusion prior often takes tens of minutes per shape, and it can produce 3D-inconsistent shapes 2306.16928 §1.
• Learning 3D shape normally needs ground-truth volumes for supervision, which are hard to collect. An agent usually observes only 2D silhouettes through its camera 1612.00814 §3.1.
• Per-scene 4D Gaussian fitting gives good quality for driving scenes, but it needs costly optimisation for every scene 2603.07552 §1. Earlier feed-forward methods lose photometric detail, and a static 3D foundation model does not represent moving traffic 2603.07552 §1 2603.07552 §3.1.
• Dynamic human reconstruction from a few cameras usually needs calibrated cameras and a long per-scene optimisation over dense video 2509.24209 §1. Static feed-forward Gaussian models handle one timestamp at a time, so they cannot give views at times between frames 2509.24209 §2.
• Stacking all frames into one transformer runs out of memory, and the Gaussian scale drifts between timestamps 2509.24209 §3.3.
• Classic structure-from-motion relies on handcrafted feature matching, which is not differentiable, and it struggles with sparse, uncalibrated views. Joint pose and geometry optimisation needs a good start and generalises poorly. Two-view point-map methods need a slow global registration step after prediction 2502.12138 §1.
• Pixel-aligned Gaussian models make one Gaussian per pixel, so their count grows with resolution and with the number of views. Full attention over all image tokens of all views is quadratic in both 2507.23277 §1 2507.23277 §3.1. The paper's example is about 200 million Gaussians for 200 views at 1K resolution 2507.23277 §1.
• Language-queryable feature fields built per scene need costly optimisation. Offline pipelines that batch a whole scan took about 2 h (LERF) and about 200 s (ConceptGraphs) in one robot comparison 2403.07563 §2 2403.07563 tab. 1.

Feed-forward does not mean fast rendering. The radiance-field variant still renders with a per-ray network query, as NeRF does 2012.02190 §6.

Splatter Image and pixelNeRF take the input cameras as known: relative poses for Splatter Image, and intrinsics plus relative poses for pixelNeRF 2312.13150 tab. 1 2012.02190 §1. One-2-3-45 does not get the input elevation; it estimates it from the generated views 2306.16928 §3.4. Perspective Transformer Nets uses the known camera matrix during training, to project its volumes onto each view. Its single-image prediction needs only the encoder and decoder, so no camera is needed at test time 1612.00814 §3.1 1612.00814 §3.2. ReconDrive takes calibrated cameras, ego poses for both frames and vehicle boxes with velocities as input 2603.07552 §3.1 2603.07552 §3.3. Its released code takes the poses and boxes from dataset annotations, not from the images alone 2603.07552 Paper vs code. GeFF takes posed RGB-D frames, and its poses come from an onboard visual-inertial odometry estimate 2403.07563 §3.3. Forge4D takes no camera input: it predicts its poses from the uncalibrated videos 2509.24209 §3.2. FLARE also takes uncalibrated images and predicts the camera poses itself 2502.12138 abstract 2502.12138 §3.1. Its novel-view model on RealEstate10K does take intrinsics as a condition 2502.12138 §4.3. iLRM takes the pose and intrinsics of every input view, and builds its viewpoint tokens from the Plücker rays they define 2507.23277 §3.2 2507.23277 app. A.

The eleven papers differ in the output representation, in how several views are combined, in where the input geometry comes from, in how time is handled, and in what supervises training.

Variants

Single view, one Gaussian per pixel

The network is an image-to-image U-Net whose output grid has the input's height and width. Each cell stores one Gaussian: opacity, depth, offset, scale, rotation and colour. The Gaussian centre is the ray point at the predicted depth plus a predicted offset, so a Gaussian can sit off its ray and cover parts of the object the camera does not see 2312.13150 §3.2 2312.13150 §3.6. The colour is either constant (Lambertian) or degree-1 spherical harmonics, which are rotated with the camera 2312.13150 §3.5. Card: Splatter Image: Ultra-Fast Single-View 3D Reconstruction.

Reported results, author-reported: 24.00 PSNR on ShapeNet-SRN cars and 24.43 on chairs, with relative pose for this method 2312.13150 tab. 1. It reaches 21.06 PSNR on Google Scanned Objects with an Objaverse-trained model, against 18.06 for OpenLRM 2312.13150 tab. 4.

Several views, union of warped primitives

The same network runs on each input view. Each view's Gaussians are moved into a common frame with the relative camera pose, and the paper takes their union as the output. Cross-view attention at the lowest U-Net resolution lets the views exchange information 2312.13150 §3.4 2312.13150 §3.6. The two-view model reports 26.01 PSNR on ShapeNet-SRN cars with relative pose 2312.13150 tab. 5. The two-view training settings are not stated 2312.13150 §4.1.2.

Two posed views, plane-sweep cost volume

The input is two images with known intrinsics and extrinsics, plus a near and far depth range for each view 2403.14627 §3 2403.14627 app. C. Features are extracted by a shallow CNN and a multi-view transformer at a quarter of the input resolution 2403.14627 §3.1 2403.14627 app. C. The other views' features are warped onto 128 depth candidates, spaced uniformly in inverse depth, and compared with the reference view by a scaled dot product. The result is a cost volume that a U-Net refines, and a softmax over the depth candidates gives a per-pixel depth 2403.14627 §3.1 2403.14627 eq. 6. A second, light U-Net adds a residual to that depth. Each pixel's depth is then unprojected to a Gaussian centre 2403.14627 §3.2. The loss is an L2 term plus 0.05 times LPIPS, on rendered target views, with no depth supervision 2403.14627 §3.3. Card: MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images.

Reported results, author-reported, on RealEstate10K at 256 × 256 with two context views 2403.14627 tab. 1. PSNR is 26.39 and LPIPS is 0.128, at 0.044 s per scene with 12.0M parameters. pixelSplat, the closest prior Gaussian method, reports 25.89 PSNR at 0.104 s with 125.4M parameters 2403.14627 tab. 1. Removing the cost volume drops the ablation to 22.83 PSNR 2403.14627 tab. 3. Trained on RealEstate10K only, it reaches 28.15 PSNR on ACID and 13.94 PSNR on DTU, against 27.64 and 12.89 for pixelSplat 2403.14627 tab. 2. The paper reports photometric training alone leaves reflective surfaces such as glass unreliable 2403.14627 §5 2403.14627 app. A.

Radiance field conditioned on pixel-aligned features

The network has a convolutional encoder that turns each input image into a feature grid, and a NeRF-style MLP that returns colour and density for a 3D point 2012.02190 §4. For each point on a target ray, the point is projected into each input view, and the feature at that pixel is looked up. The field therefore depends on local image detail. Removing the local features and using a global code lowers the ablation's 1-view score from 23.43 to 20.39 PSNR 2012.02190 §4.1 2012.02190 tab. 3. Views are averaged inside the MLP, so the number of input views can change at test time 2012.02190 §4.2 2012.02190 eq. 6. Training uses multi-view 2D images only, with no 3D supervision 2012.02190 §1. Card: pixelNeRF: Neural Radiance Fields from One or Few Images.

Reported results, author-reported, with the SRN protocol at 128 × 128 and no canonical pose needed at test time 2012.02190 tab. 2:

On 13 ShapeNet categories at 64 × 64, with one input view, the mean is 26.80 PSNR, against 23.28 for SRN and 22.70 for DVR 2012.02190 tab. 4. On categories held out from training, with one view, it reaches 22.71 PSNR, against 18.71 for SRN 2012.02190 tab. 5. On DTU real scenes, one model trained on 88 scenes renders held-out scenes from three views. Its numbers appear only in a plot, with no table 2012.02190 §5.3 2012.02190 fig. 10.

Splatter Image's comparison cites the same pixelNeRF numbers for cars and chairs, and for two-view cars 2312.13150 tab. 1 2312.13150 tab. 5. The two cards therefore agree on this baseline.

Generated views, then a cost volume

The input is one photo. A frozen view-synthesis model, Zero123, first renders the object from 8 anchor viewpoints, with 4 nearby views around each, giving 32 source views 2306.16928 §3.1 2306.16928 §4.1. The input's elevation is estimated by matching features across the nearby views 2306.16928 §3.4. A network then builds a 3D cost volume from the source views, predicts a signed distance field and colour, and marching cubes gives the mesh 2306.16928 §3.3. The network is trained on rendered Objaverse shapes, with predicted views in the input and ground-truth views as targets, so it learns to cope with the generated views being imperfect 2306.16928 §3.3 2306.16928 §4.3. Card: One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization.

Reported results, author-reported, on 20 Objaverse and 20 Google Scanned Objects shapes 2306.16928 §4.2. The average F-score is 83.5, the best in the paper's table. The average CLIP similarity is 78.1, second to Shap-E's 80.9. The pipeline takes about 45 s per shape on an A100, while Shap-E takes 27 s in the same table 2306.16928 tab. 1. The reconstruction takes about 5 s, and the paper attributes the rest of the 45 s to the view model, which it times at about 1 s per image 2306.16928 §4.2. The median elevation error is 5.4 degrees and the mean is 9.7 degrees, on 1,700 random renders 2306.16928 §4.3 2306.16928 fig. 7.

Voxel grid trained through a perspective projection

The network has a 2D convolutional encoder and a 3D decoder. From one 64 × 64 image it predicts a 32 × 32 × 32 occupancy grid 1612.00814 §3.1 1612.00814 §4. A fixed perspective projector samples the grid onto each camera view and takes the maximum along depth, which gives a silhouette 1612.00814 §3.2 1612.00814 eq. 4. The loss is the squared error between predicted and observed silhouettes, averaged over views 1612.00814 eq. 1. A volume loss against ground-truth voxels can be added, but the projection loss alone is the paper's main setting 1612.00814 §3.1. Training has two stages: the encoder is first pre-trained with a rotation curriculum, so that its code carries shape identity rather than viewpoint 1612.00814 §3.3. Card: Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision.

Reported results, author-reported, on ShapeNet chairs with voxel IoU at a 0.5 threshold 1612.00814 §4.1. Test IoU is 0.5027 with projection loss only, 0.5067 with the volume loss added, and 0.4983 for a volume-only CNN. Retrieval of the nearest training volume reaches 0.3557 1612.00814 tab. 1. The gap between the two projection-based models is 0.0040 in single runs with no variance reported 1612.00814 tab. 1. On categories never seen in training, the single-class projection model scores 0.1801 IoU on beds and 0.3937 on cabinets 1612.00814 tab. 3. The multi-class projection model reaches 0.6484 on cabinets 1612.00814 tab. 3.

Driving scenes, two frames from six cameras

The input is a rig of six calibrated cameras at two moments 0.5 s apart 2603.07552 §3.1 2603.07552 §4.1. The backbone is VGGT, a 3D foundation model whose weights stay frozen. LoRA adapters of rank 8 and scale 32 are trained on top of it 2603.07552 §3.2. Two DPT heads read its features. One predicts a depth per pixel, which is unprojected with the camera calibration into the ego-vehicle frame to give the Gaussian centre. The other predicts the remaining Gaussian attributes, and it also sees the raw image 2603.07552 §3.2. The method section gives no separate offset term for the centre, unlike the single-view variant above 2312.13150 §3.2.

Vehicles are found with SAM 2 masks. Each vehicle gets one velocity, from the displacement of its box centre between the two frames, and every pixel in its mask takes that velocity 2603.07552 §3.3 2603.07552 eq. 3. The second frame's Gaussians are moved into the first frame's ego frame, and the moving ones are shifted back by their velocity. The two sets are then concatenated. At render time a Gaussian at time t sits at its centre plus its velocity times (t minus the segment start) 2603.07552 §3.3 2603.07552 eq. 4. Card: ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction.

Reported results, author-reported, on nuScenes with 14 validation scenes 2603.07552 §4.1 2603.07552 app. A. Scene reconstruction PSNR is 32.66, against 29.58 for PVG, the best per-scene baseline, and 22.83 for DrivingForward, the feed-forward baseline 2603.07552 tab. 1. For novel views 1 to 3 m away, PSNR is 23.99, against 23.73 for DeformableGS 2603.07552 tab. 1. One scene takes about 15 s, against 23 to 46 minutes for the per-scene baselines and 5 s for DrivingForward 2603.07552 tab. 1. The baselines were re-run by the authors on 12 Hz data, not taken from their papers 2603.07552 §4.1.

Posed RGB-D frames, latent points decoded to SDF and features

The input is a short stream of posed RGB-D frames from a mobile robot. A generalizable NeRF-style encoder turns them into one latent vector per frame. The frozen teacher's 2D features for the same frames are appended to those latents 2403.07563 §3.1 2403.07563 §3.2 2403.07563 eq. 3. Two decoders read the latents at any 3D point: a signed distance network, and an isotropic feature decoder that takes no viewing direction 2403.07563 §3.2. Opacity along a ray comes from the signed distance through a sigmoid with a learnable sharpness 2403.07563 eq. 4. Features and depth are volume-rendered along novel-view rays 2403.07563 eq. 2 2403.07563 eq. 5. Training compares rendered features with the teacher's features, and rendered depth with an L2 loss. The SDF follows the iSDF loss with an Eikonal regulariser 2403.07563 §3.2 2403.07563 eq. 6. For the robot, the decoded field becomes a point cloud, and each text query is scored against it by cosine similarity 2403.07563 §3.3. Card: Learning Generalizable Feature Fields for Mobile Manipulation.

Reported results, author-reported, on a quadruped with an arm, in four rooms 2403.07563 §4.1. Overall success is 62.4%, against 43.2% for LERF run offline and 41.9% for ConceptGraphs run offline. The GeFF latency is 0.39 s on an onboard AGX Orin 2403.07563 tab. 1. Without the auxiliary teacher input the object-level score falls from 41 to 25 out of 66 trials, and LERF scores 22 2403.07563 tab. 2. Depth L2 error on 512 latent points is 0.012, against 0.040 for bilinear and 0.061 for nearest-neighbour upsampling of a downsampled grid 2403.07563 tab. 4. After objects are added, lab success falls from 7 to 4 out of 9 trials, and the meeting room from 7 to 6 out of 9 2403.07563 tab. 3. Trial counts per cell are small, and the main table does not state them 2403.07563 §4.1.

One face image, UV position map

The input is a square face crop of 256 × 256 pixels. An encoder with one convolution and 10 residual blocks reduces it to an 8 × 8 × 512 map, and 17 transposed convolutions decode a 256 × 256 × 3 position map 1803.07835 §3.2. Each cell has a fixed UV location and stores the x, y and z of one face point, including points hidden from the camera. So one output gives the 68 landmarks, dense correspondence and 3D shape 1803.07835 §3.1. The loss weights cells by region, at 16:4:3:0 for landmarks, eyes/nose/mouth, the rest of the face, and the neck 1803.07835 §3.2 1803.07835 eq. 1 1803.07835 tab. 3. Training uses 300W-LP, with more than 60K images whose ground-truth maps come from 3D Morphable Model fits 1803.07835 §3.1 1803.07835 §3.3. Card: Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network.

Reported results, author-reported. Mean NME on AFLW2000-3D (68 points) is 3.62, against 4.49 for 3DSTN and 4.94 for 3DDFA with SDM 1803.07835 tab. 1. Inference takes 9.8 ms per image on a GTX 1080, against 19.0 ms for 3DSTN and 75.7 ms for 3DDFA 1803.07835 §4.4 1803.07835 tab. 2. The dense-alignment and reconstruction gains are read from plots, not tables 1803.07835 §4.2 1803.07835 §4.3.

Uncalibrated video, state token and learned motion between frames

The input is four uncalibrated video streams of a dynamic human scene 2509.24209 §3.1 2509.24209 §4.1. The backbone is VGGT, initialised from its pre-trained weights 2509.24209 §4.1. Each view gives pixel-aligned maps for Gaussian position, opacity, colour, rotation and scale 2509.24209 §3.2. The VGGT output is in normalised units, so the paper uses a metric gauge to rescale the poses. The gauge is the mean of per-camera ratios, each the predicted translation length over the ground-truth one 2509.24209 §3.2. The paper also predicts the gauge with a head, for use at inference 2509.24209 app. C.

A learnable state token passes information from frame to frame through cross-attention, so the model does not run global attention over all frames 2509.24209 §3.3. A motion head reads the features of two neighbouring frames and predicts a backward and a forward 3D motion field for each view 2509.24209 §3.4. Nothing supervises the motion directly. A retargeting loss renders one frame moved to the previous frame and compares it with that frame's render 2509.24209 eq. 6. A flow loss matches the predicted 2D scene flow to pseudo-labels from SEA-RAFT, and it down-weights occluded pixels 2509.24209 eq. 7 2509.24209 §3.4.

For a time between two frames, each frame's Gaussians move with a constant-velocity displacement scaled by the time distance [arxiv:2509.24209 eq. 5]. Gaussians that are occluded in one warped frame are kept apart, and the rest are merged by a two-layer MLP 2509.24209 eq. 8. The authors justify the constant-velocity step only for short gaps 2509.24209 §3.4. Card: Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos.

Reported results, author-reported, on DNA-Rendering with four input views. Novel-view PSNR at input times is 29.82, against 26.12 for AnySplat, which is pose-free, and 24.30 for GPS-Gaussian, which takes poses [arxiv:2509.24209 tab. 1]. Novel view and time reach 29.04 PSNR, against 18.03 for L4GM with eight input frames, which takes poses 2509.24209 tab. 2. The pose-free numbers come from an evaluation step that optimises the novel-view poses while the Gaussians stay fixed. That step is not used at deployment [arxiv:2509.24209 app. C]. Without gauge alignment, the static model falls to 13.29 PSNR, and without the Gaussian fusion, the novel-time model falls to 27.35 2509.24209 tab. 7.

Posed views, iterative viewpoint tokens

The input is N posed images, each with its pose and intrinsics. Each view starts from a viewpoint token made only of its per-pixel Plücker rays, with no learned query 2507.23277 §3.2. Twelve update layers follow. In each layer, a viewpoint token cross-attends only to the image tokens of its own view, and then a self-attention block mixes the viewpoint tokens of all views 2507.23277 §3.2 2507.23277 eq. 3 2507.23277 eq. 4. Each layer has its own weights. A linear layer then decodes each token into Gaussian parameters 2507.23277 app. A. The paper compares each layer with a gradient step, but it does not derive that from an objective 2507.23277 §3.2. Training renders held-out views and compares them with the targets using MSE plus a perceptual term, weighted by 0.5 [arxiv:2507.23277 eq. 5]. Card: iLRM: An Iterative Large 3D Reconstruction Model.

Reported results, author-reported, at 256 × 256 on RealEstate10K with two views, in the paper's (2, F, F) setting. PSNR is 28.65, against 27.47 for DepthSplat, and rendering takes 0.025 s, against 0.065 s 2507.23277 tab. 1. On DL3DV with 12 views at 512 × 960, PSNR is 24.35, against 21.38 for DepthSplat 2507.23277 tab. 4. For 32 views at 540 × 960, it reaches 24.30 PSNR in 0.53 s. That is slightly below the 24.43 of per-scene 3D-GS, which took 8 min to optimise 2507.23277 tab. 5. In the 4-view ablation setting, removing the iterative refinement lowers PSNR from 29.24 to 28.58 2507.23277 tab. 7.

Uncalibrated sparse views, camera poses predicted first

The input is 2 to 8 uncalibrated images, and the model is trained on 8 views per sample 2502.12138 abstract 2502.12138 §4. A transformer predicts a coarse pose for each image from learnable camera latents 2502.12138 §3.1. A second transformer predicts camera-centric point maps and refines the poses in the same network 2502.12138 §3.2. A third transformer maps the local point tokens into one global frame, using the refined poses, in place of a geometric reprojection step 2502.12138 §3.2 2502.12138 eq. 4. The Gaussian centres are the global point maps, and a shallow CNN predicts the other Gaussian attributes from the appearance features and VGG features of the inputs 2502.12138 §3.3. The loss has pose, geometry and splatting terms, and the splatting term includes a depth target from a monocular estimator 2502.12138 §3.4. Card: FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views.

Reported results, author-reported. On RealEstate10K with 5 input images and no fine-tuning, AUC@30° is 79.9, against 61.1 for MASt3R and 51.6 for PoseDiffusion 2502.12138 tab. 1. On DL3DV, FLARE reaches 23.33 PSNR with 8 views and 23.04 with 2 views. MVSplat reaches 22.08 with two views, and it takes poses 2502.12138 tab. 3. The paper gives inference under 0.5 s, but not the hardware for that figure 2502.12138 §1 2502.12138 Pitfalls.

When to use which

Implementation sketch

Common core of one feed-forward pass and its training step. The paper's details are in the cited sections. Variants differ in the output representation, how many views feed it, and the perceptual term. The face variant has no render step, and the cost-volume variant replaces the fusion step, as the paragraphs below explain.

# Predict one primitive per input pixel from one view  [arxiv:2312.13150 §3.2]
def predict_primitives(image, camera):
    # Image-to-image network; one output cell per input pixel  [arxiv:2312.13150 §3.6]
    grid = unet(image, camera)
    # One Gaussian per cell: opacity, centre, scale, rotation, colour  [arxiv:2312.13150 §3.2]
    return split_into_gaussians(grid, camera)

# Merge the Gaussians of several views into one set  [arxiv:2312.13150 §3.4]
def fuse_views(per_view, relative_poses):
    # Move each view's Gaussians into the reference frame  [arxiv:2312.13150 §3.4]
    moved = [transform(g, p) for g, p in zip(per_view, relative_poses)]
    # Output is the union of the moved sets  [arxiv:2312.13150 §3.4]
    return concatenate(moved)

# One training step on one object  [arxiv:2312.13150 §3.3] [arxiv:2312.13150 eq. 4]
def training_step(input_view, target_views, alpha):
    gaussians = predict_primitives(input_view.image, input_view.camera)
    # Render the predicted set at every target camera  [arxiv:2312.13150 §3.1]
    renders = [render(gaussians, v.camera) for v in target_views]
    # Pixel error against the real target images  [arxiv:2312.13150 eq. 4]
    pixel_loss = mean_squared_error(renders, target_views)
    # Perceptual term, blended in by alpha; zero in the first stage  [arxiv:2312.13150 app. C]
    perceptual_loss = lpips(renders, target_views)
    return (1 - alpha) * pixel_loss + alpha * perceptual_loss

For the radiance-field variant, replace predict_primitives and render with a per-ray query of the feature-conditioned field, followed by volume compositing 2012.02190 §4 2012.02190 eq. 1. The pixelNeRF card has pseudocode for that query and for the view pooling. Its released code adds the feature to three of the five MLP blocks, not to all of them, so follow the card if you want to reproduce its numbers 2012.02190 Paper vs code.

For the generated-view variant, put view synthesis and elevation estimation before the network. The views are rendered first, the input elevation is estimated from them, and only then are the source poses known 2306.16928 §3.1 2306.16928 §3.4. The network takes the 32 source views and their poses and returns a signed distance field, so fuse_views is replaced by a cost-volume reconstruction 2306.16928 §3.3.

For the voxel variant, the render step becomes a perspective projection with a max over depth, and the loss compares silhouettes instead of pixels 1612.00814 §3.2 1612.00814 eq. 1 1612.00814 eq. 4. The camera enters only through that projection during training.

For the driving variant, the centres come from unprojecting the predicted depth with each camera's calibration. The second frame is moved into the first frame's ego frame, and its moving Gaussians are shifted back by their velocity. The fuse_views step therefore becomes a concatenation of the two frames 2603.07552 §3.3 2603.07552 eq. 4. The released code takes each vehicle's velocity from its annotated box, and it prompts the vehicle masks with those boxes, so it does not regress velocity or detect vehicles 2603.07552 Paper vs code. The card has a forward-pass sketch for the segment 2603.07552 §3.3.

For the cost-volume variant, predict_primitives becomes a plane sweep over the depth candidates, followed by a softmax that gives one depth per pixel. The Gaussian centres are that depth unprojected along each ray. Views are combined inside the cost volume, by averaging the correlations over the other views 2403.14627 §3.1 2403.14627 §3.2 2403.14627 eq. 4. The card's pseudocode gives the full order of steps 2403.14627 §3.

For the posed RGB-D variant, the encoder's latents have the teacher's features appended, and predict_primitives becomes a decoder that returns a signed distance and a feature at any point. Opacity comes from the distance, and the render step integrates features and depth instead of colour 2403.07563 §3.2 2403.07563 eq. 3 2403.07563 eq. 4. The training loss is a weighted sum of five terms, one of them a colour term that the text does not define 2403.07563 eq. 6 (see Contradictions).

For the face variant, there is no render step. The network's output is the position map, and the loss compares it with the ground-truth map cell by cell, weighted by region 1803.07835 §3.2 1803.07835 eq. 1. The card has the test-time crop 1803.07835 §3.2.

For the Forge4D variant, predict_primitives runs once per frame, and a state token carries information to the next frame, so no step attends over all frames 2509.24209 §3.3. The novel-time step is an interpolate between two neighbouring frames, not a fuse_views over cameras. Each frame moves by a constant-velocity displacement, and occluded Gaussians are kept apart 2509.24209 §3.4 2509.24209 eq. 5 2509.24209 eq. 8. The card has pseudocode for that step 2509.24209 §3.4.

For the iLRM variant, predict_primitives becomes a loop of twelve update layers over viewpoint tokens. Each layer cross-attends to the image tokens of its own view, then self-attends across all views 2507.23277 §3.2 2507.23277 eq. 3 2507.23277 eq. 4. Cross-view mixing happens inside those layers, so there is no fuse_views step. The card has the full forward pass and the decoding step 2507.23277 §3.2 2507.23277 app. A.

For the uncalibrated variant, pose estimation runs before the geometry: coarse poses come from camera latents, and the same network later refines them 2502.12138 §3.1 2502.12138 §3.2. fuse_views is replaced by a learned projection of the local point tokens into one global frame, which takes the refined poses as input 2502.12138 §3.2 2502.12138 eq. 4. The Gaussian centres are then the global point maps 2502.12138 §3.3. The card has the full forward pass 2502.12138 §3.1-§3.3.

Contradictions

Disagreements found in the cards, each side cited. Between Splatter Image and pixelNeRF, none: they report the same pixelNeRF baseline numbers. One-2-3-45 compares against other baselines, so no cross-check of its numbers exists here 2306.16928 tab. 1. ReconDrive's baselines are per-scene methods and DrivingForward, and none of them has a card on this page, so its numbers have no cross-check here 2603.07552 tab. 1. Perspective Transformer Nets reports voxel IoU and compares against no other paper on this page, so it cannot be checked against the PSNR numbers above.

• Between the paper and its released code, for Perspective Transformer Nets. The paper says the silhouettes come from image observations, which are trivial to segment on a white background 1612.00814 §3 1612.00814 §3.1. The released training script projects the ground-truth voxel grid and uses the result as the target 1612.00814 Paper vs code. So the released projection-only model is trained on silhouettes of ground-truth volumes, not on silhouettes taken from images alone.
• Within Splatter Image, the speed claims do not agree. The text says training is 246× faster than implicit methods. Table 7 gives about 225× against PixelNeRF and about 295× against VisionNeRF 2312.13150 §4.2 2312.13150 tab. 7.
• Within Splatter Image, for CO3D the text says 1,000× faster 2312.13150 §4.1.1. The Figure 4 caption says 100× 2312.13150 fig. 4.
• Within Splatter Image, the abstract's 38 FPS is not defined in the text 2312.13150 §4.2.
• Within pixelNeRF, the ablation's full model scores 23.43 PSNR for one view on chairs, while the main table reports 23.72. The ablation uses a shorter protocol with 25 target views, not 251 2012.02190 §5.1.1 2012.02190 tab. 3.
• Within pixelNeRF, the paper gives ω = 2.0 for two-object scenes, while the released multi-object config inherits 1.5. The card could not check which value produced the reported numbers 2012.02190 app. B.1 2012.02190 Paper vs code.
• Within One-2-3-45, the paper weights the depth loss at 1, while the released training config sets it to 0.0. The paper does not say which setting produced its model 2306.16928 app. training 2306.16928 §4.3.
• Within One-2-3-45, the paper trains for 300,000 iterations, while the released config stops at 200,000 2306.16928 app. training.
• Within ReconDrive, the paper gives two sources for vehicle motion. Section 3.1 says the velocity is regressed; section 3.3 says it is computed from annotated boxes. The released code uses the boxes and has no regression head 2603.07552 §3.1 2603.07552 §3.3 2603.07552 Paper vs code.
• Between ReconDrive and its code, the rendering loss weights differ. The paper gives 0.05 for the perceptual term and 1.0 for the L2 term. The code's effective weights are 0.1 and 2 2603.07552 §4.2 2603.07552 Paper vs code.
• Within ReconDrive, the text says DeformableGS leads on reconstruction LPIPS. Table 1 shows OmniRe at 0.1577, below DeformableGS at 0.1610 2603.07552 §4.2 2603.07552 tab. 1.
• Within ReconDrive, the text says per-scene methods take "about 30 minutes". Table 1 gives 23 to 46 minutes 2603.07552 §4.2 2603.07552 tab. 1.
• Speed numbers do not compare across variants. Splatter Image's 0.451 s is for 250 renders of one object on a V100 2312.13150 tab. 7. ReconDrive's 15 s is for one driving scene, and its hardware is not stated 2603.07552 §4.2. MVSplat's 0.044 s includes encoding and rendering on unstated hardware 2403.14627 tab. 1. GeFF's 0.39 s is a query latency on an onboard AGX Orin, while its offline baselines ran on a desktop with all frames batched 2403.07563 tab. 1 2403.07563 §4.1. PRN's 9.8 ms covers only the network, not face detection, and its README gives about 0.01 s on GPU 1803.07835 §4.4 1803.07835 Pitfalls. FLARE's under 0.5 s has no stated hardware 2502.12138 §1. iLRM's 0.025 s is on an RTX 4090, but its card gives no hardware for the 0.53 s figure at 32 views 2507.23277 tab. 1 2507.23277 tab. 5.
• Within MVSplat, the text gives its encoder time as 0.043 s, while Table 1 gives 0.044 s. The pixelSplat times differ in the same way 2403.14627 §4.2 2403.14627 tab. 1.
• Between MVSplat and its code, the cost volume's resolution and the softmax differ. The paper runs the softmax at full resolution, while the code keeps it at a quarter 2403.14627 §3.1 2403.14627 Paper vs code.
• Between PRN and its code, the paper gives no output scaling. The code applies a sigmoid times 281.6. A head trained on raw coordinates without that scale will not match the released weights 1803.07835 §3.2 1803.07835 Paper vs code.
• Between PRN and its code, the point count differs. The paper says more than 45K, and the code comments say about 40K and 45,128 1803.07835 §3.1 1803.07835 Paper vs code.
• Within GeFF, the teacher is named two ways. The method says MaskCLIP 2403.07563 §3.2, and the training details say a ViT-L CLIP model 2403.07563 §4.1. The paper does not say how the two fit together.
• Within GeFF, the total loss has a colour term 2403.07563 eq. 6, but the text gives no colour decoder or colour rendering 2403.07563 §3.2.
• Within Forge4D, the motion head's stage is described two ways. The paper trains it with the fusion in stage 3 2509.24209 §3.1. The code's comments say the original release used a motion head from a separate run 2509.24209 Paper vs code.
• Between Forge4D and its code, the training settings differ. The paper gives 5e-5 for every stage and an L2 flow loss without a stated weight. The shipped stage-3 config uses 1e-6 and an L1 flow loss with weight 10 2509.24209 app. C 2509.24209 Paper vs code.
• Within Forge4D, the speed hardware is given two ways. Section 4.3 says H200, and the appendix comparison says H20 2509.24209 §4.3 2509.24209 app. E.
• Within Forge4D, the text rounds its metric-scale error to 0.02 m, while the table gives 0.0264 m 2509.24209 §4.2 2509.24209 tab. 4.
• Within FLARE, the text says it "achieves higher performance" than DUSt3R, MASt3R and Spann3R. On ETH3D overall error, MASt3R is lower, at 0.3522 against 0.4978 2502.12138 §4.2 2502.12138 tab. 2.
• Within FLARE, the camera latents are described two ways. Section 3.1 gives one latent per image, and the appendix gives one for the reference view and one shared by the source views. The paper does not say which the results use 2502.12138 §3.1 2502.12138 app. A.
• Within FLARE, the input range differs. The abstract says 2 to 8 views, while section 4 says the model works up to 25 views, backed by one DTU table 2502.12138 abstract 2502.12138 tab. 6.
• Between the iLRM paper and its code, the output size differs. The paper's decoupled-resolution setting gives a compact output 2507.23277 §3.2. The released code implements only the full-resolution setting, which gives H × W Gaussians per view 2507.23277 Paper vs code 2507.23277 tab. 1.
• Within iLRM, the 4-view ablation baseline scores 29.24 PSNR, while the 4-view row of the main table scores 30.37. The ablations use another training setup, so the two are not directly comparable 2507.23277 tab. 7 2507.23277 tab. 1 2507.23277 Pitfalls.
• Between iLRM and its code, the training settings differ. The paper gives a peak learning rate of 2e-4, and it applies the perceptual term with weight 0.5 from the start. The code uses 4e-4 with a one-cycle cosine schedule, and its RE10K experiment is MSE only, with LPIPS weighted 0.05 after step 150,000 2507.23277 app. A 2507.23277 Paper vs code.

Papers

• Splatter Image: Ultra-Fast Single-View 3D Reconstruction: one 3D Gaussian per input pixel from a single image, rendered with Gaussian splatting, in one forward pass.
• MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images: two posed views to per-pixel Gaussians, with depth from a plane-sweep cost volume, in one forward pass.
• pixelNeRF: Neural Radiance Fields from One or Few Images: a NeRF conditioned on pixel-aligned features of posed input images, pooled across views and trained on multi-view 2D images.
• One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization: one photo to a textured mesh, using 32 views generated by a diffusion model and a cost-volume SDF network, in one pass.
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision: one photo to a 32³ voxel grid, trained through a differentiable perspective projection that compares silhouettes.
• Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network: one face image to a 256 × 256 position map of xyz face points, regressed by an encoder-decoder with no model fitting at test time.
• ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction: six camera views of two frames to driving-scene 4D Gaussians, from a frozen VGGT backbone with LoRA, in one pass.
• Learning Generalizable Feature Fields for Mobile Manipulation: posed RGB-D frames to latent points, decoded into a signed distance field and a language-aligned feature field, in one forward pass.
• Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos: four uncalibrated human videos to per-frame Gaussians, with a state token and a motion head, so times between frames are shown by moving and fusing Gaussians.
• FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views: 2 to 8 uncalibrated images to camera poses, point maps and Gaussians, in one network pass, with poses predicted first.
• iLRM: An Iterative Large 3D Reconstruction Model: posed images to per-pixel Gaussians, with viewpoint tokens refined over 12 layers by cross-attention to image tokens and self-attention across views.