Novel view synthesis

Overview

Given photographs of a scene with known camera poses, novel view synthesis renders the scene from a viewpoint that no photograph shows. Twelve of the twenty-seven cards on this page use the same basic recipe: a field maps a 3D point and a view direction to a colour and a density, and each pixel is formed by sampling that field along its camera ray and compositing the samples. KiloNeRF, DONeRF, K-Planes, HexPlane, FreeNeRF, ST-NeRF, pixelNeRF, DS-NeRF, Plenoxels and Dynamic View Synthesis all do this 2103.13744 §3.1 2103.03231 §3 2301.10241 §3.1 2301.09632 §3.4 2303.07418 §3.1 2104.14786 §4.3 2012.02190 §3 2107.02791 §3.1 2112.05131 §3.1 2105.06468 §3.2. DietNeRF and BARF do the same 2104.00677 eq. 1 2104.06405 §3.2. For dynamic scenes the point also carries a time. KiloNeRF fits its field so that rendered pixels match the photographs 2103.13744 eq. 5. K-Planes' paper does not describe its reconstruction loss in the text, so the same is not stated for it. ST-NeRF keeps one such field per moving layer 2104.14786 §4.2. pixelNeRF instead conditions one shared network on pixel-aligned image features of posed inputs, so a new scene needs no per-scene fitting 2012.02190 §4.1. Splatter Image also skips per-scene fitting. A network predicts one 3D Gaussian per pixel of an input image, and the Gaussians are drawn with Gaussian Splatting, a fast approximation of the volume-rendering integral 2312.13150 §3.2 2312.13150 §3.1. SynSin also skips per-scene fitting. It predicts a point cloud from one image and splats it into the new view, and it trains on image pairs without depth labels 1912.08804 §3.2 1912.08804 §1. FLARE also skips per-scene fitting, for two to eight photos with unknown camera poses. One network predicts the cameras, then point maps, then Gaussians, and the Gaussians are drawn directly 2502.12138 §3 2502.12138 §1. SparseGS keeps per-scene fitting but applies it to 3D Gaussians rather than to a field 2312.00206 §3.1. FSGS does the same for three photos, and grows Gaussians where they are sparse 2312.00451 §3.2. PSHuman has no field at all. From one full-body photo it generates the unseen sides of a person as views, then carves a textured mesh from them 2409.10141 §3.1 2409.10141 §3.2. Three cards cover driving logs, and all three use Gaussians rather than a field. DriveSplat fits one set of neural Gaussians to the background and to each tracked actor 2508.15376 §3.2 2508.15376 §3.3. The Hybrid entry fits 3D Gaussian splatting and cleans its renders with a diffusion model 2510.12308 §3.1 2510.12308 §3.2. ReconDrive predicts Gaussians from two frames of six cameras in one forward pass 2603.07552 §3.1 2603.07552 §3.2. Forge4D does the same for human videos filmed by four uncalibrated cameras, and adds a motion field for times between frames 2509.24209 §3.1 2509.24209 §3.4. Hierarchical 3D Gaussians targets the largest captures. It trains each block of a street-level scene separately, then arranges the Gaussians in a tree, and draws a coarser or finer cut of that tree for each view 2406.12080 §1 2406.12080 §4.2. SV3D takes a different route: it generates the views itself. A video diffusion model draws an orbit of the object from one photo, with the views kept consistent with each other 2403.12008 §1 2403.12008 §3. Three cards propose no renderer: a reality check on monocular dynamic benchmarks 2210.13445 §1, a dataset of 3D objects 2307.05663 §1 and a survey of 3D Gaussian splatting 2401.03890 §1. The last two are covered after the variants.

The cost sits in the query. The authors of KiloNeRF note that the original NeRF evaluates a deep MLP at every sample point of every pixel, which makes rendering too slow for interactive uses such as VR 2103.13744 §1 2103.13744 §3.1. K-Planes starts from the storage side: a direct grid for a 4D (space and time) scene would be prohibitively large, so it stores the field on factorised 2D planes 2301.10241 §1. HexPlane makes the same point for multi-view video: a naive 4D volume at N=512 and T=32 needs about 48 GB in float32 2301.09632 §3.1. A third problem is data: with only a few photographs, a plain NeRF can memorise the training views and invent geometry near the camera in unseen views 2303.07418 §3.2. A fourth problem is the per-scene fit itself: Splatter Image's authors note that implicit NeRF-style reconstructors can take up to 2 s to render one 128 × 128 image, and that Gaussian Splatting needs many images per scene 2312.13150 §2. A fifth problem is motion: earlier methods for moving people need dense capture or fail on large motions, and most neural methods cannot be edited 2104.14786 §1. A sixth is measurement. On a single video, standard full-image PSNR, SSIM and LPIPS can reward a model for multi-view cues that the camera never recorded 2210.13445 §4.2. A seventh is geometry from few views. Plain NeRF must solve the 3D correspondence problem implicitly, although the structure-from-motion step that gives the camera poses already produces sparse 3D points 2107.02791 §1. For 3D Gaussian splatting, the same few-view setting produces floaters and background collapse at unseen viewpoints 2312.00206 §1. FSGS adds that the standard Gaussian densification grows Gaussians from noisy gradients when views are few, and that too little parallax lets them grow large and overfit 2312.00451 §3.1. An eighth is monocular video: each moment is seen from one camera position, so the scene's shape and motion are under-determined 2105.06468 §1 2105.06468 §3.3. A ninth is the driving log. Test views lie one or two lanes from the recorded path, or face the other way, and the scene is large with sparse LiDAR depth 2510.12308 §2 2508.15376 §1. A tenth is the camera pose. NeRF needs accurate poses, and BARF shows that they can be estimated while the scene is fitted 2104.06405 §1 2104.06405 §3.2. An eleventh is scale. Radiance fields and Gaussians keep the whole scene in memory and draw every primitive, which for a capture of several kilometres exceeds GPU memory and is too slow 2406.12080 §1. The variants below attack different parts of the problem.

Variants

Grid of tiny MLPs with distillation (KiloNeRF)

KiloNeRF splits the scene's bounding box into a uniform grid with at most 16 cells on its largest side, and gives each cell its own tiny MLP of four hidden layers with 32 units each 2103.13744 §3.2 2103.13744 §4.1. Each point is routed to the network of the cell it falls into 2103.13744 eq. 6. Training has three stages: a NeRF without hierarchical sampling is trained as a teacher, each tiny network is distilled from the teacher inside its own cell, and then the whole model is fine-tuned on the photographs 2103.13744 §3.3 2103.13744 §4.1. Rendering combines empty space skipping with early ray termination and a custom kernel for evaluating many small networks of different batch sizes 2103.13744 §3.4 2103.13744 §3.5. The empty-space mechanism is described on its own page, empty space skipping.

The authors report renders of 26 to 91 ms per image on a GTX 1080 Ti, against 37 to 183 s for NeRF on the four bounded datasets they used, with image quality close to NeRF's 2103.13744 tab. 1. On the Lego scene, skipping empty space and terminating rays early accounts for a 71x speed-up, and the smaller networks for most of the rest 2103.13744 tab. 2. The paper also reports that KiloNeRF trails NSVF on the Synthetic-NSVF scenes (33.37 vs 35.13 PSNR) 2103.13744 tab. 1. Card: KiloNeRF.

Depth oracle chooses the sample positions (DONeRF)

DONeRF keeps the NeRF-style field and volume rendering, but it changes where the field is queried. A depth oracle MLP runs once per ray. It classifies the ray into log-spaced depth bins, and the shading MLP is then evaluated at only 2 to 16 points placed around the oracle's peaks, by inverting its cumulative distribution 2103.03231 §3 2103.03231 §5.2. The oracle takes 128 positions, one at each depth-class centre, as input 2103.03231 §5.2 2103.03231 tab. 1. Before the warp, distances are remapped on a logarithmic scale and pulled towards the centre of the view cell 2103.03231 eq. 3 2103.03231 eq. 5.

The oracle is trained on targets built from a reference depth per pixel. The targets are filtered in image space over a neighbourhood of 5 pixels and in depth space over 5 classes, which spreads them so that the oracle errs towards keeping a little extra empty space 2103.03231 eq. 7 2103.03231 eq. 8 2103.03231 §6.1. The ablation shows where the gain comes from. At 2 samples per ray, one oracle input with the neighbourhood filter off gives 27.33 dB, the 5-pixel neighbourhood filter gives 28.69 dB, and 128 oracle inputs give 29.96 dB 2103.03231 tab. 1. The authors attribute 1.3 to 1.8 dB to the neighbourhood filter, 0.6 to 1.5 dB to classification and about 1 dB to the multiple oracle inputs 2103.03231 §5.3.

Without the reference depth, the paper trains the same oracle on depth from a dense NeRF. Averaged over six scenes, those variants fall 0.8 to 1.5 dB short of the ground-truth versions 2103.03231 tab. 2 2103.03231 §6.4. The reference depth is a single surface per ray, so transparent surfaces and mirrors are a known problem 2103.03231 §6.4.

The authors report 31.14 dB mean PSNR at 4 samples per ray, against 30.52 dB for NeRF at 256 samples, over six Blender scenes at 400×400 2103.03231 tab. 2. At 2 samples the average is 29.50 dB, which is below NeRF 2103.03231 tab. 2. On 800×800 images with 2 samples on one RTX 3090, the authors measure 51.3 ms per frame, of which shading takes 34.9 ms 2103.03231 §6.1. Card: DONeRF.

Factorised 2D planes with product combination (K-Planes)

K-Planes stores one 2D feature plane for each pair of input dimensions. A static 3D scene uses three planes, and a dynamic scene adds three planes that pair time with each spatial axis 2301.10241 §3.1. A point is read on every plane by bilinear interpolation, and the readings are multiplied together 2301.10241 eq. 1 2301.10241 eq. 2. The authors show in an ablation on the Lego scene that multiplication outperforms addition for the explicit model (PSNR 35.29 vs 28.78) 2301.10241 tab. 2. Colour is decoded either by a linear model with a learned colour basis, or by a small MLP in a hybrid variant 2301.10241 §3.3. Scenes can use several resolution scales 2301.10241 §3.2, and unbounded scenes use a contraction together with a two-stage proposal sampler 2301.10241 §3.4. Card: K-Planes. The representation itself is covered on explicit feature grids.

On the static synthetic benchmark the authors report PSNR of 32.21 (explicit) and 32.36 (hybrid), against 33.18 for I-NGP and 33.14 for TensoRF 2301.10241 tab. 3. Training takes 38 minutes on that benchmark and 33 to 52 minutes on the static real and monocular dynamic settings, measured on one NVIDIA A30 2301.10241 tab. 3. On the monocular dynamic benchmark the reported PSNR (31.05 explicit, 31.61 hybrid) is below V4D (33.72) and TiNeuVox (32.67) 2301.10241 tab. 3. On photo collections the authors report 22.25 PSNR against NeRF-W's 27.00, while training in about 35 minutes instead of NeRF-W's roughly 384 hours 2301.10241 tab. 3. These numbers are author-reported.

Six planes for multi-view video (HexPlane)

HexPlane keeps K-Planes' idea of storing the field on 2D planes, but for multi-view video of a dynamic scene. It stores six learned planes: three spatial (XY, XZ, YZ) and three spatio-temporal (ZT, YT, XT). Each spatial plane is paired with a spatio-temporal one. Within a pair the two bilinear readings are multiplied, the three products are concatenated, and a learned matrix projects them to the feature width 2301.09632 §3.3 2301.09632 eq. 5 2301.09632 eq. 6. A tiny MLP turns the feature and view direction into colour. An emptiness voxel, which stores the maximum opacity over time, lets samples in empty cells be skipped 2301.09632 §3.4. Training starts from a coarse grid that is refined during training 2301.09632 §3.4. Card: HexPlane.

The ablation on D-NeRF supports the pairing. Using only the three spatial planes gives 20.37 PSNR and using only the spatio-temporal planes 21.11, against 31.04 for the full six-plane model 2301.09632 tab. 4. Multiplying within pairs and concatenating across them is the best of the fusion choices the authors tried, with the MLP and without it 2301.09632 tab. 5.

On Plenoptic Video, the authors report 29.47 PSNR on the flame-salmon scene, against 29.58 for DyNeRF, and 12 h of training on one V100 against 1344 h for DyNeRF 2301.09632 tab. 1. On D-NeRF, which is monocular and has no deformation field, they report 31.04 PSNR and 11 min 30 s of training 2301.09632 tab. 2. The trained model takes about 200 MB 2301.09632 §4.1. All numbers are the authors' own.

Frequency and occlusion regularisers for few views (FreeNeRF)

FreeNeRF keeps a plain NeRF and adds two training-time terms, with no pre-trained model and no patch rendering from novel poses, which the authors contrast with earlier few-view fixes 2303.07418 §1 2303.07418 §3.2. The first term is a frequency mask. Each entry of the sine/cosine encoding is multiplied by a visibility weight that starts with only the low bands visible and opens higher bands as training proceeds, reaching full length at the schedule end T 2303.07418 eq. 4 2303.07418 eq. 5 2303.07418 §3.2. The second is an occlusion penalty: the density of the M samples nearest the camera is added to the loss with weight 0.01 2303.07418 eq. 6 2303.07418 §4.1. The schedule end is 90%, 70% and 20% of training for 3, 6 and 9 views in the main text, but the appendix gives 50% for 9 views; see contradictions.

The authors report 19.92 PSNR on DTU with 3 views, against 9.10 for the mip-NeRF baseline they reproduced, and 19.63 against 16.11 on LLFF with 3 views 2303.07418 tab. 2 2303.07418 tab. 3. Blender results follow DietNeRF's 8-view setup, with 24.26 PSNR against 23.15 for DietNeRF 2303.07418 tab. 1. Training time is 1.02x to 1.04x the baseline's, measured as wall-clock time on the authors' setup 2303.07418 tab. 4. Card: FreeNeRF. The mask is explained in more detail on frequency regularization.

Semantic consistency with a frozen CLIP encoder (DietNeRF)

DietNeRF keeps the usual pixel loss on the observed views. Every K iterations it also renders one random unseen pose at reduced resolution, and compares that render with a random training image in the embedding space of a frozen CLIP ViT, using cosine similarity 2104.00677 alg. 1 2104.00677 §4.1. An unseen view has no photo to compare pixels with, so the term compares meaning. The authors assume that an image encoder gives similar features to different views of one object 2104.00677 §4.2 2104.00677 fig. 3. The semantic render uses about 15 to 20% of the rays of a full image, and the term runs only every 10 to 16 iterations 2104.00677 §4.4. A pixel-only fine-tune afterwards recovers detail 2104.00677 tab. 5.

The authors report 23.147 PSNR and 74.9 FID on eight-view Realistic Synthetic scenes, against 14.934 and 228.1 for NeRF. With a 50k-iteration pixel fine-tune they report 23.591 PSNR 2104.00677 tab. 1. On the unseen half of the Lego scene, with 14 views, the base model reaches 20.75 PSNR against 19.66 for NeRF, and the fine-tuned model 22.21 against 21.55 for Simplified NeRF 2104.00677 tab. 3. On the same Lego setting with eight views, CLIP ViT-B/32 gives 23.90 PSNR and ImageNet ViT-B/32 gives 22.06, so the choice of encoder matters 2104.00677 tab. 4. On single-view DTU, the semantic fine-tune of pixelNeRF lowers PSNR from 16.05 to 14.24 and lowers FID from 265.2 to 190.7 2104.00677 tab. 2. Card: DietNeRF.

Single-pass Gaussians from one image (Splatter Image)

Splatter Image does not fit a scene. A 2D U-Net reads an input image and outputs a grid of the same width and height. Each cell holds one 3D Gaussian: its opacity, a depth, an offset, a scale, a rotation and a colour 2312.13150 §3.2 2312.13150 §3.6. The Gaussian is placed on the pixel's camera ray at the predicted depth, then moved by the predicted offset, so that the network can put a Gaussian on a part of the object the camera does not see 2312.13150 §3.2 2312.13150 eq. 3. Training renders the predicted Gaussians at a target view with a known pose change and minimises the squared error to that image 2312.13150 §3.3 2312.13150 eq. 4. A second stage adds LPIPS, a perceptual loss, with a small weight 2312.13150 app. C. For several input views, the network runs on each view, the Gaussians are moved into one frame with the relative pose, and the sets are merged. Cross-view attention runs only at the lowest U-Net resolution 2312.13150 §3.4 2312.13150 §3.6. Colour is either constant or degree-1 spherical harmonics 2312.13150 §3.5.

The authors report 24.00 PSNR on ShapeNet-SRN cars from one view with relative pose, against 22.25 for SRN in the same table 2312.13150 tab. 1. On Google Scanned Objects they report 21.06 PSNR against 18.06 for OpenLRM 2312.13150 tab. 4. Rendering 250 views takes 0.451 s on one V100, and a single render takes 0.0017 s; the same test takes 463 s for PixelNeRF and 608 s for VisionNeRF, in the authors' table 2312.13150 tab. 7. The method is described as object-centric and as working only with relative camera poses 2312.13150 §2 2312.13150 app. B. Card: Splatter Image. The rasteriser is covered on Gaussian splatting, and the single-pass design on feed-forward reconstruction.

Poses, point maps and Gaussians in one network (FLARE)

FLARE takes two to eight photos of a scene whose camera poses are unknown, and predicts 3D Gaussians in one forward pass 2502.12138 §3. The pipeline has four stages, and camera poses are an intermediate result rather than an input 2502.12138 §3. A pose transformer reads the patch tokens of each image together with learnable camera latents, and gives each image a coarse pose made of a translation and a normalised quaternion 2502.12138 §3.1 2502.12138 eq. 1. A second transformer turns the tokens into camera-centric point maps and refines the poses in the same pass 2502.12138 §3.2 2502.12138 eq. 2. A third transformer maps those local tokens into one global frame, as a learned projection rather than a geometric reprojection 2502.12138 §3.2 2502.12138 eq. 4. The Gaussian centres are the global points, and a head adds opacity, rotation, scale and colour, fused with VGG features of the input images 2502.12138 §3.3. Rendering is by splatting, so a new view needs no fitting 2502.12138 §3.3.

Training uses eight views per sample 2502.12138 §4. The loss has three parts: a Huber pose loss, a confidence-weighted point-map loss in camera and global frames, and a splatting loss with a perceptual term and a depth term 2502.12138 §3.4 2502.12138 eq. 12. The depth target is a pretrained monocular estimate 2502.12138 §3.4. Pose noise is added during training only, so the network learns to cope with noisy poses 2502.12138 §3.2.

On RealEstate10K with five input images and no fine-tuning, the authors report an AUC@30° of 79.9, against 61.1 for MASt3R and 51.6 for PoseDiffusion 2502.12138 tab. 1. For novel views on DL3DV, FLARE with eight input views gives 23.33 dB PSNR, against 22.08 dB for MVSplat with its two closest views 2502.12138 tab. 3. The novel-view model on RealEstate10K is fine-tuned with intrinsics as a condition, so that setting uses calibration 2502.12138 §4.3. Training takes about 14 days on 64 GPUs 2502.12138 §4. Card: FLARE. The feed-forward design is covered on feed-forward reconstruction.

Feature point cloud from one image with a GAN refiner (SynSin)

SynSin renders a new view from one RGB image and a relative camera pose. It runs two networks on the image. A spatial feature network gives each pixel a 64-dimensional feature, and a depth network gives each pixel a depth 1912.08804 §3.1. The pixels are lifted to 3D points that carry their features, moved by the relative pose, and splatted into the target view by a differentiable point renderer. Each target pixel keeps its K nearest points, sorted by depth, and blends them front to back. A point's influence is zero outside a disk of radius r and falls off linearly inside it [arxiv:1912.08804 §3.2] 1912.08804 eq. 1 1912.08804 eq. 2. The splatted features still have holes, so a refinement network fills them and produces the image. A discriminator is applied during training 1912.08804 §3.3. The loss mixes an adversarial term, an L1 term and a perceptual term, with weights 1, 1 and 10 1912.08804 §3.4. Training is end-to-end, so depth is learned from the error in the final image, and no depth labels are used 1912.08804 §1 1912.08804 §3.4.

The renderer alone takes 36 ms forward and 5 ms backward for six 256 × 256 views of 512 × 512 point clouds on one V100 1912.08804 §3.2. The authors report 20.91 PSNR on Matterport3D, 22.31 on RealEstate10K and 21.94 on Replica, all from one input view 1912.08804 tab. 1. On RealEstate10K, the 3DView baseline reaches 21.88 and StereoMag reaches 25.34, but StereoMag uses two input views 1912.08804 tab. 2 1912.08804 §4.2. Trained at 256 × 256 and tested at 512 × 512, SynSin reaches 22.06 PSNR 1912.08804 tab. 3. In a user study on 100 animated RealEstate10K trajectories, raters preferred SynSin over the voxel baseline in 68.7% of either-or votes 1912.08804 tab. 4. An ablation with true depth scores higher on all pixels on Matterport3D (22.65 against 20.91), but lower on the regions the input view does not show (19.64 against 19.80). The authors say true depth does not necessarily help 1912.08804 tab. 1 1912.08804 §4.3. Card: SynSin.

Layered space-time fields for editing (ST-NeRF)

ST-NeRF targets free-viewpoint video of scenes with several performers, captured with 16 synchronised RGB cameras covering up to 180 degrees 2104.14786 §1 2104.14786 §3. Each performer and the background is one layer. A layer has a deformation MLP that moves a point at time t into a shared canonical space, and a radiance MLP that returns colour and density at the moved point 2104.14786 §4.2 2104.14786 eq. 2. Time enters through positional encoding rather than per-frame latent codes 2104.14786 §4.2. For each ray, every layer's bounding box is intersected with the ray, and the samples of all layers are merged, sorted and composited together, so a performer in front hides one behind it 2104.14786 §4.3 2104.14786 eq. 6.

Because each layer has its own box and time input, edits change only those inputs: an affine transform of the box, a retiming of the timestamp, duplication or removal of a layer, and a density scale for transparency 2104.14786 §5 2104.14786 eq. 10 2104.14786 eq. 11. The paper shows edits as figures and a supplementary video, not as numbers 2104.14786 §6.1.

The authors report PSNR 33.2161 on their captured dataset, against 28.2553 for NeRF-T, the best PSNR baseline in their table 2104.14786 tab. 1. On the synthetic scene, removing the deformation MLP lowers PSNR_test from 29.91 to 25.55 2104.14786 tab. 2. Training takes 12 to 36 h per scene on one RTX 3090, plus 2 to 3 days of refinement at full resolution. Rendering a 1920 × 1080 frame with three layers takes about 2 min, as the authors report 2104.14786 §6. The method needs 16 cameras, tracks only humans, and its box-limited networks cause ghosting after an edit 2104.14786 §6.4. Card: Editable Free-viewpoint Video Using a Layered Neural Representation.

Static and dynamic fields from one masked video (DynamicNeRF)

Dynamic View Synthesis from Dynamic Monocular Video takes one video and a binary mask of the moving objects in every frame 2105.06468 §3.1. A static NeRF is trained only on pixels outside the mask, with the usual volume rendering loss 2105.06468 §3.2 2105.06468 eq. 1-4. A second MLP takes a point and a time, and outputs forward and backward 3D scene flow, density, colour and a blend weight 2105.06468 §3.3 2105.06468 eq. 5. The same MLP is queried at the flowed points and at the neighbouring frames t-1 and t+1, so each point receives photometric constraints at three times 2105.06468 eq. 6 2105.06468 eq. 7 2105.06468 eq. 9. The two fields are blended per sample before compositing 2105.06468 §3.5 2105.06468 eq. 14. The time- conditioned field is covered on dynamic neural radiance field.

Most of the method's weight sits in its regularisers 2105.06468 §3.4. The scene flow, projected into the reference camera, must reproduce RAFT optical flow between neighbouring frames 2105.06468 §3.4 2105.06468 fig. 5. Flow is kept small and smooth, and forward and backward flow are made to cancel 2105.06468 eq. 10 2105.06468 eq. 11. A cycle term asks forward and then backward flow to return to the start 2105.06468 eq. 12-13. Depth is matched to a monocular depth estimator with a robust, shift-and-scale-invariant loss, and the blend weight is pushed towards 1 on static points 2105.06468 §3.4.

The authors report average PSNR 24.74 and LPIPS 0.118 over seven sequences of the Dynamic Scene dataset, against 23.99 and 0.205 for Li et al. 2105.06468 tab. 1. In their Playground ablation, removing the motion loss gives 22.61 PSNR and removing the depth-order loss gives 22.99, against 23.65 for the full model 2105.06468 tab. 2. Training takes about 22 hours per 12-frame sequence on one V100 [arxiv:2105.06468 supp. Implementation Details]. Card: Dynamic View Synthesis from Dynamic Monocular Video.

Checking monocular dynamic benchmarks (reality check)

This card proposes no renderer. It asks whether the usual scores for monocular dynamic view synthesis mean what they seem to. The usual test set comes from several cameras filmed at once, and the training video alternates frames from those cameras. The authors call this camera teleportation. The model then sees multi-view cues that a single phone camera could never record 2210.13445 §1 2210.13445 §4.1.

The paper offers three tools. Two effective multi-view factors, Ω and ω, measure how much multi-view signal a video carries; ω needs only the camera poses and the frame rate, and the authors recommend reporting it always and Ω when the depth and flow estimates allow 2210.13445 §3 2210.13445 eq. 1 2210.13445 eq. 2. Image metrics (mPSNR, mSSIM and mLPIPS) are computed only on pixels that the training frames saw 2210.13445 §4.2 2210.13445 app. B. PCK-T checks whether tracked keypoints land in the right place, not just whether the picture looks right 2210.13445 §4.2 2210.13445 app. C. A new iPhone dataset, with one handheld camera for training and more varied, faster motion, supplies the test sequences 2210.13445 §4.3 2210.13445 app. D.

With training teleportation removed, the paper reports a loss of 1 to 2 dB of masked PSNR and about 5% in PCK-T on existing datasets 2210.13445 §1. On its rectified Nerfies-HyperNeRF sequences, the masked PSNR of HyperNeRF is 21.16, and on the iPhone dataset it is 16.81 2210.13445 §5.1 2210.13445 §5.2. These two sets of numbers come from different datasets, so the drop is not a single-factor effect. Card: Monocular Dynamic View Synthesis: A Reality Check.

Image-conditioned field shared across scenes (pixelNeRF)

pixelNeRF runs a fully convolutional ResNet34 on each posed input image and attaches the pixel-aligned features to every query point that projects onto that pixel 2012.02190 §4.1 2012.02190 §5. Points and view directions are expressed in the input camera's frame rather than in a canonical frame 2012.02190 §4. With several inputs, each view passes through the first layers, the results are averaged, and the last layers predict density and colour 2012.02190 eq. 5 2012.02190 eq. 6. The authors place the average after the third ResNet block 2012.02190 app. B.1. Training uses only posed multi-view images, with no 3D supervision 2012.02190 §1 2012.02190 §4.2.

The authors report 23.72 PSNR for one-view chairs and 26.20 for two views, with SRN at 22.89 and 24.48 in the same SRN-protocol table 2012.02190 tab. 2. Over 13 ShapeNet categories with one input view, they report 26.80 PSNR against 23.28 for SRN and 22.70 for DVR 2012.02190 tab. 4. On unseen categories the figure is 22.71 against 18.71 for SRN 2012.02190 tab. 5. The cost sits in training and in rendering: one model takes 5 to 6 days on one GPU, and rendering is as slow as NeRF's and grows linearly with the number of inputs 2012.02190 app. B.2 2012.02190 §6. The paper's own DTU results exist only as a plot 2012.02190 §5.3. Card: pixelNeRF. The feed-forward design is covered on feed-forward reconstruction.

Depth term from structure-from-motion keypoints (DS-NeRF)

DS-NeRF adds a depth term to NeRF. The sparse 3D keypoints that the pose estimator already produces give each training ray a depth target 2107.02791 §3.2. The headline loss treats the ray's termination distribution as a Gaussian belief around that depth, with a width set by the keypoint's average reprojection error 2107.02791 §3.2. The distribution form is the one the authors evaluate most, but the released default is a plain squared error between expected depth and keypoint depth, weighted by 0.1 (card lists the detail). The paper's ablation shows the plain squared error at 19.5 PSNR and the Gaussian form at 20.2 for two NeRF Real views 2107.02791 tab. 1.

On NeRF Real with two training views, the authors report 20.2 PSNR against 13.5 for NeRF 2107.02791 tab. 1. The paper reports that DS-NeRF reaches a given PSNR in 2 to 3 times fewer iterations, at about 362 ms per iteration against 360 ms for NeRF on one RTX A5000 2107.02791 §4.5. On DTU at three views, DS-NeRF's table gives 16.9 PSNR and lists pixelNeRF-DTU at 19.3 2107.02791 tab. 2. The authors attribute the gap to DTU-trained priors 2107.02791 §4.3. The method depends on structure-from-motion for every training set, and the depth is only as good as that step 2107.02791 §3.2 2107.02791 app. A. Card: DS-NeRF.

Coarse-to-fine encoding with unknown poses (BARF)

BARF fits a NeRF and its camera poses at the same time, so it does not need the poses before training 2104.06405 §3.2 2104.06405 eq. 8. The paper's diagnosis is that the full positional encoding makes the gradients with respect to the poses incoherent, so the poses settle on wrong values 2104.06405 §3.3. BARF masks the sine and cosine bands with a smooth window. Training starts with only the raw coordinates and the lowest band open, and the bands open one after another until all ten are available 2104.06405 eq. 13 2104.06405 eq. 14 2104.06405 §3.4. The idea follows coarse-to-fine image registration, where blurring first widens the basin in which the alignment converges 2104.06405 §3.1.

The authors report a mean rotation error of 0.193 degrees on eight Blender scenes with perturbed poses, against 6.167 with the full encoding, and PSNR of 27.50 dB against 29.40 for NeRF with true poses 2104.06405 tab. 2. On eight LLFF scenes started from identity poses, the rotation error is 0.573 degrees and PSNR 23.97 dB, against 22.56 dB for a NeRF trained with structure-from-motion poses 2104.06405 tab. 3. A version with no encoding at all scores 0.699 degrees and 23.82 dB on the same scenes, so the authors call BARF competitive with it, not uniformly better 2104.06405 tab. 5. The window is hand-set: the NeRF schedule opens the bands between iterations 20K and 100K of 200K, and the authors say a per-scene schedule is out of scope 2104.06405 §4.2 2104.06405 app. B.2. The result also depends on depth parametrisation. With metric depth instead of inverse depth, the LLFF rotation error rises to 1.580 degrees 2104.06405 tab. 6. Card: BARF.

Voxel grid with spherical harmonics, no network (Plenoxels)

Plenoxels replaces the MLP with a sparse grid. Each occupied voxel corner stores an opacity and 27 degree-2 spherical harmonic coefficients, nine per colour channel 2112.05131 §3.2. A sample's opacity and colour come from trilinear interpolation of the eight nearest corners 2112.05131 §3.3. Rendering uses NeRF's volume rendering formula 2112.05131 eq. 1, and the loss is mean squared error plus a total-variation term, optimised with RMSProp 2112.05131 §3.5 2112.05131 eq. 2. Training starts from a dense coarse grid, prunes it, and splits each remaining voxel in half 2112.05131 §3.4. Sparsity and beta-distribution losses are added for real forward-facing and 360-degree scenes 2112.05131 §3.7.

The authors report 31.71 PSNR on the eight NeRF synthetic scenes against 31.85 for JAXNeRF, which they reran themselves. Training takes about 11 minutes per scene on one Titan RTX, against 1.45 days for JAXNeRF 2112.05131 tab. 2. On the real forward-facing scenes the figures are 26.29 against 26.71 for JAXNeRF, in 24 minutes 2112.05131 tab. 2. Grid resolution matters: the synthetic average is 31.71 at 512³, 28.46 at 128³ and 26.11 at 64³ 2112.05131 tab. 4. The authors say the implementation is not tuned for rendering and reach about 15 fps, and that regularisation weights were fixed per dataset 2112.05131 §1 2112.05131 §5. The official code needs CUDA (card lists the detail). Card: Plenoxels.

Depth priors for sparse-view Gaussians (SparseGS)

SparseGS keeps 3D Gaussian splatting and changes its training for few inputs 2312.00206 §3. The paper argues that alpha-blended depth sends gradients to the wrong Gaussians, and it defines a softmax-scaled depth that weights each Gaussian by its transmittance-weighted opacity times an exponential term 2312.00206 §1 2312.00206 §3.1 2312.00206 eq. 5. Depth is matched to a monocular depth prior with one minus the Pearson correlation on random patches, plus a global term 2312.00206 §3.2 2312.00206 eq. 8. For unseen views, the method uses score distillation from Stable Diffusion 2.1 on cameras placed around the scene, and warps training views to nearby poses with a depth-based loss 2312.00206 §3.3.1 2312.00206 §3.3.2. After fitting, it removes Gaussians that sit at the wrong depth 2312.00206 §3.4.

On Mip-NeRF360 with 12 views, the authors report 19.37 PSNR against 17.49 for 3DGS and 19.31 for FSGS 2312.00206 tab. 1. Their 12-view ablation goes from 17.49 for the 3DGS baseline to 18.86 with the softmax-scaled depth correlation, and to 19.37 once floater pruning is added 2312.00206 tab. 3. On LLFF, the PSNR is 19.86 against 19.88 for FSGS, while the SSIM is higher at 0.668 against 0.612 2312.00206 tab. 2. On DTU, the PSNR is 18.89, below DNGaussian's 18.91 2312.00206 tab. 2. The paper gives no training time or hardware 2312.00206 §4.1. The official code is under a non-commercial research licence, so the method should be implemented from the paper (card lists the detail). Card: SparseGS.

Gaussian growth and pseudo views for three photos (FSGS)

FSGS keeps 3D Gaussian splatting and changes three parts of training for few views 2312.00451 §3. The first is unpooling. Each Gaussian takes its three nearest neighbours, and where its mean distance to them is above a threshold, a new Gaussian is placed at the midpoint of each link to them 2312.00451 §3.2 2312.00451 eq. 4. The second is pseudo views. From iteration 2,000, a camera is made from two training cameras. Its rotation is their average, and its position is their average plus noise, and it adds an extra depth term 2312.00451 §3.3 2312.00451 eq. 5. The third is a depth prior. A pretrained monocular model gives a depth map for each training image and each pseudo view, and a Pearson correlation loss compares it with the rendered depth. Correlation ignores the prior's unknown scale 2312.00451 §3.3 2312.00451 eq. 6.

The proximity threshold is 10, in units the paper does not define, so it has to be re-tuned when the scale of the scene changes 2312.00451 §4.1. Training runs 10,000 steps, which the authors time at about 9.5 minutes on LLFF and 24 minutes on Mip-NeRF360, both on one A6000 2312.00451 §4.1.

The authors report 20.31 PSNR, 0.652 SSIM and 458 FPS on LLFF at 1/8 resolution with three views, against 19.86 PSNR for SparseNeRF and 17.43 for 3D-GS 2312.00451 tab. 1. Their ablation adds the parts in turn. Unpooling alone raises PSNR from 17.43 to 18.64, the depth term to 19.83, and the pseudo views to 20.31 2312.00451 tab. 4. On Mip-NeRF360, which uses 24 views and so is not few-shot, they report 23.70 PSNR at 1/8 resolution 2312.00451 tab. 2. FPS is measured on one A6000 for every method 2312.00451 §4.1. Card: FSGS. The rasteriser is covered on Gaussian splatting.

Generated body views and a carved mesh (PSHuman)

PSHuman is included because it produces views of a side that the photo does not show, but its output is a mesh, not a rendering field. The input is one full-body photo of a person. A diffusion model, fine-tuned from Stable Diffusion 2.1-unclip, generates six body views (front, front-left, left, back, right, front-right) and a face view, as colour and normal maps 2409.10141 §3.1 2409.10141 §4. The paper argues that six views balance cost and quality 2409.10141 §3. SMPL-X renders from the target viewpoints are fed in as an extra condition, which the authors say constrains pose and self-occlusion 2409.10141 §3.1.2. A cross-scale step blends face tokens into the front body view, to address the face distortion that multiview diffusion on a full-body image otherwise causes 2409.10141 §1 2409.10141 §3.1.1 2409.10141 eq. 5.

The carving step then fits an SMPL-X model to the normal and silhouette maps, optimises the vertices and face connectivity of a mesh against them, and colours the mesh from the generated colour views 2409.10141 §3.2 2409.10141 eq. 7 2409.10141 eq. 8 2409.10141 eq. 9. This is an optimisation for each subject. The authors say the whole process takes about one minute 2409.10141 §1.

The authors report a Chamfer distance of 0.9688 cm on CAPE-NFP, against 1.5511 for ICON, the best baseline in their table 2409.10141 tab. 1. On THuman2.1 the figures are 0.4399 against 0.5754 for SIFU 2409.10141 tab. 1. For appearance on four views of a THuman2.1 subset, they report 20.8548 PSNR against 19.6071 for GTA [arxiv:2409.10141 tab. 3]. These geometry numbers use the ground-truth SMPL-X, so the pose error is removed 2409.10141 §4. Card: PSHuman.

Orbit video diffusion from one image (SV3D)

SV3D adapts Stable Video Diffusion, an image-to-video model, to draw an object from a ring of camera positions 2403.12008 §3. It keeps SVD's UNet and removes the fps and motion conditions. The input image's latent is concatenated to the noisy frames, its CLIP embedding is added to cross-attention, and the camera's elevation and azimuth are embedded 2403.12008 §3. There are three models. SV3D-u takes the image alone and makes a static orbit. SV3D-c also takes a camera trajectory. SV3D-p is first trained on static orbits and then on dynamic ones, whose elevation follows a sum of sinusoids and loops back to the input view 2403.12008 §3.

The authors replace SVD's linear guidance ramp with a triangle. The guidance weight is 1 at the front view, rises to 2.5 at the back view and returns to 1, because the linear ramp over-sharpens the last frames of an orbit that loops back to its start 2403.12008 §3. The 3D step fits an Instant-NGP NeRF to the generated views, extracts a mesh by marching cubes, and then fits a DMTet at full resolution. Plain score distillation with SV3D over-saturates and blurs, so the authors apply it only to views that the reference orbit does not see. The lighting is 24 spherical Gaussians under a white-light Lambertian model 2403.12008 §4.

The authors report 21.26 PSNR and LPIPS 0.08 for SV3D-p on GSO static orbits, against 18.34 PSNR for Stable Zero123, the best baseline in that table 2403.12008 §3. In a user study on 22 real images with 30 users, the static orbit was preferred over Zero123-XL, Stable Zero123, EscherNet and Free3D in 96%, 99%, 96% and 98% of comparisons 2403.12008 §3. On 50 GSO objects, the masked-SDS mesh reaches a Chamfer distance of 0.024, against 0.039 for Stable Zero123 2403.12008 §4. Sampling takes about 40 seconds per orbit 2403.12008 app. D. Card: SV3D.

Hashed multi-scale Gaussians with tracked actors (DriveSplat)

DriveSplat represents a driving scene as neural Gaussians attached to anchors. The static background uses a hashed multi-scale table rather than one resolution. Background points are split into near, mid and far regions by their depth along the scene's main axis, found by principal component analysis 2508.15376 §3.2 2508.15376 eq. 7 2508.15376 eq. 8. An anchor reads one level of the table, and it is rendered only if that level is no finer than the target for its camera distance 2508.15376 §3.2 2508.15376 eq. 10. Each tracked actor has a rigid pose per frame, and non-rigid actors add a learned anchor deformation that predicts new anchor positions from position, feature and time 2508.15376 §3.3. Depth and normal maps from pretrained models supervise the fit 2508.15376 §3.4.

The authors report novel-view PSNR 33.83 on Waymo, against 33.08 for AD-GS, the best baseline in their table, and 24.53 on KITTI, against 22.01 for Deform-GS 2508.15376 tab. 1 2508.15376 tab. 2. On a Waymo subset with mostly non-rigid actors, dynamic-region PSNR is 29.46 against 28.79 for OmniRe 2508.15376 tab. 3. Initialisation matters: random points give 29.64 PSNR, and SfM plus LiDAR points give 33.83 2508.15376 tab. 6. The text puts training at 68 minutes on one L20 GPU for 30,000 iterations, while the cost table lists 69.6 minutes 2508.15376 §4.2 2508.15376 tab. 5. Card: DriveSplat.

Densified COLMAP points and one-step diffusion repair (Hybrid Gaussian Splatting)

This entry is a challenge solution rather than a new model class. It fits 3D Gaussian splatting to the source frames, after re-estimating the point cloud with COLMAP: a sparse reconstruction, then dense fusion 2510.12308 §3.1 2510.12308 fig. 3. The authors say the initial point cloud matters most, and that other primitives, densification and priors did not beat standard 3DGS once it was initialised carefully 2510.12308 §3.1. The rendered view, together with the source frame nearest in pose, goes through a single-step enhancer derived from SD-Turbo and the public Difix checkpoint 2510.12308 §3.2 2510.12308 eq. 4. The enhancer is fine-tuned for 40k iterations at learning rate 2e-5 on 36,231 pairs of corrupted and clean frames. Corrupted frames are renders from 3DGS fitted to other clips, and dynamic objects are masked with Grounded SAM 2 2510.12308 §3.2 2510.12308 §4.

On the challenge sequences, 3DGS from dense COLMAP points reaches a target-view score of 0.346, against 0.319 for the challenge point cloud [arxiv:2510.12308 fig. 3]. On the authors' devset, the pretrained enhancer scores 0.423. Fine-tuning on the curated data raises that to 0.455, and high-resolution training to 0.462 2510.12308 tab. 1. The leaderboard score is 0.432, second of 28 submissions 2510.12308 §1. Iterative refinement, which feeds enhanced renders back in as pseudo views, improved the visuals but not the metric, so it was left out of the submission 2510.12308 §4 2510.12308 fig. 4. Card: Hybrid Gaussian Splatting.

Feed-forward 4D Gaussians from six cameras (ReconDrive)

ReconDrive is feed-forward: one network, and no per-scene fitting. Its backbone is VGGT, a 3D foundation model, kept frozen except for LoRA adapters of rank 8 and scale 32 2603.07552 §3.2. Tokens for two frames of six cameras pass through the backbone together. A centre head predicts a depth per pixel, which is unprojected with calibration into the ego frame, and an attribute head predicts rotation, scale, opacity and colour from the same tokens and the raw image 2603.07552 §3.2. Vehicles are segmented in the images, and each gets one velocity from the displacement of its box centre between the two frames 2603.07552 §3.3 2603.07552 eq. 3. The second frame's Gaussians are moved into the first frame's ego coordinates and shifted back in time, so a vehicle moves linearly inside the segment 2603.07552 §3.3 2603.07552 eq. 4. Training combines a rendering loss, a warped-view projection loss and a norm penalty on scale and opacity 2603.07552 §3.4 2603.07552 eq. 8.

The authors report reconstruction PSNR 32.66 on nuScenes, against 29.58 for PVG, the best per-scene baseline 2603.07552 tab. 1. Novel-view PSNR at 1 to 3 m is 23.99, against 23.73 for DeformableGS 2603.07552 tab. 1. A scene takes about 15 s, against 23 to 46 minutes for the per-scene methods in their setup 2603.07552 tab. 1 2603.07552 §4.2. A single-frame input reaches 23.54 PSNR against 23.99 for two frames 2603.07552 tab. 3. Card: ReconDrive.

Feed-forward human Gaussians with a motion field (Forge4D)

Forge4D takes videos of one person from several cameras whose poses are not known, and predicts dynamic 3D Gaussians in one forward pass [arxiv:2509.24209 §3.1]. Its backbone is VGGT, initialised from pretrained weights [arxiv:2509.24209 §4.1]. Training has three stages. The first gives static Gaussians with a metric gauge. The second adds a learnable state token that carries information from frame to frame, so the model does not need global attention over all frames 2509.24209 §3.2 2509.24209 §3.3. The third adds a motion head and a fusion step for times between frames 2509.24209 §3.4.

The motion head predicts a backward and a forward 3D motion for each view. No motion labels are used. A retargeting loss moves one frame's Gaussians to its neighbour and compares the render with that neighbour's render, and a flow loss matches predicted 2D flow with pseudo-labels from SEA-RAFT, with occluded pixels removed 2509.24209 §3.4 2509.24209 eq. 6 2509.24209 eq. 7. For a novel time, the Gaussians of the two neighbouring frames are moved a part of the way, those occluded in one frame are kept apart, and the rest are merged by a two-layer MLP 2509.24209 §3.4 2509.24209 eq. 8.

The authors report, with four input views on DNA-Rendering, novel-view PSNR 29.82 for the static model, against 26.12 for AnySplat, which takes no poses 2509.24209 tab. 1. For novel view and time at sampling rate 2, they report 29.04 PSNR, against 18.03 for L4GM with eight input frames 2509.24209 tab. 2. On an H200 the full model takes 224 ms per input frame pair, and interpolating ten steps adds about 1.5 ms [arxiv:2509.24209 tab. 6]. For the pose-free rows, the evaluation optimises the novel-view poses while the Gaussians stay fixed, which the authors say is not done at deployment 2509.24209 app. C. Dense motion is checked only on a synthetic set, MetaHuman4D 2509.24209 tab. 5. Card: Forge4D.

Level-of-detail tree of Gaussians for city-scale scenes (Hierarchical 3D Gaussians)

This variant targets very large outdoor captures, up to several kilometres and tens of thousands of images 2406.12080 §1. The scene is cut into chunks of 50 by 50 m for walking captures and 100 by 100 m for vehicle captures, and each chunk is trained on its own 2406.12080 §6. A coarse scaffold over the full data comes first. Each chunk is then trained with the 3DGS optimiser, with depth supervision and a densification rule based on the maximum screen-space gradient 2406.12080 §6.2 2406.12080 app. C. Card: Hierarchical 3D Gaussian Representation.

Each chunk's leaves are merged into a binary tree. A parent is a single 3D Gaussian whose mean, covariance and spherical harmonics are weighted averages of its children's, with weights set by opacity times the child's surface 2406.12080 §4.1 2406.12080 eq. 3 2406.12080 eq. 8. The tree is pruned so that no parent is barely larger than its children 2406.12080 §5.2, and its interior nodes are then optimised while the leaves stay fixed 2406.12080 §5.1. Chunks are consolidated into one scene-level structure after training 2406.12080 §6.3.

At render time, each view takes a cut through the tree. A node is in the cut when its projected size meets a target granularity in pixels and its parent's does not 2406.12080 §4.2 2406.12080 fig. 3. Across a level boundary, the child and parent are blended with a weight from their own and their parent's granularity, so the switch is smooth 2406.12080 §4.2 2406.12080 eq. 10. The paper gives a start value for the child opacities that makes the children blend to the parent's colour when the transition begins 2406.12080 eq. 12 2406.12080 fig. 4.

The authors report, on one chunk per scene, a PSNR of 26.62 on SmallCity for the full-leaf render, against 25.34 for 3DGS 2406.12080 tab. 2. Their optimised cut at 6 px reaches 26.29 PSNR and 110 FPS on the same scene, against 99 FPS for 3DGS 2406.12080 tab. 2. On BigCity the leaf render gives 23.10 PSNR against 21.48 for 3DGS 2406.12080 tab. 2. On Mill 19 they report 21.52 PSNR on Building against 20.93 for Mega-NeRF, and 3 hours on two V100s against 27 to 30 hours on eight for Mega-NeRF 2406.12080 tab. 3 2406.12080 §8.2. The paper compares with Mega-NeRF using poses from its release, which is not the same pipeline as the paper's own 2406.12080 §8.2. Its average over four scenes gives 24.98 PSNR for the full method and 25.10 with the depth term removed. The text says depth supervision does not improve per-image metrics, and its benefit is qualitative 2406.12080 tab. 6 2406.12080 §8.3.

The cost is memory and storage. The implementation uses 400 bytes per Gaussian, 69% more than 3DGS, and the authors give 284 bytes as the theoretical minimum 2406.12080 §8.2. Consolidated scenes take 6 to 88 GB on disk, about 68% more than non-hierarchical 3DGS files 2406.12080 §8.1. Rendering at the finest cut is slower than 3DGS, and the speed-up appears only at coarser cuts 2406.12080 tab. 4 2406.12080 §8.2.

Data and surveys

Objaverse-XL is a dataset, not a renderer. It collects 10.2M deduplicated 3D objects from five web sources, and renders each object from 12 random camera positions 2307.05663 §3.1 2307.05663 §3.2. Zero123-XL, a view-conditioned diffusion model trained on it, reaches 19.88 PSNR on Google Scanned Objects after alignment finetuning, against 18.23 before 2307.05663 tab. 2. PixelNeRF pretrained on the data also gains PSNR on DTU and ShapeNet 2307.05663 tab. 3. The data is not under one licence: each object keeps its own 2307.05663 app. C. The paper says Polycam objects are CC BY 4.0, but the repository README restricts them to academic, non-commercial use on request 2307.05663 §3.1 README.md. Do not reuse Polycam-derived objects until that is resolved. The filtering classifiers are not in the repository, so the filters cannot be rebuilt from the card (card).

The 3D Gaussian splatting survey proposes no method. It explains the forward renderer, which projects Gaussians, sorts them into 16×16 tiles and blends them front to back 2401.03890 §3.1, and the optimisation with density control 2401.03890 §3.2. It groups follow-up work into sparse inputs, memory, photorealism and applications such as SLAM, dynamic scenes and avatars 2401.03890 §4 2401.03890 §5. Its benchmark rows come from the original papers, under different settings 2401.03890 §6. Use it as a map of the field, and read the cited papers for values. The method itself is on Gaussian splatting.

When to use which

Implementation effort differs. KiloNeRF's paper says the speed depends on custom kernels, and that without a grouped or fused kernel much of the speed-up is lost 2103.13744 §3.5. K-Planes' paper describes a pure PyTorch implementation 2301.10241 §1. The card's code check finds that the official code imports tiny-cuda-nn for its MLPs, so plan for that dependency (card lists the detail). FreeNeRF's changes are a mask on the encoding and one extra loss term 2303.07418 eq. 4 2303.07418 eq. 6. The KiloNeRF, FreeNeRF and DietNeRF repositories all lack a LICENSE file (KiloNeRF, FreeNeRF, DietNeRF), so implement from the papers rather than copying their code. DietNeRF's paper writes the semantic term as a positive dot product, but its code minimises the negative cosine. Implement the negative cosine, as the card advises. The BARF repository is MIT-licensed (card), and its window must be scaled to the run length: the code takes it as fractions of max_iter. Splatter Image needs a Gaussian rasteriser, and its repository mixes BSD-3 code with files adapted from non-commercial sources, so the card advises implementing the U-Net and renderer from the paper's description. Its paper's FiLM and camera embedding also differ in detail from what its code does. SynSin's repository has a mixed licence that includes non-commercial components, so implement its renderer and networks from the paper (card). The ST-NeRF repository has no LICENSE file and releases only part of the pipeline. Its scene-parsing stage is missing, and its training code omits the paper's layered loss and uses a different learning-rate schedule 2104.14786 §4.4 2104.14786 eq. 8. The repository has no code licence, so the card cites only its described behaviour. Implement from the paper, and check the settings against the code's values. The reality-check repository is Apache-2.0, according to its card. The card records that its code computes mPSNR differently from the paper's definition, and that its co-visibility threshold uses a strict inequality with integer rounding. Copy the threshold from the code, not from the prose 2210.13445 app. B. A masked LPIPS score can come out artificially low when the mask is small 2210.13445 app. B.

The three driving cards differ in how far their code can be used. The DriveSplat repository has no LICENSE file, so its code is unlicensed and the method should be built from the paper (card). The Hybrid entry has no official code, and its enhancer starts from a public checkpoint whose licence the paper does not state (card). ReconDrive's README names Apache-2.0, but the repository has no LICENSE file. Its release has the base model and two checkpoints, but not the stage-1 training recipe, and the code needs annotated velocities and ego poses (card).

The Hierarchical 3D Gaussians release is licensed for non-commercial research only, so its method should be built from the paper. The paper leaves several values out: the hierarchy-optimisation loss, the learning rates for that stage, the densification thresholds and the depth-loss weight. Its falloff clamp and ellipsoid-surface definition are also unclear (card lists the detail).

DONeRF's repository is CC BY-NC 4.0 and FLARE's is CC BY-NC-SA 4.0, so both methods should be built from their papers (card, card). DONeRF's equation 7 divides by floor(K/2), which is zero when the neighbourhood is one pixel. The paper does not say how that unfiltered case is computed, so treat it as no image-space filter and say so (card lists the detail). FLARE's loss weights, the confidence penalty, the batch size and the encoder are not stated, so its training recipe cannot be reproduced from the paper alone (card).

FSGS, SV3D and Forge4D also differ in how far their code can be used. FSGS's repository carries the Gaussian-Splatting License, which allows only non-commercial research, so implement FSGS from the paper (card). SV3D's code is MIT, but its released weights fall under a non-commercial licence, so commercial use needs rights to the weights of its own (card). Forge4D's code is Apache-2.0, but no checkpoints are released, and as shipped its stage-3 training script exits after one evaluation, before the optimisation loop, so the training recipe has to be built from the paper and its appendix (card).

DynamicNeRF's paper gives loss weights and an iteration count that its released configs do not match, and it names TensorFlow where the release is PyTorch. Pick one weight set and say which (card lists the differences).

Several of the later cards add their own choices. pixelNeRF's official code is BSD-2-Clause, and its feature residual enters three of the five MLP blocks, while the paper's text says each block (card lists the detail). Match the code if you want the reported numbers, and say which you chose. DS-NeRF's card records an MIT licence, but the released default loss is the plain squared error, not the Gaussian form that gives the headline numbers (card). Plenoxels' code needs CUDA (card). SparseGS' code carries a non-commercial research licence, so implement its method from the paper (card). HexPlane's paper says only "tiny MLP" for colour. The release uses a 27-wide feature, a three-layer MLP of width 128 and a ReLU density in its Plenoptic config, with a softplus default elsewhere config.py config.py nv3d.yaml. The Plenoptic Video dataset is CC BY-NC 4.0, so this page quotes only the published numbers (card lists the detail). PSHuman's code is MIT, but its texture step differs from the paper, and its SMPL-X body model files are downloaded separately under their own terms (card).

Implementation sketch

The core of the NeRF-style variants is shared: sample points along the ray, query the field, composite. Plenoxels' field is a trilinear lookup in a grid rather than an MLP 2112.05131 §3.3. SparseGS, FSGS, FLARE, DriveSplat, Hybrid, ReconDrive, Forge4D and Hierarchical 3D Gaussians splat Gaussians instead of sampling a field, so they do not fit this sketch 2312.00206 §3.1 2312.00451 §3.2 2502.12138 §3.3 2508.15376 §3.3 2510.12308 §3.1 2603.07552 §3.1 2509.24209 §3.2 2406.12080 §4.2. DONeRF keeps the loop, but its samples come from the oracle rather than from stratified sampling 2103.03231 §3. ST-NeRF runs the same loop once per layer, inside that layer's box, and merges the samples of all layers before compositing 2104.14786 §4.3. DynamicNeRF runs the loop over its static and dynamic fields and blends them per sample 2105.06468 §3.5. The variants differ in how the query is encoded, whether samples are filtered or placed first, and the training loss. FreeNeRF's masked encoding would replace the query encoding, and its occlusion penalty is added to the loss. DietNeRF keeps the loop and adds a semantic term to the loss every K iterations 2104.00677 alg. 1. BARF keeps the loop but adds the camera poses to the parameters that the loss updates 2104.06405 §3.2. Splatter Image has no ray sampling. It runs the image network once per input view, then renders the predicted Gaussians at the target pose 2312.13150 §3.2 2312.13150 §3.4. PSHuman has neither ray sampling nor a field. Its diffusion model produces the views, and the mesh is fitted to them 2409.10141 §3.2. SynSin also has no ray sampling. It splats a point cloud and refines the result with a 2D network 1912.08804 §3.2 1912.08804 §3.3.

# Render one ray  [arxiv:2103.13744 §3.4] [arxiv:2301.10241 eq. 9]
# Input:  ray, field (points, dirs) -> (rgb, sigma), occupancy grid or None
# Output: pixel colour (3,)

def render_ray(ray, field, occupancy=None):
    # K = 384 stratified samples per ray  [arxiv:2103.13744 §4.1]
    points, deltas = stratified_samples(ray, 384)
    # KiloNeRF only: drop samples in empty cells  [arxiv:2103.13744 §3.4]
    if occupancy is not None:
        points, deltas = keep_occupied(points, deltas, occupancy)
    # KiloNeRF: route each point to its cell's MLP  [arxiv:2103.13744 eq. 6]
    # K-Planes: read each plane, multiply the readings  [arxiv:2301.10241 eq. 2]
    # HexPlane: read six planes, multiply pairs, concatenate  [arxiv:2301.09632 eq. 5]
    # Plenoxels: trilinear lookup of stored corners, no MLP  [arxiv:2112.05131 §3.3]
    rgb, sigma = field(points, ray.direction)
    # alpha-composite with transmittance  [arxiv:2103.13744 eq. 2-4]
    return composite(rgb, sigma, deltas)

Contradictions

Within FreeNeRF, two numbers disagree:

• Schedule end for 9 views: the main text says 20% of training 2303.07418 §4.1. The appendix says 50% 2303.07418 app. A.1. The paper does not say which is intended.
• DTU, 3 views, object PSNR: the main table gives 19.92 2303.07418 tab. 2. The ablation at the same setting gives 20.20 at the default batch size 2303.07418 tab. 5. The paper does not explain the gap.

Across cards, the numbers are not comparable, and the cards say so:

• Synthetic-NeRF: KiloNeRF reports its NeRF baseline at 31.01 PSNR 2103.13744 tab. 1. K-Planes reports TensoRF at 33.14 and I-NGP at 33.18 on the same benchmark, and its own explicit model at 32.21 2301.10241 tab. 3. The baselines and evaluation details differ between the papers, so these rows do not rank the methods.
• Blender and few views: FreeNeRF's Blender table uses 8 training views 2303.07418 tab. 1. The Synthetic-NeRF rows of KiloNeRF and K-Planes are not stated to use that setup 2103.13744 tab. 1 2301.10241 tab. 3, so FreeNeRF's PSNR values are not comparable with theirs.
• SSIM: K-Planes notes that prior work uses different SSIM implementations, and that some values are higher than others for that reason 2301.10241 §4. Do not compare SSIM across papers without checking.
• Speed: KiloNeRF reports render times and K-Planes reports training times on different hardware 2103.13744 tab. 1 2301.10241 tab. 3. HexPlane's 112× and 672× training speed-ups compare its own runs with the published DyNeRF run 2301.09632 tab. 1. FreeNeRF's overhead is a ratio against its own baseline 2303.07418 tab. 4. Splatter Image's timings for NeRFDiff and FE-NVS are those authors' own figures, on other hardware 2312.13150 tab. 7 2312.13150 §4.2.
• NeRF baseline on Synthetic-NeRF: Plenoxels reports JAXNeRF at 31.85 PSNR on the eight scenes, from its authors' rerun 2112.05131 tab. 2 2112.05131 app. D. Its own ablation and KiloNeRF's table give NeRF at 31.01 on related setups 2112.05131 tab. 3 2103.13744 tab. 1. BARF's reference NeRF, trained with true poses on its Blender scenes, reaches 29.40 2104.06405 tab. 2, and DietNeRF's 100-view NeRF reaches 31.153 2104.00677 tab. 1. The baseline number moves with the protocol, so do not take the gap as a measured speed or quality difference.
• RealEstate10K input count: SynSin's single-view 22.31 PSNR [arxiv:1912.08804 tab. 1] and StereoMag's two-view 25.34 1912.08804 tab. 2 use different numbers of input views, so the gap does not rank the methods.
• DL3DV input count: FLARE's 23.33 dB uses eight views, and MVSplat's 22.08 dB uses its two closest views 2502.12138 tab. 3. The gap mixes the method and the input count, so it does not rank the methods.
• pixelNeRF on DTU: DS-NeRF's table lists pixelNeRF-DTU at 19.3 PSNR with three views 2107.02791 tab. 2. DietNeRF's table gives 15.55 PSNR for pixelNeRF with one source view, copied from the pixelNeRF paper 2104.00677 tab. 2, and Objaverse-XL's base model gives 15.32 for the same single-view setting 2307.05663 tab. 3. These are different view counts or training runs, so the numbers are not comparable. pixelNeRF's own DTU results are a plot 2012.02190 §5.3.
• FSGS in other papers: SparseGS reports FSGS at 19.88 PSNR on LLFF 2312.00206 tab. 2 and 19.31 on Mip-NeRF360 with 12 views 2312.00206 tab. 1. FSGS's own table gives 20.31 on LLFF at 1/8 resolution with three views, and 23.70 on Mip-NeRF360 with 24 views 2312.00451 tab. 1 2312.00451 tab. 2. The cards do not say that the protocols match, so these rows are not a measured gap.

Within DONeRF, the speed-up claims disagree. The abstract gives up to 48×, the introduction 24× to 98×, §6.3 15× to 78×, and the table caption 48× to 78× fewer FLOP 2103.03231 §1 2103.03231 §6.3 2103.03231 tab. 2. The FLOP column reproduces the §6.3 range. These are FLOP ratios, not measured speed-ups, and the paper reports no NeRF wall-clock time 2103.03231 tab. 2.

Within FLARE, the text claims more than the ETH3D table shows. It says FLARE "achieves higher performance" than DUSt3R, MASt3R and Spann3R 2502.12138 §4.2, but MASt3R has the lower overall error on ETH3D, 0.3522 against 0.4978 2502.12138 tab. 2. The text also says the model works up to 25 views, while the only 25-view evidence is one DTU table 2502.12138 §4 2502.12138 tab. 6. The paper's camera latents are described once per image and once shared by the source views, and the paper does not say which the reported numbers use 2502.12138 §3.1 2502.12138 app. A.

Within Splatter Image, two claims differ from its own table:

• Training speed-up: the text says training is 246 times faster than implicit methods 2312.13150 §4.2. The Forward column of the speed table gives about 225 times against PixelNeRF and about 295 times against VisionNeRF 2312.13150 tab. 7.
• CO3D speed-up: the text says 1,000 times 2312.13150 §4.1.1. The caption of its figure says 100 times 2312.13150 fig. 4.

Splatter Image's cars numbers use one view with relative poses on ShapeNet-SRN 2312.13150 tab. 1, while FreeNeRF's Blender rows use 8 training views 2303.07418 tab. 1. The setups differ, so these PSNR values are not compared with each other.

Within HexPlane, the text and its table differ on D-NeRF. The text says HexPlane "distinctly outperforms" the other methods there, but TiNeuVox-B has the higher PSNR, 32.67 against 31.04 2301.09632 §4.1 2301.09632 tab. 2.

Within SparseGS, the text and its table give different gains for one step:

• Depth-warping gain: the text says the UVR step (SDS and warping) adds 0.33 dB on average 2312.00206 §4.3. Table 3 gives 0.23 dB for that row, from 18.86 to 19.09 2312.00206 tab. 3. The paper does not explain the difference.

Within SynSin, the text, the tables and the appendix differ:

• Learning rates: the text gives 0.0001 for the generator and 0.01 for the discriminator 1912.08804 §3.4. The appendix gives 0.001 for "ours" 1912.08804 app. C. The paper does not say which applies, and the released scripts do not use either set as written (card).
• Best on every metric: the table caption says the full model beats the baselines on all metrics 1912.08804 tab. 1. The small fine-tune ablation has higher PSNR on Matterport3D (21.14 against 20.91) and on Replica (22.36 against 21.94) 1912.08804 tab. 1.
• KITTI: the appendix says SynSin is similar or better on all metrics against ContView, but ContView has higher PSNR (16.90 against 16.70) and SSIM (0.54 against 0.52) 1912.08804 app. A.

Within ST-NeRF, two claims differ from its own table:

• Best on every metric: the caption of its main table says the method is best on all four metrics 2104.14786 tab. 1. The text says it outperforms all baselines in PSNR, SSIM and MAE 2104.14786 §6.2. In the table, NeRF-T has the higher SSIM (0.9243 vs 0.9203), and ST-NeRF has the highest MAE, 0.1178 against 0.0219 for NeRF-T, where lower is better.
• Camera count: the synthetic dataset is described with 36 virtual cameras 2104.14786 §6, and the view ablation says 17 cameras 2104.14786 §6.3.

Within Dynamic View Synthesis from Dynamic Monocular Video, the paper and its released README differ. The paper reports average PSNR 24.74 and LPIPS 0.118 2105.06468 tab. 1. The README reports PSNR 26.10 and LPIPS 0.082 on the same seven sequences, and says the paper's numbers come from the TensorFlow code README.md. Which of the two an implementer should match is not settled by the cards. Its training frames also come from a different camera each 2105.06468 §4.1. The reality-check paper describes training on frames that alternate between cameras as camera teleportation 2210.13445 §1, and its experiments use other sequences 2210.13445 §5. Neither card says whether that affects the Dynamic Scene numbers.

Across the dynamic results, the setups differ. K-Planes reports monocular dynamic numbers on the D-NeRF synthetic scenes 2301.10241 tab. 3. HexPlane reports 31.04 PSNR on D-NeRF, with no deformation field 2301.09632 tab. 2. Its Plenoptic Video numbers come from multi-view capture, and DyNeRF's are copied from that paper 2301.09632 §4.1. The reality-check paper's experiments use the Nerfies-HyperNeRF sequences and its iPhone dataset 2210.13445 §5, so it does not say whether the D-NeRF numbers are inflated. ST-NeRF's PSNR comes from 16 synchronised cameras, not from a monocular video 2104.14786 §3. None of these dynamic numbers should be compared directly. The driving cards use Waymo, KITTI, nuScenes and a challenge set, so their numbers are not comparable with each other either 2508.15376 tab. 1 2603.07552 tab. 1 2510.12308 fig. 3.

Within DriveSplat, the text claims more than its tables show. It says DriveSplat has the best reconstruction SSIM and LPIPS on Waymo 2508.15376 §4.3, but OmniRe's SSIM is 0.939 against 0.937 2508.15376 tab. 1. On KITTI the text says DriveSplat beats every baseline in reconstruction, but OmniRe's PSNR is 28.68 against 28.59 2508.15376 tab. 2. The number of sequences in the non-rigid comparison is two in §4.3 and four in §4.6 [arxiv:2508.15376 §4.3] 2508.15376 §4.6. Its PSNR also varies by table, 33.83 and 30.96, because the subsets differ 2508.15376 tab. 1 2508.15376 tab. 9.

Within ReconDrive, two statements disagree. The paper says dynamic motion is regressed 2603.07552 §3.1, but computes it from annotated boxes 2603.07552 §3.3. The abstract and §4.2 claim the best result on eight of nine metrics, but the nine are never listed 2603.07552 §4.2. The text calls DeformableGS the LPIPS leader for reconstruction 2603.07552 §4.2, while OmniRe's reconstruction LPIPS is 0.1577 in Table 1 2603.07552 tab. 1.

Within Hybrid Gaussian Splatting, the point counts disagree. The text gives about 1e3 points for the challenge sequences and about 1e5 for COLMAP sparse 2510.12308 §3.1. The figure gives about 30K and 180K 2510.12308 fig. 3.

PSHuman's geometry numbers use ground-truth SMPL-X and CAPE or THuman2.1 scans. They are not comparable with the reconstruction numbers of the other cards 2409.10141 tab. 1.

Within PSHuman, the paper gives two step counts for the remeshing. Section 3.2 says 700 steps 2409.10141 §3.2, and section 4 gives 100 steps for the geometry stage 2409.10141 §4. The release runs 700 remeshing steps and 100 SMPL-X alignment steps remesh.yaml reconstruct.py. The texture step is also described differently in the paper and the code: per-vertex optimisation in the paper 2409.10141 eq. 9, and projection of the views onto the mesh in the code reconstruct.py.

Within FSGS, the text gives a gain of 0.40 dB over FreeNeRF on Blender, but Table 3 gives 24.64 minus 24.26, which is 0.38 dB 2312.00451 §4.2 2312.00451 tab. 3. The depth prior is named as DPT in the main text and as a MiDaS model in the appendix, and the inverse depth that the appendix model gives is compared with rendered z-depth 2312.00451 §3.3 2312.00451 app. Training.

Within SV3D, the text says that both SV3D-c and SV3D-p beat SV3D-u on static orbits. The GSO static table gives SV3D-c 20.56 PSNR against 21.14 for SV3D-u, and only SV3D-p is higher 2403.12008 §3. The paper says the sampler is a deterministic DDIM, while the released code uses an Euler sampler with churn set to zero 2403.12008 app. D sampling.py.

Within Hierarchical 3D Gaussians, the falloff clamp has two values. The text clamps the blend to 1 2406.12080 §4.1, and the rasteriser clamps at 0.99 2406.12080 §7.3. Its image counts also differ: BigCity has 38,235 images in the table 2406.12080 tab. 1 and 28,145 in a figure caption 2406.12080 fig. 8.

Within Forge4D, the static model and the novel-time model are not the same network: the static comparison turns off the state tokens and state attention 2509.24209 app. C. The paper gives H200 for its speed figures in section 4.3 and H20 in the appendix 2509.24209 §4.3 2509.24209 app. E.

Papers

• KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs: a grid of tiny per-cell MLPs, distilled from a NeRF teacher, for fast rendering of bounded scenes.
• K-Planes: Explicit Radiance Fields in Space, Time, and Appearance: factorised 2D planes multiplied together, for static, dynamic and unbounded scenes with short training.
• HexPlane: A Fast Representation for Dynamic Scenes: six planes, three spatial and three spatio-temporal, multiplied in pairs, for multi-view dynamic scenes.
• FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization: a frequency schedule on the positional encoding and a near-camera density penalty, for NeRF with few input views.
• Splatter Image: Ultra-Fast Single-View 3D Reconstruction: one Gaussian per pixel predicted by a U-Net from one image, rendered by splatting, for object-centric reconstruction.
• SynSin: End-to-end View Synthesis from a Single Image: a feature point cloud and depth map from one photo, splatted and refined with a GAN, trained without depth labels.
• Editable Free-viewpoint Video Using a Layered Neural Representation: one space-time radiance field per performer and background, composited per ray, for editable multi-performer scenes from 16 cameras.
• Dynamic View Synthesis from Dynamic Monocular Video: a static NeRF on unmasked pixels and a scene-flow dynamic NeRF, trained on one moving video with object masks.
• Monocular Dynamic View Synthesis: A Reality Check: co-visibility masks, effective multi-view factors and an iPhone dataset for judging monocular dynamic benchmarks.
• pixelNeRF: Neural Radiance Fields from One or Few Images: a NeRF conditioned on pixel-aligned features of posed inputs, shared across scenes and rendered feed-forward.
• Depth-supervised NeRF: Fewer Views and Faster Training for Free: a depth term from COLMAP keypoints, for NeRF with few views and faster convergence.
• Plenoxels: Radiance Fields without Neural Networks: a sparse voxel grid of opacity and spherical harmonics, optimised with volume rendering and no network.
• SparseGS: Sparse View Synthesis using 3D Gaussian Splatting: depth priors, diffusion and warped views for 3D Gaussian splatting from 3 to 24 photos.
• PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing: six generated body views and a face view, carved into a textured mesh from one photo of a person.
• DriveSplat: Unified Neural Gaussian Reconstruction for Dynamic Driving Scenes: neural Gaussians with a hashed multi-scale background and tracked actors, for dynamic driving logs.
• Hybrid Gaussian Splatting for Novel Urban View Synthesis: COLMAP-densified 3DGS with a one-step diffusion enhancer, for lane-shifted street-view targets.
• ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction: VGGT with LoRA and Gaussian heads, predicting 4D Gaussians from six cameras in one pass.
• BARF: Bundle-Adjusting Neural Radiance Fields: a NeRF fitted jointly with its camera poses, using a coarse-to-fine positional encoding.
• Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis: a NeRF with a frozen CLIP consistency loss on unseen poses, for few posed photos.
• Objaverse-XL: A Universe of 10M+ 3D Objects: a web-crawled dataset of 10.2M 3D objects rendered from 12 views, with Zero123-XL and PixelNeRF trained on it.
• A Survey on 3D Gaussian Splatting: a survey of 3DGS rendering, optimisation and applications, with benchmark numbers taken from other papers.
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting: proximity-guided Gaussian unpooling and pseudo views with a monocular depth correlation loss, for 3D Gaussian splatting from three photos.
• Hierarchical 3D Gaussians: a tree of merged Gaussians, cut per view by projected size and trained in chunks, for city-scale street captures.
• SV3D: a video diffusion model that draws an orbit of 21 views of an object from one image, then fits a mesh.
• Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos: a feed-forward model that predicts dynamic human Gaussians and a motion field from four uncalibrated videos, for novel views and times.
• DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks: a depth oracle picks two to sixteen samples per ray for a NeRF-style shading network, for real-time static scenes.
• FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views: one transformer that predicts poses, point maps and Gaussians from two to eight uncalibrated photos.