A Survey on 3D Gaussian Splatting

In plain words

Earlier neural 3D scene models (NeRF) answer each pixel by asking a neural network many questions along a camera ray, which is slow. 3D Gaussian splatting (3DGS) stores the scene as a large set of small, soft 3D blobs instead. Each blob has a position, a size and orientation, a transparency, and a colour that can change with the viewing angle. To draw an image, the blobs are flattened onto the picture and blended front to back, a step that graphics hardware can run in parallel. The blobs are learned by adjusting them until their renders match the input photos. New blobs are added where the scene looks under-covered, and blobs that do nothing useful are removed.

This survey explains how the method works, groups the follow-up research into themes, and compares reported results across tasks such as SLAM, dynamic scenes, human avatars and surgical video.

TL;DR

• 3DGS represents a scene as an explicit set of learnable 3D Gaussians, rendered by projection and alpha blending on 16×16 tiles §3.1.
• It is trained with an L1 plus D-SSIM loss and density control (cloning, splitting, pruning) §3.2.
• The survey proposes no new algorithm. Its benchmark numbers are author-reported from the original papers, under different settings §6.
• Use it as a map of the field. For implementation values, read the original 3DGS paper and the follow-up papers, not this survey.

Problem

Neural radiance fields such as NeRF are costly to render: each pixel needs many sampled points along a ray, which makes real-time rendering at high resolution hard, especially on limited hardware §2 §3.1. Implicit networks are also hard to edit, because changing weights does not map to a clear change in geometry or appearance §1.

3DGS aims for an explicit, parallel representation that renders in real time while keeping the fitting power of volumetric radiance fields §1 §2.

Assumptions: the scene is learned from multi-view images §2. Initial points come from structure-from-motion (SfM) or random sampling §3.2.2. The survey notes that SfM is error-prone on complex scenes §4.4. It does not state where the camera poses come from.

Method

The survey describes two parts: how a view is rendered from learned Gaussians §3.1, and how the Gaussians are optimized §3.2. The steps below are the survey's description, not code from the original repository.

Scene representation

Each 3D Gaussian has a centre μ, an opacity α, a 3D covariance Σ and a colour c §3.1. The colour is stored as spherical-harmonics coefficients, so it varies with view direction §3.1.

The covariance is not optimized directly. It is built from a quaternion q (rotation) and a 3-vector s (scale), so it stays positive semi-definite §3.2.1:

\bm{\Sigma} = \bm{R}\bm{S}\bm{S}^{\top}\bm{R}^{\top} eq. 8

where R comes from q and S = diag(s) §3.2.1.

Forward rendering

1. Frustum culling. Gaussians outside the camera frustum are dropped §3.1.
2. Projection to 2D. The 2D covariance is Σ' = J W Σ Wᵀ Jᵀ eq. 4, where W is the viewing transform and J the Jacobian of an affine approximation of the projection, using the first two Taylor terms §3.1.
3. Tiles. The image is split into non-overlapping 16×16 tiles. A Gaussian is copied once for each tile it touches, and each copy gets a tile ID §3.1.
4. Sorting. Tile ID and depth are packed into one key, so a single sort orders all copies for blending §3.1.
5. Alpha blending. Per pixel, the colour is C = Σₙ cₙ α'ₙ Πⱼ₌₁ⁿ⁻¹ (1 − α'ⱼ) eq. 5. The effective opacity is α'ₙ = αₙ · exp(−½ (x' − μ'ₙ)ᵀ Σ'ₙ⁻¹ (x' − μ'ₙ)) eq. 6. Tiles and pixels are independent, so this runs in parallel §3.1.

The survey presents the per-pixel sorted-list version (eqs. 5–6) as the simpler form of the tile-parallel renderer §3.1.

# Render one view from learned 3D Gaussians  [§3.1] [fig. 2]
# Input:  gaussians with centre, covariance, opacity, SH colour
# Output: image (H, W, 3)

def render(gaussians, camera):
    # Drop Gaussians outside the view frustum  [§3.1]
    visible = frustum_cull(gaussians, camera)

    # Project to 2D: Sigma' = J W Sigma W^T J^T  [eq. 4]
    means_2d, covs_2d, depths = project(visible, camera)

    # Copy each Gaussian once per 16x16 tile it touches  [§3.1]
    # Pack (tile id, depth) into one sort key  [§3.1]
    copies = duplicate_per_tile(visible, means_2d, covs_2d, depths)
    copies = sort_by_key(copies)

    # Tiles and pixels are independent, so this can run in parallel  [§3.1]
    # Blend the sorted list front to back  [eq. 5] [eq. 6]
    image = zeros(H, W, 3)
    for tile in tiles:
        for pixel in tile:
            image[pixel] = alpha_blend(copies[tile], pixel)

    return image

Optimization

The loss combines L1 and D-SSIM eq. 7:

\mathcal{L} = (1-\lambda)\mathcal{L}_1 + \lambda\mathcal{L}_{\text{D-SSIM}} eq. 7

with λ ∈ [0, 1] and L_D-SSIM = 1 − SSIM §3.2.1. Most properties are updated by backpropagation. For rotation and scale, the survey says the gradients are derived directly, rather than by automatic differentiation through the covariance chain §3.2.1.

Density control runs during training §3.2:

• Initialization. Start from sparse SfM points or random points §3.2.2.
• Densification. At regular intervals, Gaussians with a large view-space positional gradient (above a threshold) are either cloned, when small and in under-reconstructed areas, or split into two smaller Gaussians, when large and in over-reconstructed areas §3.2.2.
• Pruning. Gaussians with α below a threshold, or that are too large in world or view space, are removed. Opacity is also set close to zero after a set iteration, to stop density building up near input cameras §3.2.2.

# Fit Gaussians to posed training images  [§3.2]
# Input:  views (camera, ground-truth image), initial points
# Output: optimized Gaussians

def fit(views, initial_points, num_steps):
    # Initialise from SfM points or at random  [§3.2.2]
    gaussians = init_gaussians(initial_points)

    for step in range(num_steps):
        # Choosing a view: sampling rule not stated in the survey
        view = pick_view(views)
        rendered = render(gaussians, view.camera)

        # Loss = (1 - w) * L1 + w * (1 - SSIM)  [eq. 7]
        loss = (1 - ssim_weight) * l1(rendered, view.image)
        loss = loss + ssim_weight * (1 - ssim(rendered, view.image))

        # Backpropagate into centre, opacity, colour, q and s  [§3.2.1]
        gaussians = gradient_step(gaussians, loss)

        # Densify and prune at regular intervals  [§3.2.2]
        if densify_now(step):
            gaussians = clone_or_split(gaussians)
            gaussians = prune(gaussians)

    return gaussians

Directions covered by the survey

These are summarised in tab. S1 with representative works.

• Sparse input (§4.1): depth-based regularization, or feed-forward models that predict Gaussians from one or few images tab. S2.
• Memory (§4.2): fewer Gaussians (pruning, anchors) and attribute compression (codebooks, quantization). The survey reports storage reductions of often 10–20× §4.2.
• Photorealism (§4.3): aliasing, reflective and transparent objects, and blur.
• Optimization (§4.4): extra regularization, better densification, and COLMAP-free joint pose estimation.
• Extra properties (§4.5): language or semantic features and 4D (spatio-temporal) Gaussians.
• Hybrid representations (§4.6): deformation fields and grids alongside Gaussians.
• Rendering (§4.7): ray-tracing alternatives and order-independent transparency.
• Kernels (§4.8): surface-aligned primitives and non-Gaussian kernels.

Applications (§5): robotics and SLAM (§5.1), dynamic scenes (§5.2), generation and editing (§5.3), avatars (§5.4), endoscopy (§5.5), large-scale scenes (§5.6), and physics (§5.7).

Hyperparameters

The survey states few values. Most of its method description has no numbers, so the table lists the unstated ones too.

Results

Author-reported. The "This paper" column is the best or highlighted 3DGS-based method in the survey's own tables. The numbers come from the original works, not from runs by the survey authors. Setups: Replica, 8 scenes for SLAM and mapping tab. S4 tab. S5; D-NeRF synthetic dynamic scenes tab. S6; ZJU-MoCap, 6 subjects tab. S7; EndoNeRF, 2 surgical sequences tab. S8.

Two more reference points. Plain 3DGS scores 23.19 PSNR on D-NeRF, which the survey says fails on dynamic scenes; this figure is from another paper tab. S6. Qualitative trade-offs for sparse inputs are in tab. S2, with no numbers.

Paper vs code

No official code.

The survey links two GitHub repositories of its first author: guikunchen/Awesome3DGS, a paper list, and guikunchen/3DGS-Benchmarks, a comparison collection that links the original codebases §1 §6. They are a reading list and benchmark notes, not method code, so no code field is set. The survey's rendering description refers to the original 3DGS implementation §3.1; that is a different paper and is not cited as this card's code. Because this is a survey, no GitHub search for a method repository was run.

Pitfalls

• Survey, not method paper. The card explains 3DGS as the survey describes it. For values such as learning rates, densification thresholds and SH degree, the survey says "not stated". Take them from the original 3DGS paper and the cited follow-up work.
• Text and table disagree on a headline percentage. The localization text says SplaTAM cuts error "~50%" from 0.52 cm to 0.36 cm §6.1. The reduction is (0.52 − 0.36) / 0.52 ≈ 31%.
• Speed claim with no named baseline. The surgical text says EndoGaussian gives "approximately 200-fold" speed-up at 10% of the GPU resources §6.5. The table gives 195.09 FPS against 0.91 FPS for LerPlane-9k (≈214×) and 0.04 FPS for EndoNeRF and EndoSurf (≈4,900×). Memory 2 GB vs 20 GB is 10%, which matches LerPlane-9k tab. S8.
• "Clear advantage" in localization is not uniform. The 3DGS rows average between 0.36 cm (SplaTAM) and 3.27 cm (Gaussian-SLAM) in tab. S4. The worst of them is behind NICE-SLAM (1.06) and Point-SLAM (0.52).
• Cross-paper numbers. Each row comes from a different paper with its own data splits and settings. The survey says results vary with densification schedules and pruning resets §6. Several baselines are copied from other papers, e.g. D-NeRF 3DGS from [wu20234d] and EndoNeRF non-GS rows from [liu2024endogaussian] tab. S6 tab. S8.
• FPS hardware. The mapping table measures FPS on an NVIDIA 4090 tab. S5. The surgical table cites the same hardware, taken from EndoGaussian tab. S8.
• Memory is compared only in one table. The survey says footprint depends on the scene §6. Only the surgical table gives memory, so there is no cross-method memory comparison for SLAM or rendering tab. S8.
• Ordering in the renderer. The survey says tile ID and depth are packed into one sort key. The bit layout is not stated.
• Directions are sketched, not specified. Each direction in §4 and §5 gets a paragraph or two, with no equations or settings. Read the cited paper before implementing any of them.
• Tables are in the Supplementary Material. The performance tables are numbered S4–S8 in the source, which is how the survey cites them in §6.
• Version. The card follows arXiv v9 (updated 2026-04-09). Other versions may have different numbers.

Lineage

• Builds on: neural radiance fields and their volume-rendering integral (NeRF, Mildenhall et al.) §2 eq. 3; point-based and EWA splatting (Zwicker et al., 2001) §2. No cards on this site yet.
• Original 3DGS paper: Kerbl et al., cited as [kerbl20233d], titled "3D Gaussian Splatting for Real-Time Radiance Field Rendering" in the survey's bibliography. Not summarised here.
• Compared against: NeRF-based SLAM (NICE-SLAM, Point-SLAM, ESLAM) tab. S4 tab. S5; dynamic NeRFs (D-NeRF, FFDNeRF, HexPlane) tab. S6; surgical NeRFs (EndoNeRF, EndoSurf, LerPlane) tab. S8.
• Superseded by: not stated in this card.