--- kind: concept slug: empty-space-skipping title: Empty space skipping summary: >- Empty space skipping evaluates no network samples where a precomputed map marks the space as empty. KiloNeRF builds an occupancy grid from a trained NeRF teacher and pairs it with early ray termination, reporting a 71x speed-up on Lego. DONeRF avoids the grid: a depth oracle places 2 to 16 samples per ray near likely surfaces, and needs training-time depth. Both results are author-reported. tags: [3d, rendering, inference-speed, distillation, sample-placement] papers: ["2103.13744", "2103.03231"] related: [knowledge-distillation, neural-radiance-field, novel-view-synthesis] status: draft updated_at: 2026-11-01 --- ## Overview When a renderer samples points along each ray, most of them often lie in empty air and contribute nothing to the pixel. Empty space skipping (ESS) stores a coarse map of which regions of the scene are occupied, and does not evaluate samples that fall into regions marked empty. Only the remaining samples reach the network [arxiv:2103.13744 §3.4]. The map is a uniform occupancy grid with one bit per cell. Its resolution is 16 times the resolution of the tiny-network grid, so at most 256 cells per axis [arxiv:2103.13744 §3.4] [arxiv:2103.13744 §4.1]. Two cards cover this concept. KiloNeRF is the grid-based variant. DONeRF reaches the same goal, fewer network evaluations on empty space, without an occupancy grid: it places a few samples near surfaces that a learned oracle predicts. The paper names NSVF's self-pruning voxels as ESS-style work [arxiv:2103.03231 §2]; NSVF's paper is at [arXiv:2007.11571](https://arxiv.org/abs/2007.11571). The KiloNeRF paper uses ESS in two places: during fine-tuning and during inference [arxiv:2103.13744 §3.4]. It pairs ESS with early ray termination (ERT), which stops sampling a ray once its remaining transmittance drops below a threshold. ERT applies at inference only [arxiv:2103.13744 §3.4]. NSVF, the baseline the paper compares against, is also reported to use ESS and ERT [arxiv:2103.13744 §4.1]. ## Variants ### Occupancy grid from a trained density field (KiloNeRF) The occupancy grid is extracted from a trained NeRF teacher, not from the images. Each occupancy cell is filled by querying the teacher's density on a 3x3x3 sub-grid inside the cell. The cell is marked occupied if any of those densities exceeds a threshold of 10 [arxiv:2103.13744 §3.4] [arxiv:2103.13744 §4.1]. The grid is then fixed for fine-tuning and rendering. Card: [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](/papers/2103.13744). The paper reports the effect of ESS and ERT together on a single scene. On Lego, NeRF with ESS and ERT renders in 788 ms, against 56185 ms for the baseline NeRF, a factor of 71. The tiny networks give a further factor of about 36 [arxiv:2103.13744 tab. 2] [arxiv:2103.13744 §4.3]. The paper does not report ESS alone without ERT, so the separate contribution of ESS is not known from the card. ### Early ray termination as a companion step ERT is not ESS, but the paper uses it in the same render loop. A ray leaves the active set once its transmittance falls below 0.01. Samples are evaluated in passes ordered by distance from the camera, so terminated rays drop out between passes [arxiv:2103.13744 §3.4] [arxiv:2103.13744 §4.1]. Samples are taken stratified and equidistant, at most 384 per ray [arxiv:2103.13744 §3.4] [arxiv:2103.13744 §4.1]. ### Depth oracle instead of a grid (DONeRF) DONeRF keeps no occupancy grid. A single MLP, the depth oracle, is evaluated once per ray. It outputs one score per depth class. The scores are read as a piecewise-constant density, and samples are placed at uniform steps of its inverse CDF, so most of them fall where the oracle expects a surface [arxiv:2103.03231 §3] [arxiv:2103.03231 §5.1]. A shading network then evaluates only the placed samples, 2 to 16 per ray [arxiv:2103.03231 §3]. The oracle is trained against a per-ray depth target that is blurred in image and depth space, and the blur is not applied at inference [arxiv:2103.03231 §5.1]. Card: [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](/papers/2103.03231). Author-reported, over six Blender scenes at 400×400: mean PSNR is 31.14 dB at 4 samples per ray and 29.50 dB at 2, against 30.52 dB for NeRF at 256 samples [arxiv:2103.03231 tab. 2]. The paper's timing breakdown covers 2 samples only: 51.3 ms per 800×800 frame on an RTX 3090 with TensorRT in half precision, of which 34.9 ms is shading [arxiv:2103.03231 §6.1]. The paper gives no wall-clock time for NeRF, so no measured speed-up against NeRF is reported [arxiv:2103.03231 §6.3]. ## When to use which | Situation | Use | Why | |---|---|---| | Bounded scene with a known bounding box, and a trained density model to build the grid from | Occupancy grid from the teacher, plus ERT at inference | The paper gets 26-91 ms renders against 37-183 s for NeRF on a GTX 1080 Ti, with comparable quality [arxiv:2103.13744 tab. 1] | | No trained density field, or a scene whose geometry the teacher does not capture | Not covered by the KiloNeRF card; consider DONeRF if depth is available | The grid is built only from the teacher, and geometry the teacher misses at density 10 or below is skipped and never recovered [arxiv:2103.13744 §3.4] [arxiv:2103.13744 §4.1] | | Camera poses stay in a bounded view cell and renders need only a few samples per ray | DONeRF with 4 samples per ray | Mean 31.14 dB at 4 samples, against 30.52 dB for NeRF at 256 [arxiv:2103.03231 tab. 2]; the view cell and pose range come from the paper's setup [arxiv:2103.03231 §3] [arxiv:2103.03231 §6.2] | | Only 2 samples per ray are affordable | Prefer 4 or more with DONeRF | Averaged over the six scenes, DONeRF-2 is 29.50 dB, below NeRF's 30.52 dB [arxiv:2103.03231 tab. 2] | | Training-time depth is not available, but a dense NeRF can be trained | DONeRF with depth from that NeRF (noGT) | Without ground-truth depth, averages fall 0.8 to 1.5 dB below the ground-truth variants [arxiv:2103.03231 tab. 2] [arxiv:2103.03231 §6.4] | | Unbounded scene | Not covered by either card | The KiloNeRF method assumes a known bounding box for the grid [arxiv:2103.13744 §3.2] [arxiv:2103.13744 §5]; DONeRF's view cell bounds where the camera may be [arxiv:2103.03231 §1] | | Fine-tuning cost matters | ESS during training too | The KiloNeRF paper uses ESS in fine-tuning; ERT is inference-only [arxiv:2103.13744 §3.4] | | Shading cost is the bottleneck and must scale predictably | Plan for shading to dominate, roughly linear in samples | In DONeRF, shading takes 34.9 ms at 2 samples and 270.4 ms at 16 [arxiv:2103.03231 §6.1] | | No custom GPU kernels | Be careful about expectations | Per-network sorting and fused evaluation are part of the KiloNeRF speed path; the authors note that sorting is cheap once most samples are removed [arxiv:2103.13744 §3.5] | | Compute budget for building the pipeline | Plan for the full training chain | Teacher training takes about two days on one GTX 1080 Ti, distillation up to 8 h, fine-tuning 17 h [arxiv:2103.13744 §4.1] | ## Implementation sketch Pseudocode in our own words. The occupancy build runs once, before training or rendering. The render loop is the common core: samples are filtered by the grid, evaluated, composited, and rays are dropped when they become opaque. ```pseudocode # Build the occupancy grid once from a trained density field [§3.4] def build_occupancy(density_fn, bbox_min, bbox_max, cells, sub=3, tau=10.0): # one bit per cell; cells are 16x finer than the network grid [§4.1] occ = zeros(cells, dtype=bool) for c in all_cells(cells): # query a sub x sub x sub grid of points inside the cell [§3.4] pts = sub_grid_points(c, sub, bbox_min, bbox_max) # the cell is occupied if any density exceeds tau [§3.4] [§4.1] occ[c] = max(density_fn(pts)) > tau return occ # Render with ESS and ERT [§3.4] [§3.5] def render(rays, field, occ, max_samples=384, eps=0.01): # per-ray state for the compositing of eq. 2-4 [eq. 2] [eq. 3] color = zeros(num_rays, 3) trans = ones(num_rays) live = all_rays # visit samples nearest-first, in passes [§3.4] for k in range(0, max_samples, pass_size): if not live: break pts = samples_for(live, k, pass_size) # ESS: drop samples whose occupancy cell is empty [§3.4] pts = pts[occ[cell_index(pts)]] # evaluate only what is left, grouped by network [§3.5] rgb, sigma = field(sort_by_cell(pts)) color, trans = composite(color, trans, rgb, sigma) # ERT: retire rays whose transmittance fell below eps [§3.4] live = live[trans[live] >= eps] return color ``` Where variants differ: whether a grid is used at all (DONeRF replaces the occupancy test with one oracle evaluation per ray, and the DONeRF paper card has its own sampler pseudocode), the grid source (a teacher model in KiloNeRF), the sub-sampling used to fill a cell, the threshold, and whether ESS runs during training. The papers' values for these are in the Variants section. ## Contradictions Speed-up figures are not comparable across the two papers. KiloNeRF reports measured render times against NeRF on a GTX 1080 Ti [arxiv:2103.13744 tab. 1]. DONeRF's speed-up range is a ratio of FLOP per pixel, not a measured time, and the paper reports no NeRF wall-clock time [arxiv:2103.03231 tab. 2] [arxiv:2103.03231 §6.3]. The DONeRF paper also gives different FLOP-ratio ranges in different places, including §6.3 and the table caption [arxiv:2103.03231 §6.3] [arxiv:2103.03231 tab. 2]. Neither figure should be quoted as a like-for-like speed-up. ## Papers - [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](/papers/2103.13744): uses an occupancy grid from a teacher NeRF, with ESS during fine-tuning and inference and ERT at inference, for a reported 1258-2167x render speed-up over NeRF. - [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](/papers/2103.03231): a per-ray depth oracle places 2 to 16 shading samples without an occupancy grid; author-reported 31.14 dB mean PSNR at 4 samples per ray.