---
kind: concept
slug: depth-completion
title: Depth completion
summary: >-
  Fills the holes in a sparse depth map, such as LIDAR, into a dense one. Unguided methods use only the depth map; guided methods also use a colour image. The one card here is a fixed chain of classical morphology and blur steps with no training data, about 90 Hz on a CPU, and the best KITTI test RMSE at submission. Its constants were tuned on KITTI.
tags: [computer-vision, depth, lidar, cpu-inference, no-training]
papers: ["1802.00036"]
related: [guided-depth-upsampling, monocular-depth-estimation, multi-view-stereo]
status: draft
updated_at: 2026-11-01
---

## Overview

A LIDAR sensor returns depth at only a few pixels. In the bottom two thirds of
KITTI depth maps, 5–7% of pixels carry a return [arxiv:1802.00036 §5]. Depth
completion turns that sparse map into a dense one, which downstream tasks such
as 3D detection, obstacle avoidance and SLAM need [arxiv:1802.00036 §1].

This page covers the unguided case, where only the sparse depth map is the
input [arxiv:1802.00036 §3]. Guided methods also take a colour image; the
source paper notes that they depend on image quality and on calibration and
synchronisation between the camera and the depth sensor [arxiv:1802.00036 §2].
Learned unguided methods need a GPU and large training sets [arxiv:1802.00036 §2].

The goal is stated as an error to be driven to zero against an unknown target
function [arxiv:1802.00036 §3, eq. 1]. No card on this page optimises that
objective. The method is a hand-designed estimator, and its error is not
analysed [arxiv:1802.00036 §3].

The invariant the variants share: valid depths are grown outward into empty
pixels with a grey-scale maximum filter, so they never get overwritten by
zeros. The cards here differ only in the kernels and blurs they choose.

## Variants

### Classical morphological pipeline (IP-Basic)

A fixed sequence of about eight steps: invert the depths, dilate, close small
holes, fill the rest, optionally extend columns upward and fill large holes,
then median and Gaussian blur, and invert back [arxiv:1802.00036 §4]. Nothing
is learned, so no training data is needed, and it runs on an ordinary CPU.
The authors report 0.011 s per frame on an Intel Core i7-7700K, about 90 Hz
[arxiv:1802.00036 §5.1, tab. 1]. Card:
[IP-Basic card](/papers/1802.00036).

The kernel choices came from a greedy search on the KITTI validation split, in
this order: size, then shape, then blur [arxiv:1802.00036 §5.2]. The best
initial kernel was a 5×5 diamond [arxiv:1802.00036 tab. 2]. The best final
blur was median followed by Gaussian [arxiv:1802.00036 tab. 3].

### Learned and guided methods

The source paper compares against learned and guided approaches [arxiv:1802.00036
§2, tab. 1]. No card on this wiki covers them yet, so this page does not
describe their designs.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Sparse LIDAR only, no training data, CPU budget | Classical pipeline (IP-Basic) | No training; 0.011 s per frame on an i7-7700K, author-reported [arxiv:1802.00036 tab. 1] |
| Need the lowest RMSE and MAE on the KITTI test server, as of submission | Classical pipeline (IP-Basic) | 1288.46 mm RMSE and 302.60 mm MAE, against 1419.75 mm and 416.14 mm for the best learned baseline [arxiv:1802.00036 tab. 1] |
| Need the lowest iRMSE on the same server | Learned baseline (NN+CNN) | 3.25 1/km against 3.78 1/km for the classical pipeline [arxiv:1802.00036 tab. 1] |
| A colour image is available and calibrated and synchronised with the depth sensor | Guided methods (not covered by a card here) | Guided methods use the colour image [arxiv:1802.00036 §2] |
| Different sensor, depth range or domain than KITTI | Re-tune before use | The constants were picked greedily on KITTI validation [arxiv:1802.00036 §5.2]; the paper makes no transfer claim |
| Downstream code needs the column extension | Check it first | The authors suggest turning extension and large-hole fill off for practical use, since they add many extrapolated points [arxiv:1802.00036 §5.2] |

## Implementation sketch

The pipeline below is the paper's algorithm in our own words [arxiv:1802.00036
§4]. Kernel sizes and the blur follow the paper's final choices. The valid
threshold is not stated in the paper, so `EPS` is a placeholder.

```pseudocode
# Sparse-to-dense depth completion without training  [arxiv:1802.00036 §4]
# Input:  depth (H, W) in metres; 0 marks no return
# Output: dense depth (H, W) in metres
MAX_D = 100.0   # inversion constant; KITTI depths are at most 80 m  [§4]
EPS = 0.0       # valid threshold, not stated in the paper

def complete_depth(depth, extend=False):
    # Invert so that max-filters grow valid depths, not zeros  [§4]
    d = where(depth > EPS, MAX_D - depth, 0)
    # Grow each valid pixel with a 5x5 diamond max-filter  [§4, tab. 2]
    d = dilate(d, diamond(5))
    # Join nearby patches with a 5x5 closing  [§4]
    d = close(d, full(5))
    # Fill only the still-empty pixels from a 7x7 max-filter  [§4]
    holes = d <= EPS
    d[holes] = dilate(d, full(7))[holes]
    if extend:
        # Optional: copy the top valid value of each column upward  [§4, §5.2]
        d = extend_columns_up(d)
        # Fill the remaining holes from a 31x31 max-filter  [§4]
        holes = d <= EPS
        d[holes] = dilate(d, full(31))[holes]
    # Remove outliers, then smooth  [§4, tab. 3]
    d = median(d, 5)
    # Gaussian smoothing; sigma is not stated in the paper  [§4]
    d = gaussian(d, 5)
    # Undo the inversion on valid pixels  [§4]
    return where(d > EPS, MAX_D - d, 0)
```

## Contradictions

None across papers: this page has one card. Within that paper, the
classical pipeline does not win every metric. It has the lower test RMSE and
MAE, but the learned baseline has the lower iRMSE [arxiv:1802.00036 tab. 1].
The released code also differs from the paper's stated configuration in
places (defaults, blur, and whether the extension is on). Those differences
can change a reproduction. The [card](/papers/1802.00036) lists them with code
citations.

## Pitfalls

- **Tuning is specific to KITTI.** The 100 m inversion, the kernel sizes and
  the blurs were chosen on KITTI validation, in a fixed order [arxiv:1802.00036
  §5.2]. The paper does not show they transfer to other sensors.
- **Validation and test numbers differ.** Tables 2 and 3 are validation
  ablations; table 1 is the test server [arxiv:1802.00036 §5, tab. 1–3]. Do not
  compare an ablation row with a test row.
- **Depths above the inversion constant.** Depths above 100 m become negative
  after inversion. The paper does not discuss this case [arxiv:1802.00036 §4].
- **Dilation must keep the larger value.** The method relies on a grey-scale
  maximum filter so that zeros do not overwrite real depths [arxiv:1802.00036
  §4]. Replacing the dilation with a mean or median filter changes the method.
- **Top-of-frame extension is a strong assumption.** It assumes a tall object
  continues above the highest return in its column [arxiv:1802.00036 §4]. The
  authors say it adds many extrapolated points and suggest turning it off in
  practice [arxiv:1802.00036 §5.2].
- **Ranking is time-bound.** "First among published methods" was true at
  submission [arxiv:1802.00036 §5.1]. Later leaderboard entries are not covered
  here.
- **Runtime setup.** The 0.011 s figure is on an i7-7700K. The paper does not
  say whether it ran single-threaded [arxiv:1802.00036 tab. 1].

## Papers

- [IP-Basic](/papers/1802.00036): an eight-step classical pipeline with no training data, run on a CPU at about 90 Hz, with the best KITTI test RMSE at submission.