Mathematical morphology

Overview

Morphological operations change an image by looking at a small neighbourhood around each pixel, called the kernel, and combining the values in it. The paper's pipeline uses OpenCV's grey-scale operations, in which dilation keeps the larger value in the neighbourhood, so a bright value spreads into its neighbours 1802.00036 §4. Closing joins small gaps by combining dilation with erosion, and a masked fill writes values only into pixels that were empty 1802.00036 §4.

The shared idea is that the operation is fixed by a kernel shape and size, and that the order of the steps matters. The paper's central point is that the order needs the values to be inverted first: otherwise the empty value zero would overwrite real depths in a dilation 1802.00036 §4. The variants below differ only in the kernel and in which optional steps run. The one card for this concept is Fast Depth Completion on the CPU, which uses the operations to fill sparse LIDAR depth 1802.00036 §3.

Variants

Inverted-depth dilation pipeline (paper's method)

Eight steps in fixed order: invert depth, dilate with a 5×5 diamond kernel, close with a 5×5 full kernel, fill empty pixels from a 7×7 full dilation, optionally extend the top of the frame and fill large holes from a 31×31 full dilation, apply a median and a Gaussian blur, then invert back [arxiv:1802.00036 §4]. On the KITTI depth completion test set it reports 1288.46 mm RMSE and 302.60 mm MAE, with a runtime of 0.011 s on an Intel Core i7-7700K [arxiv:1802.00036 tab. 1]. Card: Fast Depth Completion on the CPU.

Optional extension and large-hole fill

The top-of-frame extension copies the highest valid value in each column upward, and the 31×31 fill then closes the rest. The authors present both as steps of the algorithm 1802.00036 §4, but they also say the two steps add many extrapolated points, and suggest turning them off for practical use 1802.00036 §5.2. On the validation split, the paper's final median plus Gaussian blur gives 1350.93 mm RMSE 1802.00036 tab. 3; the table does not split out the effect of the extension alone. Card: Fast Depth Completion on the CPU.

When to use which

Implementation sketch

Pseudocode of the paper's pipeline, in the style of the paper cards. Values are in metres, and zero marks an empty pixel 1802.00036 §4.

# Inverted-depth morphology for sparse depth  [arxiv:1802.00036 §4]
# Input:  depth (H, W) in metres; 0 marks an empty pixel
# Output: depth (H, W) in metres

def complete(depth, max_depth=100.0, extend=False):
    # Invert so that the grey-scale max keeps real values  [arxiv:1802.00036 §4]
    valid = depth > 0
    depth = where(valid, max_depth - depth, depth)
    # Dilate with a 5x5 diamond kernel  [arxiv:1802.00036 §4] [arxiv:1802.00036 tab. 2]
    depth = dilate(depth, diamond_5x5)
    # Close small gaps with a 5x5 full kernel  [arxiv:1802.00036 §4]
    depth = close(depth, full_5x5)
    # Fill only the empty pixels from a 7x7 full dilation  [arxiv:1802.00036 §4]
    empty = depth == 0
    depth[empty] = dilate(depth, full_7x7)[empty]
    if extend:
        # Copy the highest valid value of each column upward  [arxiv:1802.00036 §4]
        depth = extend_to_top(depth)
        # Fill the rest of the empty pixels from a 31x31 full dilation  [arxiv:1802.00036 §4]
        empty = depth == 0
        depth[empty] = dilate(depth, full_31x31)[empty]
    # Remove outliers with a 5x5 median, then smooth with a 5x5 Gaussian  [arxiv:1802.00036 tab. 3]
    depth = gaussian(median(depth, 5), 5)
    # Undo the inversion  [arxiv:1802.00036 §4]
    valid = depth > 0
    return where(valid, max_depth - depth, depth)

Points to check when porting: the paper's own output formula subtracts from 100 m for every pixel, while the released code keeps empty pixels at zero 1802.00036 §4. The card records this and other code-only details Fast Depth Completion on the CPU.

Contradictions

Within the one paper, the authors describe the extension and large-hole fill as steps of the algorithm 1802.00036 §4, and later suggest turning them off for practical use because they add extrapolated points [arxiv:1802.00036 §5.2]. Both statements are the paper's; the page does not resolve them.

No contradictions between papers are known, because only one card exists.

Papers

• Fast Depth Completion on the CPU: classical morphology and blur chain for sparse LIDAR depth, with no training, author-reported 1288.46 mm test RMSE on KITTI.