---
kind: concept
slug: continuous-convolution
title: Continuous convolution
summary: >-
  A convolution defined over continuous coordinates, so it runs on unevenly spaced points such as lidar returns and can output at any location. The kernel is a small MLP of the relative offset between points. One card so far, reporting gains on indoor segmentation and lidar motion; the reported numbers are author-reported and the training details are not stated.
tags: [convolution, point-cloud, lidar, architecture]
papers: ["2101.06742"]
related: [convolutional-neural-network, deformable-convolution, graph-neural-network, multi-layer-perceptron, semantic-segmentation]
status: draft
updated_at: 2026-10-24
---

## Overview

A standard convolution slides a fixed grid of learned weights over its input,
so it needs the data to sit on a grid. Point clouds from lidar or depth sensors
do not. A continuous convolution removes the grid: the kernel is a function of
the relative position between two points, so it can be evaluated for any
offset. The output is a sum over sampled support points, weighted by that
kernel [arxiv:2101.06742 §3.1]. The kernel is learned as a small multi-layer
perceptron, which lets the same layer take scattered inputs and produce outputs
at new positions [arxiv:2101.06742 §3.1] [arxiv:2101.06742 §3.2].

The only card for this concept is one paper, which uses the operator in a deep
residual network for indoor and driving point clouds. The design space below
therefore has one variant. Other approaches to point data, such as voxel grids
and graph convolutions, are named there only as the paper's comparisons and
special cases.

## Variants

### Parametric continuous convolution (MLP kernel)

The kernel is an MLP of the offset z between points. The authors argue that an
MLP is a universal approximator for continuous functions and that polynomial
kernels are either not expressive enough or unstable to train
[arxiv:2101.06742 §3.1]. Locality is added by multiplying the kernel with a
window that is 1 for the K nearest neighbours or for points within a radius r
[arxiv:2101.06742 §3.3]. For cost, the layer computes one kernel value per
point pair and output channel, not one per input-output channel pair, and mixes
input channels with a shared linear map [arxiv:2101.06742 §3.3]. At inference
the batch-norm layer is merged into the final fully connected layer of the kernel
MLP, which the authors report as a 3x speed-up [arxiv:2101.06742 §3.3]. Card:
[Deep Parametric Continuous Convolutional Neural Networks](/papers/2101.06742).

Two networks are built from this layer. A 13-class indoor model stacks 8
continuous convolution layers with 32 hidden dimensions and a global max-pooled
feature concatenated to each point [arxiv:2101.06742 §4.1]. A driving model
stacks 16 residual layers in one configuration and 7 layers on top of a 3D-FCN
in another [arxiv:2101.06742 §4.2] [arxiv:2101.06742 tab. 2].

## When to use which

The paper compares its operator with the alternatives on accuracy and cost.
The numbers are author-reported and come from the paper's own setups, so they
are not directly comparable to other benchmarks.

| Situation | Use | Why |
|---|---|---|
| Points are unevenly spaced in 3D and the output must be defined at arbitrary positions | Parametric continuous convolution | The kernel is defined for any offset, so no grid is needed [arxiv:2101.06742 §3.1] |
| Input lies on a regular grid (pixels, voxel cells) | Ordinary convolution | The paper states that a regular grid gives ordinary convolution as a special case [arxiv:2101.06742 §3.3] |
| Memory for a dense voxel grid is a concern | Continuous convolution on points | The authors argue voxel grids are memory-heavy because most cells are empty [arxiv:2101.06742 §2] |
| Accuracy on indoor scans matters most | Continuous convolution, deep residual stack | Reported 58.27 mIOU against 48.92 for SEGCloud and 41.09 for PointNet [arxiv:2101.06742 tab. 1] |
| Model size is the constraint | Continuous convolution alone | Reported 9.34 MB for the driving PCCN against 74.66 MB for 3D-FCN, at a lower mIOU of 46.35 against 49.28 [arxiv:2101.06742 tab. 2] |
| Best driving accuracy is the goal, and a voxel backbone is acceptable | 3D-FCN followed by continuous convolution layers | Reported 58.06 mIOU, the best of the four driving models, at 74.67 MB [arxiv:2101.06742 tab. 2] |
| Per-point motion between two lidar sweeps | Continuous convolution, 7 residual layers, MSE at each layer | Reported 7.810 cm end-point error against 8.161 cm for 3D-FCN [arxiv:2101.06742 tab. 3] |
| Latency budget | Check the benchmark setup first | One 8-layer model with 50 neighbours ran a forward pass in 33 ms, plus 28 ms for the neighbour search, on a GTX 1080 Ti [arxiv:2101.06742 §4.2] |
| Public data or code is required to reproduce | Not supported by the sources | The driving and flow data are in-house, and no official code was found [arxiv:2101.06742 §4.2] [arxiv:2101.06742 §4.3] |

## Implementation sketch

Language-agnostic pseudocode of the common core. The neighbour window and
the kernel MLP are the parts the paper leaves open, see Pitfalls on the card.

```pseudocode
# Parametric continuous convolution layer  [arxiv:2101.06742 §3.2] [§3.3]
# Inputs:  features (N, F), support_locs (N, D), output_locs (M, D)
# Output:  out (M, O)

def continuous_conv(features, support_locs, output_locs, kernel_mlp, W1):
    # Offsets from each output point to each support point  [§3.1]
    offsets = output_locs[:, None, :] - support_locs[None, :, :]  # (M, N, D)
    # Locality window: K nearest support points or those within radius r  [§3.3]
    window = nearest_or_radius_mask(offsets)                  # (M, N)
    # One kernel value per point pair and output channel, from the MLP  [§3.3]
    kernel = kernel_mlp(offsets) * window[..., None]          # (M, N, O)
    # Linear mixing of input channels, shared across point pairs  [§3.3]
    mixed = features @ W1                                     # (N, O)
    # Weighted sum over support points, the Monte-Carlo estimate  [§3.1]
    out = (kernel * mixed[None, :, :]).sum(axis=1)            # (M, O)
    return out
```

## Contradictions

None known across papers: only one card exists. Within the paper, the layer
formula [arxiv:2101.06742 §3.2] reverses the index names defined in §3.1, and the
text gives different dataset sizes in its introduction, experiments and
conclusion [arxiv:2101.06742 §1] [arxiv:2101.06742 §4] [arxiv:2101.06742 §5].

## Papers

- [Deep Parametric Continuous Convolutional Neural Networks](/papers/2101.06742): introduces parametric continuous convolution with an MLP kernel and applies it to indoor segmentation, driving segmentation and lidar motion estimation.