---
kind: concept
slug: neural-sde
title: Neural SDE
summary: >-
  A neural stochastic differential equation evolves a hidden state in continuous time, so irregular or partly missing series need no fixed step. Three designs (Langevin-type, linear-noise, geometric) add a spline-driven controlled path and stability conditions. Author-reported gains over RNN, ODE and CDE baselines, but these designs use more memory than the Neural CDE baselines.
tags: [stochastic-differential-equations, irregular-time-series, continuous-time, time-series]
papers: ["2402.14989"]
related: [neural-ode, time-series-classification, time-series-imputation, sde-parameter-estimation]
status: draft
updated_at: 2026-11-12
---

## Overview

A neural SDE keeps a latent state z(t) that changes continuously in time and
is pushed by Brownian noise. Observations arrive at uneven times, so the state
is read wherever a value was recorded [arxiv:2402.14989 §1]. A naive neural
SDE can lack a unique strong solution, can destabilise during training, and
its Euler discretisation can blow up in finite time [arxiv:2402.14989 §2.2].

The designs share three pieces. The initial state is an affine map of the
input, z(0) = h(x) [arxiv:2402.14989 §3.1]. The drift receives a controlled
map of the state and a spline through the observations [arxiv:2402.14989 eq. 10]
[arxiv:2402.14989 §3.4]. A two-layer read-out maps the final state to the
prediction [arxiv:2402.14989 eq. 9]. What differs is the form of the drift and
diffusion, and that choice decides which stability guarantees apply.

## Variants

### Langevin-type (LSDE)

The drift depends on the controlled state only, and the diffusion is a
function of time, dz = γ(z̄) dt + σ(t) dW [arxiv:2402.14989 eq. 4]. The
paper assumes a potential U with ∇U = γ and a diffusion that tends to a
positive constant. Under these conditions the process has a unique invariant
(Gibbs) measure [arxiv:2402.14989 §3.1]. Card: [Neural LSDE](/papers/2402.14989).

### Linear-noise (LNSDE)

The diffusion is multiplied by the state, dz = γ(t, z̄) dt + σ(t) z dW
[arxiv:2402.14989 eq. 5]. The drift depends on time as well as on the
controlled state. For constant σ with |σ|² > 2L_γ, the paper gives an
exponential bound on how far a perturbed input moves the output distribution
for large T [arxiv:2402.14989 §3.3]. Card: [Neural LNSDE](/papers/2402.14989).

### Geometric (GSDE)

The state enters the equation in relative form, dz/z = γ(t, z̄) dt + σ(t) dW
[arxiv:2402.14989 eq. 6]. With a tanh or sigmoid activation in γ, the paper
proves that the solution is almost surely non-negative and that state 0 is
absorbing [arxiv:2402.14989 §3.2] [arxiv:2402.14989 app. A]. The bound
condition is |σ|² > 2K_γ [arxiv:2402.14989 §3.3]. Card:
[Neural GSDE](/papers/2402.14989).

### Naive Neural SDE (baseline, not a recommended design)

The baseline the paper calls "Neural SDE", which does not use the three
stability-oriented designs above. The paper's test on one dataset at a 50% missing
rate shows that a non-Lipschitz diffusion and the naive network diffusion both
fail to stabilise training [arxiv:2402.14989 §2.2] [arxiv:2402.14989 fig. 1].
The paper leaves it out of the Speech table for that reason [arxiv:2402.14989
§4.1].

### Reported results across the variants

All numbers are author-reported. The authors rerun the robustness baselines
with their code, and quote others from prior papers [arxiv:2402.14989 app. E].

- Interpolation on PhysioNet Mortality: the three designs have the lowest MSE
  of all rows in the table at every observed fraction. The gaps between them
  are under 0.1 (×10⁻³) at each level, with standard deviations of similar size
  [arxiv:2402.14989 tab. 1].
- Sepsis classification, with observation intensity (OI): AUROC 0.909 (LSDE),
  0.911 (LNSDE), 0.909 (GSDE), against 0.900 for ANCDE [arxiv:2402.14989 tab. 2].
  Without OI: 0.879, 0.881 and 0.884 for the three, against 0.823 for ANCDE
  [arxiv:2402.14989 tab. 2].
- Speech Commands: accuracy 0.927 (LSDE), 0.923 (LNSDE), 0.913 (GSDE),
  against 0.922 for LEAP [arxiv:2402.14989 tab. 3].
- 30 UEA/UCR datasets with missing values: average rank among the Neural SDE
  family is best for LNSDE at every missing rate (5.4 regular, 5.0 at 30%, 4.3
  at 50%, 4.2 at 70%). The three designs differ by at most 1.8 average rank at
  each level [arxiv:2402.14989 tab. 4].
- Ablation on the same 30 datasets: removing the controlled path drops accuracy
  to about 0.52 for all three designs. Under the reading the card uses for the
  undefined O/X labels, the nonlinear diffusion network beats an affine one
  [arxiv:2402.14989 tab. 5] [arxiv:2402.14989 §4.3].
- MuJoCo forecasting, regular sampling: test MSE 0.013 (LSDE), 0.012 (LNSDE),
  0.013 (GSDE), against 0.026 for ANCDE and 0.022 for LEAP
  [arxiv:2402.14989 app. E].

## When to use which

The choice depends on the memory budget, the need for the stability guarantees,
and whether the series are irregular and partly missing.

| Situation | Use | Why |
|---|---|---|
| Irregular, partly missing series, accuracy is the goal and memory is not tight | LNSDE | Best average rank on the 30-dataset study [arxiv:2402.14989 tab. 4]; top Sepsis AUROC with OI [arxiv:2402.14989 tab. 2] |
| Same, but the state must stay non-negative | GSDE, after checking the implementation | Positivity is proved for the exact geometric form [arxiv:2402.14989 §3.2]; the official code does not enforce it [code: benchmark_classification/models_sde/neuralsde.py#L279-L282] |
| Memory is tight, or CDE-style models suffice | Neural CDE or ANCDE | Sepsis memory 122 to 285 MB against 280 to 588 MB for the Neural SDEs [arxiv:2402.14989 tab. 2]; Speech 175 to 180 MB against 1164 to 1565 MB [arxiv:2402.14989 tab. 3] |
| Regular, long-horizon forecasting, where a higher memory cost is acceptable | LSDE or LNSDE | MuJoCo error about half that of ANCDE, at 249 to 273 MB against 79 MB [arxiv:2402.14989 app. E] |
| Need the proved robustness bound as a number | Not available from this paper | The bound's constants (L_F, L_h, L_γ, K_γ, c_1, c_2) are not given numerically [arxiv:2402.14989 §3.3] |
| Naive design without stability structure | Avoid | Training instability on the test dataset [arxiv:2402.14989 §2.2] |

Average-rank differences are small. Choose between the three designs by memory
and by the checks listed under Contradictions, not by rank alone.

## Implementation sketch

The loop below is shared. The three designs differ only in how drift and
diffusion are formed from the controlled state, as in the variant sections.

```pseudocode
# Forward pass for one series, shared by the three designs
# [arxiv:2402.14989 §3.1] [arxiv:2402.14989 eq. 10]
# Inputs: observation times, values, spline through the values
# Output: read-out for one series

def forward(times, values, spline, design):
    # Initial state is an affine map of the first observation [§3.1]
    z = h(values[0])
    for t, dt, dW in brownian_steps(times):
        # Controlled state seen by the drift  [eq. 10] [§3.4]
        z_bar = zeta(t, z, spline(t))
        if design == "LSDE":
            # Drift in z_bar only, diffusion in time only  [eq. 4]
            drift = gamma(z_bar)
            diff = sigma(t)
        elif design == "LNSDE":
            # Diffusion scales with the state  [eq. 5]
            drift = gamma(t, z_bar)
            diff = sigma(t) * z
        else:
            # Relative form: dz = z * (gamma dt + sigma dW)  [eq. 6]
            drift = z * gamma(t, z_bar)
            diff = z * sigma(t)
        # Explicit Euler-Maruyama step, diagonal noise  [app. D.4]
        z = z + drift * dt + diff * dW
    # Two-layer read-out of the terminal state  [eq. 9]
    return F(z)
```

Notes for implementers:

- The paper does not state the Euler step size [arxiv:2402.14989 app. D.4].
  The official code uses the smallest gap between observation times, floored
  at 10⁻³ [code: benchmark_classification/models_sde/neuralsde.py#L30-L48].
- The paper's σ is an unrestricted network with sinusoidal time encoding. The
  code differs in several places, including a learned sigmoid scale on the
  diffusion, a ReLU on the diffusion output, and tanh clipping of drift and
  diffusion. See Contradictions before reusing the code.
- The appendix selects a Hermite cubic with backward differences for the
  controlled path. The main text also names natural cubic splines
  [arxiv:2402.14989 §2.1] [arxiv:2402.14989 app. F].

## Contradictions

- **Within the paper, LNSDE on Speech.** Table 3 gives 0.923 ± 0.001, while
  the appendix gives 0.924 ± 0.000 for the same configuration [arxiv:2402.14989
  tab. 3] [arxiv:2402.14989 app. E].
- **Within the paper, Mortality configuration.** The proposed rows of Table 1
  equal one grid cell at every level. The bold LSDE cell at 50% is 3.793, a
  different configuration, lower than the 3.799 reported. The text does not say
  which configuration Table 1 uses [arxiv:2402.14989 tab. 1] [arxiv:2402.14989 app. E].
- **Within the paper, compute cost.** The text says the methods need more
  compute than Neural CDE models [arxiv:2402.14989 §5]. The appendix runtime on
  BasicMotions gives 89 to 96 seconds for 100 epochs for the proposed methods,
  against 102.5 seconds for Neural CDE [arxiv:2402.14989 app. D.4]. These point
  in different directions for that one dataset, and the paper does not reconcile
  them.
- **Paper against code, GSDE form.** The paper's GSDE is dz/z = γ dt + σ dW.
  The official code replaces the factor z by a tanh of the state and clips
  afterwards [code: benchmark_classification/models_sde/neuralsde.py#L219-L231].
  The positivity proof therefore does not describe the code's model.
- **Paper against code, diffusion scale.** The code multiplies the diffusion by
  a learned sigmoid and applies ReLU to the diffusion network. The paper
  specifies neither [code: benchmark_classification/models_sde/neuralsde.py#L163]
  [code: benchmark_classification/models_sde/neuralsde.py#L279-L282].
- **Paper against code, model selection.** The paper keeps the parameters with
  the lowest validation loss. The code keeps the checkpoint with the highest
  validation accuracy [code: benchmark_classification/common_sde.py#L178-L181].
- **Undefined symbols in the ablation.** The caption of Table 5 does not define
  O and X, and the meaning used here is inferred from the text [arxiv:2402.14989
  §4.3]. Read the ablation with that caveat.

## Papers

- [Stable Neural SDEs for irregular series (Oh et al., 2024)](/papers/2402.14989):
  defines the Langevin-type, linear-noise and geometric designs, with a
  controlled path in the drift, stability results and a missing-data
  robustness bound. Tested on interpolation, classification, forecasting and
  30 UEA/UCR datasets with missing values.
