Concept · 1 paper · updated 2026-11-12
Neural SDE
A neural stochastic differential equation evolves a hidden state in continuous time, so irregular or partly missing series need no fixed step. Three designs (Langevin-type, linear-noise, geometric) add a spline-driven controlled path and stability conditions. Author-reported gains over RNN, ODE and CDE baselines, but these designs use more memory than the Neural CDE baselines.
stochastic-differential-equationsirregular-time-seriescontinuous-timetime-series
Markdown6 min read
AI-compiled overview, updated 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
Overview
A neural SDE keeps a latent state z(t) that changes continuously in time and is pushed by Brownian noise. Observations arrive at uneven times, so the state is read wherever a value was recorded 2402.14989 §1. A naive neural SDE can lack a unique strong solution, can destabilise during training, and its Euler discretisation can blow up in finite time 2402.14989 §2.2.
The designs share three pieces. The initial state is an affine map of the input, z(0) = h(x) 2402.14989 §3.1. The drift receives a controlled map of the state and a spline through the observations 2402.14989 eq. 10 2402.14989 §3.4. A two-layer read-out maps the final state to the prediction 2402.14989 eq. 9. What differs is the form of the drift and diffusion, and that choice decides which stability guarantees apply.
Variants
Langevin-type (LSDE)
The drift depends on the controlled state only, and the diffusion is a function of time, dz = γ(z̄) dt + σ(t) dW 2402.14989 eq. 4. The paper assumes a potential U with ∇U = γ and a diffusion that tends to a positive constant. Under these conditions the process has a unique invariant (Gibbs) measure 2402.14989 §3.1. Card: Neural LSDE.
Linear-noise (LNSDE)
The diffusion is multiplied by the state, dz = γ(t, z̄) dt + σ(t) z dW 2402.14989 eq. 5. The drift depends on time as well as on the controlled state. For constant σ with |σ|² > 2L_γ, the paper gives an exponential bound on how far a perturbed input moves the output distribution for large T 2402.14989 §3.3. Card: Neural LNSDE.
Geometric (GSDE)
The state enters the equation in relative form, dz/z = γ(t, z̄) dt + σ(t) dW 2402.14989 eq. 6. With a tanh or sigmoid activation in γ, the paper proves that the solution is almost surely non-negative and that state 0 is absorbing 2402.14989 §3.2 2402.14989 app. A. The bound condition is |σ|² > 2K_γ 2402.14989 §3.3. Card: Neural GSDE.
Naive Neural SDE (baseline, not a recommended design)
The baseline the paper calls "Neural SDE", which does not use the three stability-oriented designs above. The paper's test on one dataset at a 50% missing rate shows that a non-Lipschitz diffusion and the naive network diffusion both fail to stabilise training 2402.14989 §2.2 2402.14989 fig. 1. The paper leaves it out of the Speech table for that reason [arxiv:2402.14989 §4.1].
Reported results across the variants
All numbers are author-reported. The authors rerun the robustness baselines with their code, and quote others from prior papers 2402.14989 app. E.
- Interpolation on PhysioNet Mortality: the three designs have the lowest MSE of all rows in the table at every observed fraction. The gaps between them are under 0.1 (×10⁻³) at each level, with standard deviations of similar size 2402.14989 tab. 1.
- Sepsis classification, with observation intensity (OI): AUROC 0.909 (LSDE), 0.911 (LNSDE), 0.909 (GSDE), against 0.900 for ANCDE 2402.14989 tab. 2. Without OI: 0.879, 0.881 and 0.884 for the three, against 0.823 for ANCDE 2402.14989 tab. 2.
- Speech Commands: accuracy 0.927 (LSDE), 0.923 (LNSDE), 0.913 (GSDE), against 0.922 for LEAP 2402.14989 tab. 3.
- 30 UEA/UCR datasets with missing values: average rank among the Neural SDE family is best for LNSDE at every missing rate (5.4 regular, 5.0 at 30%, 4.3 at 50%, 4.2 at 70%). The three designs differ by at most 1.8 average rank at each level 2402.14989 tab. 4.
- Ablation on the same 30 datasets: removing the controlled path drops accuracy to about 0.52 for all three designs. Under the reading the card uses for the undefined O/X labels, the nonlinear diffusion network beats an affine one 2402.14989 tab. 5 2402.14989 §4.3.
- MuJoCo forecasting, regular sampling: test MSE 0.013 (LSDE), 0.012 (LNSDE), 0.013 (GSDE), against 0.026 for ANCDE and 0.022 for LEAP 2402.14989 app. E.
When to use which
The choice depends on the memory budget, the need for the stability guarantees, and whether the series are irregular and partly missing.
| Situation | Use | Why |
|---|---|---|
| Irregular, partly missing series, accuracy is the goal and memory is not tight | LNSDE | Best average rank on the 30-dataset study 2402.14989 tab. 4; top Sepsis AUROC with OI 2402.14989 tab. 2 |
| Same, but the state must stay non-negative | GSDE, after checking the implementation | Positivity is proved for the exact geometric form 2402.14989 §3.2; the official code does not enforce it neuralsde.py |
| Memory is tight, or CDE-style models suffice | Neural CDE or ANCDE | Sepsis memory 122 to 285 MB against 280 to 588 MB for the Neural SDEs 2402.14989 tab. 2; Speech 175 to 180 MB against 1164 to 1565 MB 2402.14989 tab. 3 |
| Regular, long-horizon forecasting, where a higher memory cost is acceptable | LSDE or LNSDE | MuJoCo error about half that of ANCDE, at 249 to 273 MB against 79 MB 2402.14989 app. E |
| Need the proved robustness bound as a number | Not available from this paper | The bound's constants (L_F, L_h, L_γ, K_γ, c_1, c_2) are not given numerically 2402.14989 §3.3 |
| Naive design without stability structure | Avoid | Training instability on the test dataset 2402.14989 §2.2 |
Average-rank differences are small. Choose between the three designs by memory and by the checks listed under Contradictions, not by rank alone.
Implementation sketch
The loop below is shared. The three designs differ only in how drift and diffusion are formed from the controlled state, as in the variant sections.
# Forward pass for one series, shared by the three designs
# [arxiv:2402.14989 §3.1] [arxiv:2402.14989 eq. 10]
# Inputs: observation times, values, spline through the values
# Output: read-out for one series
def forward(times, values, spline, design):
# Initial state is an affine map of the first observation [§3.1]
z = h(values[0])
for t, dt, dW in brownian_steps(times):
# Controlled state seen by the drift [eq. 10] [§3.4]
z_bar = zeta(t, z, spline(t))
if design == "LSDE":
# Drift in z_bar only, diffusion in time only [eq. 4]
drift = gamma(z_bar)
diff = sigma(t)
elif design == "LNSDE":
# Diffusion scales with the state [eq. 5]
drift = gamma(t, z_bar)
diff = sigma(t) * z
else:
# Relative form: dz = z * (gamma dt + sigma dW) [eq. 6]
drift = z * gamma(t, z_bar)
diff = z * sigma(t)
# Explicit Euler-Maruyama step, diagonal noise [app. D.4]
z = z + drift * dt + diff * dW
# Two-layer read-out of the terminal state [eq. 9]
return F(z)Notes for implementers:
- The paper does not state the Euler step size 2402.14989 app. D.4. The official code uses the smallest gap between observation times, floored at 10⁻³ neuralsde.py.
- The paper's σ is an unrestricted network with sinusoidal time encoding. The code differs in several places, including a learned sigmoid scale on the diffusion, a ReLU on the diffusion output, and tanh clipping of drift and diffusion. See Contradictions before reusing the code.
- The appendix selects a Hermite cubic with backward differences for the controlled path. The main text also names natural cubic splines 2402.14989 §2.1 2402.14989 app. F.
Contradictions
- Within the paper, LNSDE on Speech. Table 3 gives 0.923 ± 0.001, while the appendix gives 0.924 ± 0.000 for the same configuration [arxiv:2402.14989 tab. 3] 2402.14989 app. E.
- Within the paper, Mortality configuration. The proposed rows of Table 1 equal one grid cell at every level. The bold LSDE cell at 50% is 3.793, a different configuration, lower than the 3.799 reported. The text does not say which configuration Table 1 uses 2402.14989 tab. 1 2402.14989 app. E.
- Within the paper, compute cost. The text says the methods need more compute than Neural CDE models 2402.14989 §5. The appendix runtime on BasicMotions gives 89 to 96 seconds for 100 epochs for the proposed methods, against 102.5 seconds for Neural CDE 2402.14989 app. D.4. These point in different directions for that one dataset, and the paper does not reconcile them.
- Paper against code, GSDE form. The paper's GSDE is dz/z = γ dt + σ dW. The official code replaces the factor z by a tanh of the state and clips afterwards neuralsde.py. The positivity proof therefore does not describe the code's model.
- Paper against code, diffusion scale. The code multiplies the diffusion by a learned sigmoid and applies ReLU to the diffusion network. The paper specifies neither neuralsde.py neuralsde.py.
- Paper against code, model selection. The paper keeps the parameters with the lowest validation loss. The code keeps the checkpoint with the highest validation accuracy common_sde.py.
- Undefined symbols in the ablation. The caption of Table 5 does not define O and X, and the meaning used here is inferred from the text [arxiv:2402.14989 §4.3]. Read the ablation with that caveat.
Papers
- Stable Neural SDEs for irregular series (Oh et al., 2024): defines the Langevin-type, linear-noise and geometric designs, with a controlled path in the drift, stability results and a missing-data robustness bound. Tested on interpolation, classification, forecasting and 30 UEA/UCR datasets with missing values.
Source and license
Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.