wikillm

Paper · 2022-05-12 · cs.LG, math.DS, nlin.CD

Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems

Fits daily 2021 COVID-19 case counts with a damped harmonic oscillator embedded in a 2x32 tanh network. Compares plain NN, PINN, self-adaptive PINN, Bayesian inference (NUTS), Bayesian NN (HMC) and Bayesian PINN (HMC). PINN gives c=1.251, k=374.6, x0=0.558; Bayesian PINN gives much wider posteriors. Author-reported.

Physics-informed neural networkBayesian neural networkAdaptive loss weightingphysics-informedbayesian-inferencedynamical-systemsepidemiology

arXivPDFMarkdown10 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

The paper asks what happens when you train a neural network on real, noisy data and also tell it the rules the data should roughly follow. The data are the daily number of new COVID-19 cases worldwide during 2021. The rule is a simple model of something that swings up and down and settles, like a spring with friction. The authors assume the case curve behaves like that spring.

They compare several ways of fitting the curve. A plain neural network fits the known days well but goes flat on the days it has never seen. Adding the spring rule to the training makes the network follow the wave into the unseen days. Bayesian methods go a step further: instead of one answer they give a range of plausible answers, so you can see how sure the model is. A Bayesian version of the combined model is the most expensive and the least stable of the group.

The point is a guide for choosing a method. Extra knowledge helps, the right weighting between data and rules matters, and uncertainty ranges are useful but cost more.

TL;DR

  • A small network (two hidden layers of 32 nodes, tanh) takes time as input and predicts the daily case count. A damped-oscillator equation is added to the loss, with its constants learned along with the network. §2.2 §3.2
  • Trained on the first 225 of 365 days, the plain PINN follows the 7-day average into the last 140 days. The plain NN does not and predicts a flat line. fig. 3 §3.4
  • Bayesian inference over only three physics constants gives credible intervals and extrapolates well, but fits the training days less tightly. §4.1 §4.3
  • Bayesian PINN has about 2,312 unknowns, the widest posteriors, and the authors say it is sensitive to scaling. tab. 1 §4.3 fig. 15
  • All results are author-reported on one dataset. No official code was found.

Problem

Plain neural networks fit the training window but fail to predict beyond it. Physics-informed networks can use a known law to extrapolate, but depend on a weighting between data and physics that has to be chosen. Bayesian models give uncertainty but need many samples. The paper asks how these families compare on real, noisy, incomplete data where the true physics is not known exactly §1. Its test case is the 2021 daily COVID-19 case series §2.1 fig. 1.

Method

Data. The series is the reported daily number of new cases worldwide from 1 January to 31 December 2021, 365 days, from a public database. A seven-day moving average is used as a smoothed version §2.1 fig. 1. The paper trains on the raw series and on the smoothed series separately §2.1.

Neural network (NN). The network maps time t to case count x. It has one input, two hidden layers of 32 nodes with tanh activation, and one output. Its hidden layers follow z_1 = tanh(W_1 z_0 + b_1), z_2 = tanh(W_2 z_1 + b_2), and z_3 = W_3 z_2 + b_3, with x ≈ z_3 eq. 1 §2.2. This gives 1088 weights and 65 biases, 1153 parameters in total §2.2.

Damped harmonic oscillator (physics). The physics residual is r = ẍ + 2ζω_0 ẋ + ω_0² x = 0, with the mass m fixed to 1 eq. 2 eq. 3 §2.3 §4.1. The learned physics constants are the viscous damping c, the stiffness k, and an offset x_0 §2.3. The underdamped solution is x(t) = 2A_0 cos(ωt + φ) exp(−δt), with δ = c/(2m) and ω = √(ω_0² − δ²) eq. 6 §2.3. The paper treats the underdamped case only §2.3.

Neural network (NN) loss. Training minimises a mean squared error on the training days, L_data, over n_trn points eq. 7 eq. 9 eq. 10 §3.1. Parameters are updated with Adam §3.1.

Physics-informed neural network (PINN). The loss is a weighted sum, (1 − ε) L_data + ε L_phys eq. 12 §3.2. L_phys is the mean squared residual over the sampled days, with ẋ and ẍ obtained by automatic differentiation of the network eq. 14 §3.2. The network weights, c, k and x_0 are all trained together with Adam eq. 15 §3.2.

Pseudocode
# PINN loss for one step  [§3.2] [eq. 12]
# Input:  times (n,); observed cases for the first n_trn of them
# Output: scalar loss; gradients flow to theta, c, k, x0

def pinn_loss(theta, c, k, x0, eps, times, observed):
    # Network output and its time derivatives by autodiff  [§3.2]
    x = network(theta, times)
    x_dot = d(x) / d(times)
    x_ddot = d(x_dot) / d(times)
    # Mean squared error on the training days  [eq. 13]
    loss_data = mean((x[:n_trn] - observed) ** 2)
    # Mean squared residual on all sampled days, with m = 1  [eq. 14] [§2.3]
    residual = x_ddot + c * x_dot + k * (x - x0)
    loss_phys = mean(residual ** 2)
    # Weighted sum of data and physics  [eq. 12]
    return (1 - eps) * loss_data + eps * loss_phys

Self-adaptive PINN (SAPINN). The fixed weight ε is replaced by a function ε(t) that is learned together with the other parameters eq. 16 eq. 19 §3.3. The paper cites McClenny and Braga-Neto (2020) for this idea §3.3. How ε(t) is parametrised and initialised is not stated.

Bayesian inference (BI). Only c, k and x_0 are inferred. The likelihood is a product of daily normal terms with the model curve as mean eq. 21 eq. 25 §4.1. The noise width σ has a half-Cauchy prior with β = 1.0 §4.1 eq. 26. The three constants have independent log-normal priors: log c ~ N(log 2.2, 0.5), log k ~ N(log 350, 0.5), log x_0 ~ N(log 0.56, 0.5) eq. 27 §4.1. Posterior sampling uses the NUTS sampler in PyMC3 with two chains, 2000 tuning samples, 4000 kept samples and target acceptance 0.85 §4.1.

Bayesian neural network (BNN). The NN weights get independent zero-mean normal priors with standard deviation 2.0 eq. 31 §4.2. The daily likelihood width is fixed at σ = 0.05 eq. 30 §4.2. Posterior sampling uses Hamiltonian Monte Carlo through TensorFlow Probability, with 3000 tuning and 3000 kept samples §4.2.

Bayesian PINN (BPINN). The network weights, c, k and x_0 share one joint posterior eq. 32 eq. 33 eq. 34. The priors are the BI priors for the physics constants and the BNN priors for the weights eq. 35 §4.3. Sampling uses HMC with 50 leapfrog steps and an initial step size of 5·10⁻⁴, with 3000 samples for tuning and 3000 kept §4.3.

Scaling. The paper reports that rescaling the physics equation so that the coefficient on x becomes unity improves the BPINN a lot fig. 15 §5. The rescaled form given in that caption is ẍ/k + c ẋ/k + (x − x_0) fig. 15.

Hyperparameters

NameValueWhereNotes
Input / outputtime t → daily cases x (1 → 1)§2.2 eq. 1time unit and data scaling not stated
Hidden layers2 × 32 nodes§2.2 fig. 5chosen from a grid over 1–2 layers, 2–32 nodes
Activationtanheq. 1 §2.2
NN parameters1088 weights, 65 biases (1153)§2.2
Mass m1 (fixed)§2.3 §4.1§3.3 lists m as a parameter; see Pitfalls
Optimizer (NN, PINN, SAPINN)Adam§3.1 §3.2 §3.3learning rate, betas, batch size: not stated
Training epochs (PINN)60K§3.2 fig. 8parameters "converge after 60K"
PINN weight ε10⁻³§3.4 fig. 8picked by reading fig. 7 and fig. 6
SAPINN weight ε(t)learned§3.3mean 0.007, SD 0.003 after training
Training windowfirst 225 of 365 days (140 predictive days)§2.1 fig. 3 fig. 8
Sweep of ε (figures)10⁻∞, 10⁻⁵, 10⁻⁴, 10⁻³, 10⁻², 10⁰fig. 6
Sweep of training size150, 175, 200, 225, 250, 275 (and 300 in fig. 13)fig. 10 fig. 13
BI samplerNUTS (PyMC3), 2 chains§4.1tune 2000, keep 4000, target accept 0.85
BI priorslog c ~ N(log 2.2, 0.5); log k ~ N(log 350, 0.5); log x_0 ~ N(log 0.56, 0.5)eq. 27
BI likelihood width σhalf-Cauchy, β = 1.0§4.1 eq. 26
BNN likelihood width σ0.05§4.2 eq. 30
BNN priorsW, b ~ N(0, 2.0)eq. 31 §4.2
BNN samplerHMC (TensorFlow Probability), 3000 tune, 3000 kept§4.2leapfrog steps and step size: not stated
BPINN samplerHMC, 50 leapfrog steps, step size 5·10⁻⁴§4.33000 tune, 3000 kept; daily σ: not stated
Hardware, run time, seedsnot stated

Results

Author-reported. Setup: one public series (worldwide daily COVID-19 cases, 2021, 365 days), one physics model, training on the first 225 days, predictions for the remaining 140 days. Baselines are the other methods in the paper, not external work §2.1 §3.4 §4.4.

ModelFitted physics constants (learned)MetricSource
PINN, ε = 10⁻³, 60K epochsc = 1.251, k = 374.6, x_0 = 0.558δ = 0.626, ω = 19.35, T = 0.325 yrfig. 8 §3.2
BI (daily data)c = 1.111 ± 0.126, k = 402.4 ± 1.7, x_0 = 0.541 ± 0.002δ = 0.556, ω = 20.06, T = 0.313 yrfig. 11 §4.1
BPINN (daily data)c = 1.312 ± 0.391, k = 319.4 ± 38.6, x_0 = 0.571 ± 0.020δ = 0.65, ω = 17.87, T = 0.352 yrfig. 12 §4.3
SAPINN, learned weightε(t): mean 0.007 ± 0.003not stated§3.3
Bayesian PINN vs BIdamping +18%, stiffness −21%, offset +6%SD ratio: damping ×3, stiffness ×23, offset ×10§4.3

Qualitative results, read from figures; no numeric error table is given:

  • Plain NN fits training days well and predicts a flat line after day 225 fig. 3 fig. 14 §3.4.
  • PINN and BI stay close to the 7-day average across the whole year; BI intervals are wider when trained on raw daily data than on the average fig. 3 §4.4.
  • The predictive data loss falls as training size grows for PINN, SAPINN and BPINN; BPINN improves only slightly with more data fig. 13.
  • Comparison of model families and their unknown counts is summarised in tab. 1.

Parameter counts from tab. 1: NN 1153, PINN 1156, SAPINN 1157, BI 6, BNN 2306, BPINN 2312.

Paper vs code

No official code.

Searches: (1) the arXiv TeX source has no code link; its only GitHub URL is the Johns Hopkins case-data repository §2.1. (2) GitHub repository search for "BPINN" returns only unrelated third-party repositories. (3) An org-scoped search for "bpinn" in the Karniadakis group's organisation (PredictiveIntelligenceLab) returns none. (4) A keyword search for COVID, damped oscillator and PINN returns none. I did not check the Kuhl group's GitHub, and a general web search was not available in this run.

Pitfalls

  • Amplitude and frequency in the BI model. The analytic form used for BI is x = x_0 + A_0 cos((k − c²/2) t) exp(−c t / 2) eq. 24. The frequency term differs from the ω = √(ω_0² − δ²) form in eq. 6 (which, with m = 1 and δ = c/2, is √(k − c²/4)). Also, eq. 6 uses 2A_0 while eq. 24 uses A_0. The paper does not say which form the BI code uses. Re-derive or test against the figures before reusing either.
  • Amplitude A_0 is not inferred. A_0 appears in eq. 24 but is not in the parameter set {c, k, x_0} §4.1 eq. 23, and no value is given for it. Its handling is not stated.
  • Mass m. Section 3.3 lists ϑ = {m, c, k, x_0} for SAPINN, but elsewhere m = 1 is fixed §2.3 §4.1. Treat m as fixed.
  • Parameter count typo. Section 4.2 gives 1053 × 2 = 2306 unknowns for the BNN, but 1153 × 2 = 2306 is intended §2.2 tab. 1.
  • Window sizes disagree. The loss breakdown in fig. 7 uses 255 descriptive days (1/255 sum) and days 256–365 for prediction (1/140 sum). Elsewhere the training window is 225 days fig. 3 §3.4. The two sets of numbers do not match.
  • Time and data scale are not stated. Periods are reported in years fig. 8 §3.2, but the paper does not say how t is scaled. The fitted offset x_0 ≈ 0.56 implies the case counts were scaled before training, but the scaling is not given.
  • The PINN loss weight is chosen by eye. ε = 10⁻³ is read off the figures §3.4 fig. 7 and not tuned on held-out data. Results change strongly with ε fig. 6.
  • Scaling sensitivity. The paper says PINN and BPINN depend on how the physics equation is scaled fig. 15 §5. The scaling used for the reported BPINN results is not stated. The rescaled form given in fig. 15 is the only one shown.
  • BPINN noise width. The daily likelihood width σ for the BPINN is not stated. The BNN uses σ = 0.05 eq. 30.
  • HMC settings. The BNN leapfrog steps and step size are not stated. The BPINN description says both burn-in and total samples are 3000 and also says first 3000 tune, next 3000 estimate; how these combine is unclear §4.3.
  • Learning rate and schedule. Adam is used §3.1 §3.2, but the learning rate, schedule and random seeds are not stated. The PINN curve is shown for one run only.
  • Uncertainty and cost. BI credible intervals depend on the chosen priors and on σ, which the paper treats as a free parameter with a half-Cauchy prior §4.1. BPINN is the most expensive method and needs a large training set tab. 1. The paper calls it "less robust" §4.3.
  • One series, one physics model. Conclusions rest on one 365-day series and one oscillator model. The authors state that they have only demonstrated this on one problem §5.

Lineage

  • Builds on: physics-informed neural networks (Raissi, Perdikaris and Karniadakis, 2019) §1; the B-PINN work of Yang, Meng and Karniadakis (2021) for the BNN priors and HMC setup §4.2; self-adaptive PINNs (McClenny and Braga-Neto, 2020) for the learned weighting §3.3.
  • Compared against: plain NN, PINN, SAPINN, classical Bayesian inference, BNN and BPINN, all within this paper §3.4 §4.4 tab. 1.
  • Superseded by: not stated.

Source and license

Adapted from “Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems” by Kevin Linka, Amelie Schafer, Xuhui Meng, Zongren Zou, George Em Karniadakis, Ellen Kuhl (arXiv:2205.08304), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.