CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method

In plain words

A physics-informed neural network learns a flow field by penalising it when it breaks the governing equations at a set of sample points. The usual way to measure how much an equation is broken is automatic differentiation, which gives exact slopes at each point. When the points are few, the network can make those slopes match at every point and still be wrong between them, so the answer looks fine to the training loss but is far from the true flow.

This paper's idea is to also look at the neighbours of each point, as classic numerical methods do with finite differences. The slope is then estimated from nearby values, which ties neighbouring points together. The authors combine the two: the neighbour-based estimate and the exact slope at the point are mixed so the result is more accurate than the neighbour estimate alone. They test this on fluid flow problems, mainly the incompressible Navier-Stokes equations, and report that it keeps working with few sample points where the usual approach fails.

TL;DR

• Standard PINNs compute derivatives of the PDE residual with automatic differentiation (AD) at each collocation point and need many points.
• CAN-PINN adds derivative terms built from neighbouring network outputs (numerical differentiation, ND) and couples them to the AD slope at the point §2.3 eq. 6a.
• Two schemes are derived for the Navier-Stokes equations: can(uw2) for convection and can(cd) for pressure gradients §2.3.1 §2.3.2.
• Author-reported: lower error than AD-only and ND-only PINNs on several flow problems. With few CPUs its per-iteration cost is the highest of the three §3 fig. 11c.
• Only the lid-driven cavity forward problem has released code [Paper vs code].

Problem

AD-based PINN losses evaluate each PDE residual exactly at the collocation points, and nothing ties one point to the next. With sparse points the network can drive the residual to a very small value while its solution is far from the truth §1 §2.2 fig. 1. The authors report that such AD-only PINNs (a-PINNs) need dense sampling and long training, and that their training loss can be misleading §2.2 §3.1.1 §3.2.1.

Method

Overview. The PINN is a fully connected network û(x, t; w), trained by minimising a weighted sum of PDE, boundary and initial-condition losses §2.1 eq. 1a–c eq. 2a–e. The Navier-Stokes instantiation uses the velocity components u, v and pressure p as network outputs §3.3 eq. 27a–c.

Input features (all PINN variants). Inputs (x, t) are first mapped through a sinusoidal layer γ(v) = sin(2π(Wv + b)) §2.1 eq. 3. Weights W are sampled from a normal distribution N(0, σ²) with σ = 1, and b starts at zero [tab. 1, note 2]. Hidden layers use sine activations and He-uniform initialisation; the output layer is linear tab. 1, note 2.

Numerical-differentiation (n-PINN) baseline. The first derivative of u at a point is written as a difference between face values at x ± Δx/2 eq. 4. Three stencils are given eq. 5a–c:

• 1st-order upwind, n(uw1): face value û(x) eq. 5a;
• 2nd-order upwind, n(uw2): face value 3/2 û(x) − 1/2 û(x − Δx) eq. 5b;
• 2nd-order central, n(cd2): face value (û(x + Δx) + û(x − Δx)) / 2 eq. 5c.

Derivative terms are evaluated from the network at neighbouring points, so the network is called at x ± Δx and at x ± 2Δx §2.2 eq. 5b. The points are not required to form a mesh §2.2.

Coupling step (can-PINN). The face value is a linear combination of the network output and its AD derivative: u_e|can = a1 · û(x) + a2 · û_x(x) eq. 6a. Taylor expansions of û(x) and û_x(x) about the face eq. 6b, 6c give a1 = 1 and a2 = Δx/2 eq. 7a, 7b. Thus u_e|can(uw2) = û(x) + (Δx/2) û_x(x) eq. 8a, and u_w|can(uw2) = û(x − Δx) + (Δx/2) û_x(x − Δx) eq. 8b. The derivative is then (u_e − u_w)/Δx eq. 9 and, with the modified-equation expansion, carries a leading error term of order Δx² eq. 10 eq. 11. The authors state the theoretical accuracy order as two §2.3.1.

Central pressure gradient, can(cd). Central differencing of pressure alone lets velocity and pressure decouple on collocated grids §2.3.2. The face pressure is therefore corrected with the AD slope at the centre and at the neighbour eq. 12a, 12b:

# Face pressure, can(cd)  [§2.3.2] [eq. 12a] [eq. 12b]
# Inputs: p and p_x (AD) at x, and at its east neighbour x + dx
# Output: p_e, the pressure at the east face

def face_pressure_east(p, p_x, p_east, p_x_east, dx):
    # Average of the two nodal values  [eq. 12a]
    average = (p + p_east) / 2.0
    # Correction with the difference of AD slopes, scaled by dx/8  [eq. 12a]
    correction = (dx / 8.0) * (p_x_east - p_x)
    return average - correction

The gradient is then (p_e − p_w)/Δx eq. 13. The authors give the same form for the west face eq. 12b. Modified-equation analysis gives accuracy order two eq. 14a, 14b.

Convection and pressure used in the N-S experiments. Convection uses can(uw2) and pressure uses can(cd); the viscous terms use central second differences from neighbouring values §3.3 §2.3. The paper calls this can(uw2-conv, cd-p). It also trains a variant with plain central pressure differences, can(uw2-conv), to show the effect of the can pressure scheme §3.3.

Dispersion and dissipation. The authors compare the wavenumber error of can(uw2) and can(cd) with first- and second-order upwind schemes using a Fourier analysis §2.4 eq. 15a–20b fig. 3. Their reading: both can schemes have smaller dispersion error, can(uw2) has smaller dissipation, and can(cd) is non-dissipative §2.4 fig. 3.

Sampling and boundaries. Collocation points are sampled from the domain, for example on an equispaced grid, by Latin hypercube or by importance sampling §2.1. Boundary and initial conditions enter through the losses in eq. 2d and 2e §2.1.

Hyperparameters

Values for the lid-driven cavity (LDC), the case with released code, first. Other rows come from tab. 1 and the text.

Results

Author-reported. Setups below are the paper's own; no independent reproduction is in the source.

Paper vs code

The paper names no code link. The official repository was found as chiuph/CAN-PINN: its description matches the paper's title and its owner name matches the first author's handle. Its README gives A*STAR as copyright holder and cites this paper. Checked at commit a92594e623cfa7260efa88019a27c53a1cd664dd.

• Scope. The repository has three notebooks for forward LDC at Re = 400: d01 can-PINN, d02 n-PINN and d03 a-PINN. No code for the ODE, flow-mixing, backward-facing-step, channel or inverse experiments is in the repository d01_can-pinn_lid_driven_cavity_re400.ipynb d02_n-pinn_lid_driven_cavity_re400.ipynb d03_a-pinn_lid_driven_cavity_re400.ipynb.
• Hyperparameters match Table 1 for LDC. Re = 400, 51 × 51 grid, batch 475 + 0 + 25, learning rate 1e-3, λ = 1, min learning rate 5e-6, 2,000 epochs × 100 batches d01_can-pinn_lid_driven_cavity_re400.ipynb d01_can-pinn_lid_driven_cavity_re400.ipynb d01_can-pinn_lid_driven_cavity_re400.ipynb. The same values appear in d02 and d03.
• Update count. Table 1, note 4 says the PINN weights are updated every 100 iterations. The code calls one Adam step per mini-batch; the sampler reports 100 batches per epoch and training runs 2,000 epochs, so 200,000 steps d01_can-pinn_lid_driven_cavity_re400.ipynb d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Weight initialisation. The paper says N(0, σ²); the code uses a truncated normal with stddev σ = 1 d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Momentum residual. The paper writes the momentum equations in conservative form eq. 27b. The code's momentum residual adds a term −u · div, where div is the discrete divergence from face velocities, and the same term in v d01_can-pinn_lid_driven_cavity_re400.ipynb. The paper has no such term.
• Negative face velocity. The paper derives only the positive-flow branch of can(uw2) eq. 8a, 8b. The code uses a mirror-image formula when the face velocity is negative d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Upwind switch. The paper does not say which velocity selects the upwind side. The code uses the average of the two velocities at the face d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Loss form. The paper's loss is a weighted sum with λ_BC and λ_IC eq. 2a. The code adds the PDE loss divided by λ and the boundary loss with no separate weight d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Continuity and viscous terms. Central differences, as the paper states §3.3 d01_can-pinn_lid_driven_cavity_re400.ipynb. No discrepancy found.
• Pressure gradient. The can(cd) correction matches eq. 12a and 12b in form d01_can-pinn_lid_driven_cavity_re400.ipynb. No discrepancy found.
• Logged result. The saved output of the can-PINN run ends at training loss 8.11e-06 after 2,000 epochs d01_can-pinn_lid_driven_cavity_re400.ipynb. The notebook does not log the MSE against the ground truth.
• Dependencies. The README badges name Python 3.8 and TensorFlow 2.4 README.md. The notebooks use the tf.compat.v1 graph API d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Licence. No LICENSE file exists. The README states "All rights reserved" README.md. Cite the code by path and line only; do not copy snippets.

Pitfalls

• AD-only loss can mislead. With sparse points the PDE residual can be near zero while the solution is wrong. Do not judge convergence by training loss alone §1 fig. 1 §3.2.1 §3.3.1.
• Upwind branches. The positive-flow face formulas are in the paper eq. 8a, 8b. The negative branch and the choice of which velocity sets the side are not stated. Use the code's mirror formula and face-average rule, and say so in the implementation d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Boundary samples inside the batch. The sampler draws 475 points from all 2,601, including boundary points. The residual drops boundary points, so a batch yields fewer than 475 PDE residuals d01_can-pinn_lid_driven_cavity_re400.ipynb d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Corner double-count. Left and bottom boundary terms both include the bottom-left corner, so that corner counts twice in the boundary loss d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Pressure is unconstrained. The boundary loss enforces only u and v. The paper states no pressure condition, and the code adds none, so pressure is fixed only up to a constant §3.3 d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Cost. Each can-PINN residual evaluates the network at eight neighbours (x ± Δx, x ± 2Δx, and the same in y) and takes AD slopes at the neighbours. The authors report it as the slowest per iteration with few CPUs d01_can-pinn_lid_driven_cavity_re400.ipynb fig. 11c.
• Inverse-problem weights. The text states the data-loss to PDE-loss weight as 100 to 1. It gives no boundary-loss weight for that experiment §3.3.3.
• Learning-rate schedule. Factor and patience are not stated in the paper. The code uses factor 0.5 and patience 50 epochs, which is 5,000 steps d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Sampling density. At 2,601 points a-PINN fails while can-PINN works. At 40,401 points a-PINN reaches about 2e-4 and can-PINN below 1e-5. Larger sets need more iterations and larger batches, so the cost rises with the point count §3.3.2 tab. 2.
• Code is forward only. The notebooks are not a reproduction of the inverse or multi-problem results.

Lineage

• Builds on: finite-difference and multi-moment schemes the paper cites §2.3 §2.3.1; physics-informed neural networks [2]; sinusoidal features [37].
• Compared against: AD-based PINNs (a-PINN) and numerical-differentiation PINNs (n-PINN) with 1st-order upwind, 2nd-order upwind and central stencils §2.2 eq. 5a–c.
• Superseded by: not stated in this card's sources.