Finite-difference method

Overview

A finite-difference stencil estimates a derivative from function values at a few neighbouring points, for example a difference between values at x and at x − Δx. In a physics-informed neural network (PINN) the function is the network output, so the stencil is evaluated on the network at several points and the equation residual is built from those estimates 2110.15832 §2.2 2110.15832 eq. 4.

The alternative is automatic differentiation (AD), which gives the exact slope of the network at the point itself. The paper's problem is that AD residuals are evaluated point by point, with nothing linking one point to the next. With few sample points the residual can be near zero while the solution is far from the truth 2110.15832 §1 2110.15832 fig. 1. The paper's answer is to use both: the stencil, which ties neighbouring points together, and the AD slope, which keeps the estimate tied to the point 2110.15832 §2.3.

Variants

Stencil only, numerical differentiation (n-PINN)

The derivative is taken from neighbouring network outputs alone. The paper gives three stencils for a face value, which is then differenced 2110.15832 eq. 5a–c:

• first-order upwind, n(uw1): the face value is the network output at the point;
• second-order upwind, n(uw2): the face value is 3/2 of the output at the point minus 1/2 of the output at the upwind neighbour;
• second-order central, n(cd2): the face value is the mean of the two neighbours.

The network is evaluated at x ± Δx and, for n(uw2), at x ± 2Δx. The paper says the points need not form a mesh 2110.15832 §2.2. This baseline is called n-PINN in the paper; see CAN-PINN.

Coupled AD and stencil (can-PINN)

The face value is a linear combination of the network output and its AD derivative, u_e = a1 · û(x) + a2 · û_x(x) 2110.15832 eq. 6a. Taylor expansions about the face fix a1 = 1 and a2 = Δx/2 2110.15832 eq. 7a–b. For second-order upwind the result is u_e = û(x) + (Δx/2) · û_x(x), and the derivative is the difference of the two faces over Δx 2110.15832 eq. 8a–9. The authors state the accuracy order as two 2110.15832 §2.3.1.

The paper also gives a coupled central scheme for pressure, can(cd). The face pressure is the mean of the two nodal values, corrected with the difference of the AD slopes at those nodes, scaled by Δx/8 2110.15832 eq. 12a–b. The authors report accuracy order two for it 2110.15832 eq. 14a–b. The paper's Navier-Stokes experiments use can(uw2) for convection and can(cd) for pressure 2110.15832 §3.3.

The paper derives the positive-flow branch of can(uw2) only. It does not state the negative-flow branch, nor which velocity selects the upwind side 2110.15832 eq. 8a–b.

Reference: AD only (a-PINN)

Not a finite-difference method. It is the baseline in the paper: the residual uses AD derivatives at the collocation points only 2110.15832 §2.2. The stencil variants are compared against it.

When to use which

All of these numbers are author-reported. The source has no independent reproduction.

Implementation sketch

The coupled upwind face value and the derivative built from two faces. The paper's positive-flow branch only 2110.15832 eq. 8a–9.

# Coupled face value, can(uw2), positive flow  [arxiv:2110.15832 eq. 8a]
# Inputs: u_hat and its AD slope u_x at the point, step dx
# Output: the value at the east face

def can_uw2_face(u_hat, u_x, dx):
    return u_hat + (dx / 2.0) * u_x

# Derivative from the two faces  [arxiv:2110.15832 eq. 9]
# Inputs: east and west face values, step dx

def can_derivative(u_e, u_w, dx):
    return (u_e - u_w) / dx

The west face uses the same formula at x − Δx, with the AD slope taken at that neighbour 2110.15832 eq. 8b. Each residual therefore needs the network and its AD slope at the neighbours, not only at the point. The paper reports this as its per-iteration cost 2110.15832 fig. 11c.

Contradictions

None known. The only cited paper is the source of every number on this page. Its results are author-reported and have no independent reproduction in the source.

Papers

• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method: couples AD slopes with stencils of neighbouring network outputs in a PINN residual for fluid flow.