---
kind: concept
slug: finite-difference-method
title: Finite-difference method
summary: >-
  Estimating a derivative from function values at neighbouring points instead of the exact slope at one point. In physics-informed networks, stencils are applied to network outputs. Coupling a stencil with the automatic-differentiation slope at the point is the variant reported as more accurate with sparse samples, at higher cost per iteration. Numbers are author-reported.
tags: [numerical-methods, pde, physics-informed, numerical-differentiation, stencil]
papers: ["2110.15832"]
related: [physics-informed-neural-network, automatic-differentiation, finite-element-method]
status: draft
updated_at: 2026-11-01
---

## Overview

A finite-difference stencil estimates a derivative from function values at a
few neighbouring points, for example a difference between values at x and at
x − Δx. In a physics-informed neural network (PINN) the function is the
network output, so the stencil is evaluated on the network at several points
and the equation residual is built from those estimates [arxiv:2110.15832 §2.2]
[arxiv:2110.15832 eq. 4].

The alternative is automatic differentiation (AD), which gives the exact slope
of the network at the point itself. The paper's problem is that AD residuals
are evaluated point by point, with nothing linking one point to the next. With
few sample points the residual can be near zero while the solution is far from
the truth [arxiv:2110.15832 §1] [arxiv:2110.15832 fig. 1]. The paper's answer
is to use both: the stencil, which ties neighbouring points together, and the
AD slope, which keeps the estimate tied to the point [arxiv:2110.15832 §2.3].

## Variants

### Stencil only, numerical differentiation (n-PINN)

The derivative is taken from neighbouring network outputs alone. The paper
gives three stencils for a face value, which is then differenced
[arxiv:2110.15832 eq. 5a–c]:

- first-order upwind, n(uw1): the face value is the network output at the point;
- second-order upwind, n(uw2): the face value is 3/2 of the output at the point minus 1/2 of the output at the upwind neighbour;
- second-order central, n(cd2): the face value is the mean of the two neighbours.

The network is evaluated at x ± Δx and, for n(uw2), at x ± 2Δx. The paper says
the points need not form a mesh [arxiv:2110.15832 §2.2]. This baseline is
called n-PINN in the paper; see [CAN-PINN](/papers/2110.15832).

### Coupled AD and stencil (can-PINN)

The face value is a linear combination of the network output and its AD
derivative, u_e = a1 · û(x) + a2 · û_x(x) [arxiv:2110.15832 eq. 6a]. Taylor
expansions about the face fix a1 = 1 and a2 = Δx/2 [arxiv:2110.15832 eq. 7a–b].
For second-order upwind the result is u_e = û(x) + (Δx/2) · û_x(x), and the
derivative is the difference of the two faces over Δx [arxiv:2110.15832 eq. 8a–9].
The authors state the accuracy order as two [arxiv:2110.15832 §2.3.1].

The paper also gives a coupled central scheme for pressure, can(cd). The face
pressure is the mean of the two nodal values, corrected with the difference of
the AD slopes at those nodes, scaled by Δx/8 [arxiv:2110.15832 eq. 12a–b]. The
authors report accuracy order two for it [arxiv:2110.15832 eq. 14a–b]. The
paper's Navier-Stokes experiments use can(uw2) for convection and can(cd) for
pressure [arxiv:2110.15832 §3.3].

The paper derives the positive-flow branch of can(uw2) only. It does not state
the negative-flow branch, nor which velocity selects the upwind side
[arxiv:2110.15832 eq. 8a–b].

### Reference: AD only (a-PINN)

Not a finite-difference method. It is the baseline in the paper: the
residual uses AD derivatives at the collocation points only
[arxiv:2110.15832 §2.2]. The stencil variants are compared against it.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Few collocation points, where AD-only residuals are not enough | Coupled can-PINN | On the lid-driven cavity at 2,601 points, the paper reports about two orders of magnitude lower error than AD-only, and about one order lower than n-PINN [arxiv:2110.15832 §3.3.1] |
| Dense collocation points (40,401 in the paper's cavity study) | Coupled can-PINN, if its higher cost is affordable | At 40,401 points the paper reports AD-only at about 2e-4 and can-PINN below 1e-5 [arxiv:2110.15832 §3.3.2]. The paper's summary of this study does not give an n-PINN figure at this size |
| Tight compute per PINN (about 4 CPUs) | Stencil-only n-PINN | It is the fastest per iteration and can-PINN the slowest. With over 20 CPUs the times are similar [arxiv:2110.15832 §3.3.2] [arxiv:2110.15832 fig. 11c] |
| Need for smaller dispersion error | can(uw2) or can(cd) over first- and second-order upwind | The authors' Fourier analysis finds smaller dispersion error for both can schemes. can(cd) is reported as non-dissipative and can(uw2) as less dissipative [arxiv:2110.15832 §2.4] [arxiv:2110.15832 fig. 3] |
| Flow-mixing problem where variance across runs matters | can(uw2) | The paper reports lower variance across runs for can(uw2) than for can(cd) [arxiv:2110.15832 §3.2.1] |
| Inverse problem with few observations | Coupled can-PINN | Reported as better than AD-only, with the inferred Reynolds number within about 10% at Re = 400 [arxiv:2110.15832 §3.3.3] |

All of these numbers are author-reported. The source has no independent reproduction.

## Implementation sketch

The coupled upwind face value and the derivative built from two faces. The
paper's positive-flow branch only [arxiv:2110.15832 eq. 8a–9].

```pseudocode
# Coupled face value, can(uw2), positive flow  [arxiv:2110.15832 eq. 8a]
# Inputs: u_hat and its AD slope u_x at the point, step dx
# Output: the value at the east face

def can_uw2_face(u_hat, u_x, dx):
    return u_hat + (dx / 2.0) * u_x

# Derivative from the two faces  [arxiv:2110.15832 eq. 9]
# Inputs: east and west face values, step dx

def can_derivative(u_e, u_w, dx):
    return (u_e - u_w) / dx
```

The west face uses the same formula at x − Δx, with the AD slope taken at
that neighbour [arxiv:2110.15832 eq. 8b]. Each residual therefore needs the
network and its AD slope at the neighbours, not only at the point. The paper
reports this as its per-iteration cost [arxiv:2110.15832 fig. 11c].

## Contradictions

None known. The only cited paper is the source of every number on this page.
Its results are author-reported and have no independent reproduction in the
source.

## Papers

- [CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method](/papers/2110.15832): couples AD slopes with stencils of neighbouring network outputs in a PINN residual for fluid flow.