Physics-informed neural network

Overview

A physics-informed neural network (PINN) is a network trained to satisfy a differential equation. Its loss is a weighted sum of terms: the equation residual at collocation points, the initial and boundary conditions, and any measured data 2307.00379 §2.1 2307.00379 eq. 6. Each term is a mean over its points 2307.00379 eq. 10. The weakness that the reweighting variants respond to is that a plain mean lets a small region with large residuals disappear into the average while the total loss keeps falling. That can hurt multiscale problems and can block information from reaching the interior of the domain from the boundary 2307.00379 §2.2.

A 2024 study by the same authors gives a second account of the stall. With a quadratic loss, the batch-averaged gradient can point in a steady direction while individual point gradients cancel. Some points then stay unresolved while the loss looks settled 2403.18494 §2.1 2403.18494 §2.3. Its remedy is a reweighting of the same kind, described below.

A third response leaves the weights alone and changes how the derivatives in the residual are estimated. With sparse points the network can match the autodiff slopes at every point and still be wrong between them. CAN-PINN mixes each autodiff slope with finite-difference-style values from neighbouring points 2110.15832 §1 2110.15832 §2.3.

Most variants share one invariant: a physics penalty is added to a training loss, and the network weights take gradient steps on it. These variants differ in three choices: how the penalty is formed (a derivative of the network output at sample points, a derivative built from neighbouring network values, or a discrete operator on a mesh), how the terms or points are weighted, and how boundary conditions are enforced. One family does not use a penalty at all. It puts the law into the output: the network predicts an energy, and the stress is that energy's derivative, so the law holds whatever the weights are 2005.12183 §3.3. Architecture and input encoding are a further, separate choice. An optional layer attaches error bars to the prediction, which a plain PINN does not provide. It treats the network weights as random and averages over posterior samples or over an ensemble 2201.07766 §3.1 2201.07766 §4.1 2201.07766 §4.2.

The name is used loosely. The survey that classifies informed machine learning lists physics-informed deep learning, physics-guided neural networks and semantic-based regularisation as overlapping names for related methods 1903.12394 §1. It finds that scientific knowledge is mostly given as equations or simulation results, and that equations most often change the learning algorithm, which here means the loss 1903.12394 §3.2.2 1903.12394 §3.2.3.

Variants

Fixed global weights (vanilla PINN)

One constant weight per loss term, set before training. The reference setup for 1D Allen-Cahn uses 100 on the initial-condition term and 1 on the PDE residual 2307.00379 §3.1. The weights do not respond to the errors. The card's comparison tables give the vanilla baseline as 4.98e-1 relative L2 on Allen-Cahn and 8.14e-2 on 2D Helmholtz. The authors took these baseline values from the literature 2307.00379 tab. 1 2307.00379 tab. 3. On Allen-Cahn, the vanilla network converges only with a scheduled learning rate 2307.00379 §3.4. Card: Residual-based attention in physics-informed neural networks.

Residual-based attention (RBA) on the PDE residual

Each collocation point keeps a multiplier. The multiplier decays by a factor gamma each step and adds the point's residual, divided by the largest residual magnitude in the current set, scaled by a learning rate eta* 2307.00379 eq. 12. The normalisation keeps each new term in [0, 1]. The multipliers are state: they are held constant in the loss and are never trained by gradient descent, so the method needs no auxiliary network 2307.00379 §2.2. With gamma below 1, the multipliers stay below eta* / (1 - gamma) 2307.00379 eq. 13.

Reported results, all author-reported and five-seed means:

• 1D Allen-Cahn: 5.8e-5 relative L2, against 1.11e-4 for self-adaptive weights re-run by the authors 2307.00379 tab. 1.
• 2D Helmholtz with Fourier features: 8.21e-6, against 4.80e-5 for self-adaptive weights 2307.00379 tab. 3.
• Early convergence about one order of magnitude faster than the vanilla PINN, even without the global loss weights 2307.00379 §3.4.

The ablations are less favourable. On Allen-Cahn, mMLP with Fourier features and no RBA gives 1.0e-4, the same as RBA with Fourier features 2307.00379 tab. 2. On Helmholtz, mMLP with Fourier features and no RBA gives 1.50e-5, and RBA with Fourier features gives 5.57e-5 2307.00379 tab. 4. These ablation rows are single-seed, so the effect of RBA alone is not settled by this card. Card: Residual-based attention in physics-informed neural networks.

RBA with the decay tied to the rate (gamma = 1 - eta*)

The 2024 study uses the same normalised moving average but sets the decay to gamma = 1 - eta* 2403.18494 §2.5 2403.18494 eq. 26. The multipliers then stay in [0, 1], so the weighted residual term is never larger than the unweighted one. The released Allen-Cahn and cavity code uses eta* = 0.001 and gamma = 0.999 AC.py. The initial-condition term keeps weight 1 in the 2024 Allen-Cahn setup 2403.18494 eq. 25.

Reported results are read from figures, from single runs:

• Allen-Cahn: 10% relative L2 error in about 10k iterations with RBA, against about 100k for the vanilla PINN 2403.18494 fig. 4.
• Lid-driven cavity at Re = 1000: vanilla training stalls near 60% error, and RBA reaches about 6% 2403.18494 fig. 5.
• Burgers: the paper reports vanilla as the best of the two, with RBA slightly worse 2403.18494 app. C.

The paper reports no seed spread, so treat these as single-run results. Card: Learning in PINNs: Phase transition, diffusion equilibrium, and generalization.

RBA with shared multipliers for inverse problems with mini-batches

For the 3D Navier-Stokes inverse problem, reconstructing a flow from particle tracking velocimetry (PTV) data, the authors give each data point its own multiplier but share one multiplier per training group for boundary and collocation points. Training uses ordered mini-batches of one point per group 2307.00379 §3.3 2307.00379 fig. 6. The validation errors are lower than for self-adaptive weights and for global weights alone, but the split and iteration count are inconsistent in the paper (see Pitfalls). This setting is paper-only: the repository has no code or data for it 2307.00379 §3.3.

2307.00379 tab. 5

Exact Dirichlet conditions through an approximate distance function

The solution is written as a fixed term that matches the boundary data plus a network term multiplied by a function that vanishes on the boundary 2307.00379 §2.3.3 2307.00379 eq. 20 2307.00379 eq. 21. The boundary is then satisfied by construction and its loss term is dropped. On Helmholtz with the distance function (x^2 - 1)(y^2 - 1), the RBA result is 7.73e-5 relative L2, against 8.21e-6 with the Fourier variant 2307.00379 tab. 3 2307.00379 eq. 32. The two runs differ in boundary handling as well as in encoding, so the gap does not isolate either.

Exact periodic conditions through a period-matched Fourier embedding

Input coordinates are mapped through sines and cosines whose period equals the domain length, which makes the periodicity exact 2307.00379 §2.3.3 2307.00379 eq. 22. The embedding details and the Helmholtz truncation are in Fourier features.

Energy potential with stress as its derivative (thermodynamics-based network)

No penalty is added here. The network predicts a stored-energy function and the changes of a few internal variables. The stress and the dissipation are then derivatives of that energy, computed by automatic differentiation, so the first law holds by construction 2005.12183 §3.3 2005.12183 eq. 13. The second law is not built in. It holds in the examples because the training data have non-negative dissipation, and consistency on unseen inputs is shown only by those examples 2005.12183 §5.1.2 2005.12183 §5.1.3. The temperature sub-network can be dropped for isothermal processes, which the reported cases are 2005.12183 §3.3 2005.12183 fig. 7b.

Because the stress is a derivative of the network, the loss compares derivatives, so the activation's second derivative must not vanish. The authors show this with a toy map from x to x² and 2x. On that toy problem, ReLU applied to z ends at an error figure L of 1521.2 × 10⁻⁴, while the energy network's activation (written ELU_{z²} in the paper) ends at 0.13 × 10⁻⁴ 2005.12183 §3.2 2005.12183 tab. 2. The reported elasto-plastic results are author-reported and use data generated from the analytic reference model, not measurements 2005.12183 §4.2. A standard ANN with a matched hyperparameter count gives the wrong energy and dissipation on the 1D cases, and generalises worse on cyclic and random paths 2005.12183 §5.1.4. In 3D, the network agrees with the reference model for uniaxial cyclic loading across increment sizes, with less accuracy at the largest increments. The authors also report larger errors for it in the mean and deviatoric stress 2005.12183 §5.2.2 2005.12183 §5.2.3. Card: Thermodynamics-based Artificial Neural Networks for constitutive modeling.

Discrete equilibrium penalty on a graph network

This variant does not differentiate the network output. A graph neural network predicts the stress at every node of a periodic micro-structure mesh. The training loss adds, to a normalised mean-squared error on the stress, a weight times the mean squared discrete divergence of the predicted stress. The divergence is built from the finite-element shape-function derivatives, so it acts on the mesh operator, not on autodiff gradients 2507.05291 §3.3 2507.05291 eq. 41 2507.05291 eq. 42. Hole and outer-boundary nodes contribute zero to the penalty 2507.05291 §4.3. Periodicity is built into the graph, as extra edges between opposite faces, and not as a loss term 2507.05291 §3.1 2507.05291 fig. 2. The paper states that enforcing equilibrium on GNN stress predictions was unresolved before this work 2507.05291 §1.3.

Reported results, author-reported and single values with no spread across seeds:

• Linear elastic mesh: NMSE 1.08e-2, against 1.13e-2 for the same network without the penalty. Mean divergence 3.23e-4, against 1.11e-3 for that network 2507.05291 tab. 1.
• Hyperelastic mesh: NMSE 2.18e-2, against 5.36e-2 for the network without the penalty 2507.05291 tab. 2.
• Inference on the hyperelastic set is about 500 times faster than finite elements on CPU. The comparison is GPU inference against CPU finite elements and excludes training 2507.05291 §5.3.1.

The linear NMSE gain is small. Card: Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity.

Coupled automatic and numerical differentiation (CAN-PINN)

The penalty is still a residual at sample points, but each derivative in it is built differently. A face value is a linear mix of the network value and its autodiff slope. The mixing weights come from Taylor expansions about the face, which give a1 = 1 and a2 = Δx/2 2110.15832 eq. 6a 2110.15832 eq. 7a 2110.15832 eq. 7b. For the upwind convection term, the paper's second-order scheme, can(uw2), takes the east face value as û(x) plus Δx/2 times the slope at x 2110.15832 eq. 8a. The pressure gradient uses a central difference, can(cd), whose face values are corrected with the autodiff slopes at the centre and at the neighbour 2110.15832 §2.3.2 2110.15832 eq. 12a. In a Fourier analysis against first- and second-order upwind schemes, both can schemes have smaller dispersion error. can(uw2) has smaller dissipation than those upwind schemes, and can(cd) is non-dissipative 2110.15832 §2.4 2110.15832 fig. 3.

Reported results, all author-reported:

• Lid-driven cavity, Re = 400, 2,601 points, 50 runs: the AD-only PINN's u and v MSE is above 1e-2. can-PINN is about two orders lower and the numerical-differentiation PINN about one order lower 2110.15832 §3.3.1 2110.15832 fig. 10a.
• The same cavity with 40,401 points: the AD-only PINN reaches about 2e-4 and can-PINN goes below 1e-5 2110.15832 §3.3.2 2110.15832 fig. 11a.
• Backward-facing step, Re = 200, 16,000 points, 25 runs: reattachment point x1 against a ground truth of 2.608 gives 4.761 for the AD-only PINN, 2.601 for n-PINN and 2.604 for can-PINN 2110.15832 tab. 3.

These tables compare against AD-only and numerical-differentiation PINNs only, not against the reweighting variants above. Card: CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method.

Batch-wise gradient SNR (diagnostic, no loss change)

This is a monitor, not a training method. At fixed parameters, the paper takes the gradient of each batch and computes the norm of the mean batch gradient divided by the norm of the batch standard deviation, over the weights 2403.18494 §2.1 2403.18494 eq. 7. For full-batch Adam on PINNs, it names three phases 2403.18494 §2.1 2403.18494 fig. 2:

• Fitting: high SNR and a fast decline.
• Diffusion: SNR falls while the error barely moves.
• Diffusion equilibrium (DE): a sudden rise to a plateau where SNR is flat, after which the error drops steeply.

The paper links DE to residuals that are homogeneous across collocation points 2403.18494 §2.4. An SNR built from Adam's moment estimates tracks the batch SNR and reaches its equilibrium at DE 2403.18494 §2.2 2403.18494 fig. 7. For the order parameter phi, the paper writes SNR = phi / sqrt(1 - phi^2), which needs the population standard deviation 2403.18494 app. B. Card: Learning in PINNs: Phase transition, diffusion equilibrium, and generalization.

Network components used with RBA

• Modified MLP (mMLP): two encoders, each a tanh layer on the input, mixed into every hidden layer by a learned gate 2307.00379 §2.3.1 2307.00379 eq. 14 2307.00379 eq. 17.
• Weight normalisation: each weight vector is split into a length and a direction 2307.00379 §2.3.2 2307.00379 eq. 18.

On Allen-Cahn, the ablation gives 1.58e-3 with RBA and mMLP, and 4.57e-5 with RBA, mMLP and Fourier features 2307.00379 tab. 2. On AIV, weight normalisation alone gives 2.71e-1 on v and RBA with it gives 2.08e-1 2307.00379 tab. 6. These are single-seed or validation-split numbers.

Error bars from posterior samples (U-PINN)

The predictive mean is the average of the network over M posterior samples. The total variance adds a noise variance to the spread of those samples 2201.07766 §3.1. The paper calls the PINN version U-PINN. Its posterior methods are HMC, Langevin dynamics, mean-field variational inference, MC dropout and a Laplace fit at the trained mode. Its ensemble methods are independent runs, snapshots from one cyclic-learning-rate run, and SWAG 2201.07766 §4.1 2201.07766 §4.2.

On a function-approximation test with 32 noisy points, the mean's relative L2 error ranges from 21.9 to 29.2 (×10²) across the methods shown. The uncertainty RMSCE differs more: 7.1 for LA, 7.2 for both HMC and SEns, 13.6 for SWAG and 15.5 for LD 2201.07766 §6.1.

A functional prior replaces the weight prior with a generator pre-trained on historical solutions, and inference runs on the generator's low-dimensional input 2201.07766 §4.3. On the diffusion–reaction case, HMC with this learned prior has a relative L2 error of 1.6 (×10²) on u, against 3.7 for HMC with the standard prior 2201.07766 §6.2. That is one case among several. For stochastic PDEs the paper adds PI-GAN variants that sample the generator and two polynomial-chaos variants, NNPC and NNPC+ 2201.07766 §4.4. Card: Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons.

When to use which

The card gives no run that compares RBA against a tuned vanilla PINN with a learning-rate schedule and the same architecture, apart from the convergence figure 2307.00379 §3.4. The 2024 study compares only against vanilla PINN training with uniform residual weights 2403.18494 §3.1. Treat the reported gains as the authors' best case. No card compares the discrete penalty with an autodiff residual on the same problem, so the two cannot be ranked from this page. The error-bar comparisons come from the UQ paper's own prototype problems and baselines, so they do not rank the residual-weighting variants.

Implementation sketch

The loss with one weight per term, following the card's Allen-Cahn setup; a per-point multiplier lam can replace the scalar weights, as in adaptive-loss-weighting.

# PINN loss: weighted mean residual plus boundary and initial terms
# [§2.1] [eq. 6] [eq. 10]
# theta: network weights; x_r: collocation points; x_b, u_b: boundary data
def pinn_loss(theta, x_r, x_b, u_b, w_r, w_b):
    # PDE residual at each collocation point  [§2.1]
    r = pde_residual(theta, x_r)                 # (N_r,)

    # Mismatch on the boundary and initial points  [§2.1]
    e_b = net(theta, x_b) - u_b                  # (N_b,)

    # Squared residuals, mean over each term, one fixed weight per term
    # [§3.1] (w_b = 100, w_r = 1 on Allen-Cahn)
    return w_r * mean(r ** 2) + w_b * mean(e_b ** 2)

For RBA, replace w_r with a per-point multiplier lam updated by eq. 12 before the loss is formed, and hold lam constant when taking the gradient 2307.00379 eq. 12 2307.00379 §2.2. For a distance-function boundary, drop the w_b term and build the boundary into the output 2307.00379 §3.2. For the tied-decay form, set gamma = 1 - eta* so that the multiplier stays at most 1 2403.18494 §2.5.

For an energy-potential network, do not output the stress. Output the energy and the internal-variable increments, then take the stress and the dissipation as derivatives of the energy with automatic differentiation 2005.12183 §3.3 2005.12183 eq. 13. The paper's stress derivative is a partial one with respect to the energy's own inputs; the card reads it that way (see Pitfalls).

For a mesh-based penalty, the residual is a discrete operator applied to the predicted nodal stress rather than an autodiff term. The P-DivGNN loss has this form: a normalised mean-squared error plus a weight times the mean squared divergence over nodes 2507.05291 eq. 41 2507.05291 eq. 42.

For monitoring, compute the batch SNR of eq. 7 from per-batch gradients at fixed parameters, with no optimiser step 2403.18494 §2.1.

For error bars, run the network at each of the M posterior samples or ensemble members. Take the mean of the predictions as the prediction, and the spread of the predictions plus the noise variance as the total variance. Divide the spread by M, as the paper does 2201.07766 §3.1.

For CAN-PINN, build an upwind face value from the network value and its autodiff slope rather than from the network value alone. For positive flow, the east face is û(x) plus Δx/2 times the slope, and the gradient is the difference of the two faces over Δx 2110.15832 eq. 8a 2110.15832 eq. 9. The negative-flow branch and the choice of upwind side are not in the paper; see Pitfalls. The pressure face value is corrected in the same way, with a Δx/8 factor on the difference of the two slopes 2110.15832 eq. 12a.

Pitfalls

• Multiplier bounds. The upper bound is eta* / (1 - gamma), plus any offset 2307.00379 eq. 13. Check the values you pick against it. The paper's Allen-Cahn value of eta* = 1e-2 with gamma = 0.999 gives a bound of 10 (derived by the card, not stated in the paper).
• Normalisation by the maximum. The divisor is the largest residual in the current set, so one large outlier shrinks every other new term 2307.00379 §3.3. The 2024 form divides by the same maximum 2403.18494 eq. 26.
• Warm-up under the tied decay. With eta* = 0.001 and gamma = 0.999 the multipliers approach their steady value with a time constant of about 1000 iterations. For the first few hundred iterations the residual term is almost switched off. This is derived from the update, not stated in the paper 2403.18494 §2.5 AC.py.
• The residual term is only down-weighted. With the multipliers at most 1, the data term keeps weight 1, so the balance between the two shifts over training. This is inferred, not stated 2403.18494 §2.5 AC.py.
• Do not differentiate the multipliers. They are state. The released notebook detaches them allen-cahn_rba-sa.ipynb.
• Adam, not plain gradient descent. Algorithm 1 is written for gradient descent, but the experiments used Adam 2307.00379 §3.1 2307.00379 alg. 1.
• Fixed Fourier periods. The embedding is tied to the domain length, so a different domain needs a new period 2307.00379 §2.3.3.
• Ablation numbers. The Allen-Cahn ablation uses the best seed of the main table, and the Helmholtz ablation's full-model number is its best seed 2307.00379 §3.1.1 2307.00379 §3.2.1. Compare five-seed means in the main tables.
• Baselines. Apart from the self-adaptive rows, the baselines in the Allen-Cahn and Helmholtz tables were not re-run 2307.00379 §3.1 2307.00379 §3.2.
• AIV inconsistencies. The text gives a 60/40 training/validation split for PTV data, but the appendix table gives 9.4e3 training and 1.4e4 validation points, which is the reverse 2307.00379 §3.3 2307.00379 app. A.3.3. The text gives 2.5e5 iterations, and the ablation figure caption gives 2.5e4 2307.00379 §3.3 2307.00379 fig. 7. The paper does not say which value is correct.
• Timing. The authors say the code is not tuned for speed, and the three cases ran on different GPUs, so the timings are not comparable across cases 2307.00379 app. A.2 2307.00379 app. A.3.3.
• The stress scale sets the meaning of the penalty weight. In P-DivGNN the divergence is computed on standardised predictions. The NMSE does not depend on scale, but the penalty does, so a change of units or normalisation needs a new weight 2507.05291 §4.2 gnn_train.py.
• Early epochs can mislead. The mean stress is uniform over the nodes, so the divergence starts near zero. The paper's NMSE drops later with the penalty, around epoch 60 rather than 40 2507.05291 §4.2 2507.05291 fig. 5. Do not judge a run by its first epochs.
• Nodal stress needs averaging. Element stresses are constant or discontinuous, so the paper averages interpolated stress from the surrounding elements at each node before the divergence is taken 2507.05291 §4.1 2507.05291 §2.3.
• Test-set checkpoint selection. The P-DivGNN training script keeps the checkpoint with the lowest test loss. The paper says the test set is used for results only, so its reported numbers are selected on that same data 2507.05291 §4.1 gnn_train.py. Hold out a validation split before comparing with it.
• Coverage and loss form in the survey. The survey does not say how it chose its papers, and it gives no count 1903.12394 §3.1. Its knowledge term in eq. 1 has no sum over samples, and the survey does not say which functional measures violation 1903.12394 eq. 1 1903.12394 §5.1. Read eq. 1 as a form, not a recipe.
• DE onsets are found by eye. The 2024 paper gives no detector, and it says the first-order nature of the onset is unexplained 2403.18494 §4. Do not treat a DE label as a measured quantity.
• SNR batch layout. The paper gives a batch-independent SNR, SNR / sqrt(m), and effective batch sizes that account for symmetry. The released code does not implement these 2403.18494 app. A.
• Architecture is fixed in the 2024 study. All cases use 6 x 128 tanh networks with no sweep, and the DE onset may depend on architecture and stiffness 2403.18494 §3.1.
• Single runs in the 2024 study. The 10x speed-up on Allen-Cahn and the cavity result are single runs with no spread 2403.18494 fig. 4 2403.18494 fig. 5.
• Error bars depend on the prior and noise model. Cheap methods match expensive ones mainly when the prior or the noise scale is learned during inference 2201.07766 §7. With fixed priors, the paper says the epistemic part does not grow reliably with less data or more noise 2201.07766 §6.1.
• Calibration and out-of-distribution inputs. Post-training calibration uses data from the training distribution, and the paper warns that performance may get worse out of distribution 2201.07766 §5.2. The combined uncertainty of deep ensembles did not always cover the errors in the stochastic statistics 2201.07766 §7.
• Gaussian predictive assumption. The comparison metrics assume Gaussian predictive distributions, often fitted to posterior samples 2201.07766 §3.1 2201.07766 §5.1.
• Error-bar cost is indicative. The paper's wall-clock table is for one function example and is not a benchmark. HMC with 50 leapfrog steps costs about 4 to 5 times standard training, and a serial deep ensemble costs M times standard training 2201.07766 §6.6.
• Hyperparameters are per experiment. Dropout rate, weight decay, learning rate and cycle length were tuned for each experiment. The paper gives only cross-validation and meta-learning as options 2201.07766 §6.1 2201.07766 §4.1 2201.07766 §4.2.
• Partial or total derivative. Stress and dissipation are written as partial derivatives of the energy network with respect to its strain and internal variable inputs. The internal variables also depend on strain through their own network, so a total derivative would add that path and change the tangent. The paper does not say which it uses. The card reads the partial form, which the step list supports; this is an interpretation 2005.12183 §3.3.
• Mixed units in the energy-potential reference. The 1D yield function combines a modulus in GPa with stress and yield values in MPa, and the paper gives no unit system. Choose one and convert every input and target 2005.12183 eq. 25 2005.12183 tab. 3.
• Activation rule for the leaky-ReLU network. The second-derivative rule of §3.2 is shown for the energy network. The paper does not say whether it also applies to the leaky-ReLU network that predicts the internal variables 2005.12183 §5.1.1.
• Energy-potential training details are missing. Not stated: learning rate for the applications, epochs, the loss weights, the leaky-ReLU slope and the 3D batch size 2005.12183 §5.1.1 2005.12183 §5.2.1. The learning rate of 1e-5 is stated for the toy example only [arxiv:2005.12183 §3.2.1].
• Out-of-range accuracy drops. Strain increments beyond the training range reduce accuracy, and the tangent claim outside that range rests on plots only 2005.12183 §5.1.3 2005.12183 §6.
• Extensions are untested. The non-isothermal architecture and the claimed extension to damage or viscosity are not tested 2005.12183 fig. 7a 2005.12183 §5 intro.
• Real materials need data the paper does not provide. Each training pair needs the free energy and dissipation at the next step. The examples take both from analytic models; for real materials they must come from micromechanical simulation or experiment, which the paper does not show 2005.12183 §1 2005.12183 §4.2.
• Upwind branch for CAN-PINN. The paper gives the positive-flow face formulas only 2110.15832 eq. 8a 2110.15832 eq. 8b. The negative branch and the rule for choosing the upwind side are not stated. The released code uses a mirror formula and the average of the two face velocities d01_can-pinn_lid_driven_cavity_re400.ipynb. Say which rule you use.
• Residual points per batch. The sampler draws the batch from all points, boundary points included, so a batch yields fewer PDE residuals than its nominal size d01_can-pinn_lid_driven_cavity_re400.ipynb d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Corner double count. Left and bottom boundary terms both include the bottom-left corner, so it counts twice in the boundary loss d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Pressure is fixed only up to a constant. The boundary loss enforces u and v alone, and the paper states no pressure condition 2110.15832 §3.3.
• Forward-only code. The released notebooks cover the Re = 400 cavity forward case. They do not reproduce the inverse or the other reported problems d01_can-pinn_lid_driven_cavity_re400.ipynb.

Contradictions

The two RBA cards describe the same moving-average update with different constants and bounds. The 2023 card uses an independent gamma and gives Allen-Cahn eta* = 1e-2 with gamma = 0.999, so the multiplier can reach 10 2307.00379 eq. 13 2307.00379 app. A.2. The 2024 card sets gamma = 1 - eta*, so the multiplier stays at most 1 2403.18494 §2.5. Its Allen-Cahn setup also keeps weight 1 on the initial term, where the 2023 card uses 100 2307.00379 §3.1 2403.18494 eq. 25. The 2024 card credits RBA to an earlier paper by the same authors and does not link it by arXiv id 2403.18494 §2.5. The two cards do not reconcile these settings, so do not assume the 2024 constants transfer to the 2023 loss.

Within arxiv:2307.00379; its code is at commit c8ca4cb5 under the MIT licence:

• Allen-Cahn RBA learning rate. The paper gives eta* = 1e-2 2307.00379 app. A.2; the code uses 1e-3 and explains that the initial-condition term is scaled to compensate allen-cahn_enhanced.ipynb.
• Helmholtz loss form. The paper writes the squared, weighted residual 2307.00379 eq. 10. The PyTorch code matches it; the JAX code multiplies the squared residual by the weight, which is a different loss for weights above zero helmholtz_fourier.ipynb.
• Optimiser. Algorithm 1 is plain gradient descent; all runs used Adam 2307.00379 alg. 1 2307.00379 §3.1.
• Helmholtz residual points. The paper gives 1201; the code uses a 101 x 101 grid, which is 10201 points 2307.00379 app. A.2 helmholtz_fourier.ipynb.
• Helmholtz L-BFGS. The paper gives 2e3 iterations; the code runs at most 1005 2307.00379 §3.2 helmholtz_fourier.ipynb.
• mMLP gate order. The PyTorch code matches the paper's Eq. 17; the JAX code swaps the roles of the two encoders. Both train 2307.00379 eq. 17 helmholtz_fourier.ipynb.
• Helmholtz initialisation. The paper says Xavier; the code uses Glorot normal with zero bias 2307.00379 §3.2 helmholtz_fourier.ipynb.

Within arxiv:2403.18494; its code is at commit 312b830d under the MIT licence:

• SNR parameters. The paper uses all network parameters 2403.18494 §2.1. The code uses weight matrices only, with biases excluded snr.py func.py.
• Adam SNR. The paper gives a simplified update without bias correction 2403.18494 §2.2. The code uses bias-corrected moments and includes biases snr.py. The diagnostic is off by default.
• Smoothing window. The paper uses 100 past iterations 2403.18494 §3.1. The notebook uses a window of 60 stored rows, and the code stores a row every 100 iterations, and every 10 before iteration 300, so the window covers thousands of iterations Paper_plots.ipynb snr.py.
• RBA normalisation. The paper divides by the largest residual magnitude and bounds the multipliers in [0, 1] 2403.18494 §2.5 2403.18494 eq. 26. The Helmholtz code first subtracts the minimum and divides by the range, and the shifted weights enter the training loss HM.py. The appendix calls the shift cosmetic 2403.18494 app. D.
• Cavity collocation count. The paper gives 25,600 points 2403.18494 app. D; the code uses 10,000 CV.py.

Within arxiv:2507.05291; its code is at commit 7847c02e under GPL-3.0-or-later, and the card paraphrases it without quoting:

• Learning rate. The paper uses 1e-2 for both datasets 2507.05291 §4.2 2507.05291 §5.2. The released configs set 0.001 config_train_div.yml.
• Hyperelastic divergence weight. The paper sets 1e-2 for hyperelastic data 2507.05291 §5.2. The hyperelastic config sets 10 config_train_div_hyperelast.yml. The paper also says the weight was not tuned 2507.05291 §5.3.
• Early stopping. The paper uses patience 20 on validation NMSE 2507.05291 §4.2. The released configs turn early stopping off config_train_div.yml.
• Linear divergence numbers. Section 4.3 gives 4.56e-4 for P-DivGNN and 9.43e-4 for FEM 2507.05291 §4.3. Table 1 gives 3.23e-4 and 9.18e-4 2507.05291 tab. 1. The paper does not explain the difference.
• Periodic edge feature. The paper sets the feature to zero 2507.05291 §3.1. The code standardises all edge features, so these edges do not enter as zero models.py.

Within arxiv:2005.12183; the card records no official code, only a related repository by the first author that does not contain this paper's training setup:

• Early stopping. The application section stops training when the test error rises 2005.12183 §5. The text also uses a validation set for early stopping 2005.12183 §3.2.1 2005.12183 §5.1.1. Use validation for stopping and keep the test set for final numbers.

Within arxiv:2201.07766; the card records no code release:

• Calibration spread. The text says post-training calibration makes all methods roughly equally calibrated 2201.07766 §6.1, while the calibrated RMSCE column of the same table ranges from 3.8 for MFVI to 8.0 for MCD 2201.07766 §6.1. Do not take either as settled.

Within arxiv:2110.15832; the code is at commit a92594e6 with no licence file, so it is cited by path only:

• Momentum residual. The paper's conservative momentum form has no extra divergence term 2110.15832 eq. 27b. The code's momentum residual adds minus u times the discrete divergence, and the same in v d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Boundary weight. The paper weights the boundary and initial terms by λ_BC and λ_IC 2110.15832 eq. 2a. The code divides the PDE loss by λ and adds the boundary loss with no separate weight d01_can-pinn_lid_driven_cavity_re400.ipynb.
• Weight initialisation. The paper gives N(0, σ²); the code uses a truncated normal with σ = 1 d01_can-pinn_lid_driven_cavity_re400.ipynb.

Papers

• Residual-based attention in physics-informed neural networks: decaying per-point multipliers from normalised PDE residuals, with no gradients or extra networks; author-reported gains on Allen-Cahn, Helmholtz and an inverse flow problem.
• Learning in PINNs: Phase transition, diffusion equilibrium, and generalization: full-batch Adam on PINNs, a batch-gradient SNR with three training phases, and RBA with the decay tied to the rate; author-reported single-run gains on Allen-Cahn and a cavity flow.
• Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity: a graph network for periodic micro-structure meshes with a penalty on the discrete stress divergence; author-reported gains on linear and hyperelastic meshes.
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems: a taxonomy of prior knowledge in learning, including equations in the loss and the overlapping names for physics-informed methods; a survey with no experiments of its own.
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons: posterior and ensemble error bars for PINNs and DeepONets, functional GAN priors and post-training calibration; author-reported comparisons on prototype problems, no code released.
• Thermodynamics-based Artificial Neural Networks for constitutive modeling: a network predicts a free-energy potential and internal-variable increments, and stress follows by differentiation so the first law holds by construction; author-reported on isothermal 1D and 3D elasto-plasticity, with data from the analytic models.
• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method: couples autodiff slopes with finite-difference-style neighbour terms for sparse collocation points; author-reported gains over AD-only and numerical-differentiation PINNs on flow problems, with code for the forward cavity case only.