Adaptive collocation sampling

Overview

A physics-informed neural network (PINN) learns a field by driving the PDE residual towards zero at a set of collocation points. Regular grids are the default. Adaptive collocation sampling replaces part of the grid with points where the current error is largest. The invariant is that the training loss is still built from residuals at the chosen points, so only the point selection changes 2110.07374 §5.1. The selection error uses the local PDE residual only; the global work-balance term is excluded because it cannot be evaluated per point 2110.07374 §5.1.

Variants

Residual-driven refinement on a coarse regular grid

Each cycle keeps a fixed coarse regular grid, draws uniform random candidates over the domain, scores them by local residual, and keeps the worst ones. The two sets are merged, with the ratio γ = n_reg / n_ada setting how many points come from each source 2110.07374 §5.1. The paper reports γ = 2.2 2110.07374 §5.1. Gradient clipping with a threshold α is applied during these cycles; the paper does not state α 2110.07374 §5.1. The study uses one regular pre-training pass and one adaptive cycle, each with 5 x 10^3 BFGS iterations 2110.07374 §5.3. Card: Physics informed neural networks for continuum micromechanics.

The paper applies this scheme to both a single PINN and a conservative PINN (cPINN) with a domain split 2110.07374 §5.3. Wight and Zhao (2020) also propose an adaptive scheme; the paper calls it related but different 2110.07374 §5.1. That work has no card here, so it is named only.

When to use which

The only evidence here is one linear-elasticity problem with a single stiff inclusion, with author-reported results. Treat the statements below as what that study shows, not as general rules.

Implementation sketch

Common core of the scheme, with the points that the paper leaves open marked.

# Adaptive collocation cycles  [arxiv:2110.07374 §5.1]
def adaptive_cycles(net, grid_coarse, n_ada, n_cand, cycles, clip):
    # Pre-train on a fine regular grid first  [arxiv:2110.07374 §5.3]
    train(net, regular_grid_fine())
    for cycle in range(cycles):
        # Candidate count is not stated in the paper  [arxiv:2110.07374 §5.1]
        cand = uniform_points(n_cand)
        # Local PDE residual only; no global work term  [arxiv:2110.07374 §5.1]
        err = local_residual(net, cand)
        # Keep the n_ada points with the largest error
        extra = cand[argsort(-err)[:n_ada]]
        pts = concat(grid_coarse, extra)
        # Gradient clipping threshold alpha is not stated  [arxiv:2110.07374 §5.1]
        train(net, pts, clip_norm=clip)
    return net

Contradictions

Within the one paper, the text and the plotted data disagree. The text says the adaptive PINN has lower error than the PINN in the first half of the study. The table shows this only at 16^2 and 32^2; at 8^2 the adaptive PINN is higher 2110.07374 §5.3. The paper also states that the adaptive variants do not converge from 64^2 points upwards 2110.07374 §5.3. Other figures are author-reported and come from one problem, so none can be checked against another paper here.

Papers

• Physics informed neural networks for continuum micromechanics: adaptive residual-driven sampling compared with regular grids, and a cPINN domain split, on a stiff-inclusion elasticity problem.