Sobolev training

Overview

Sobolev training adds the error on a target's derivative to the usual error on its value. A network that is trained on values alone can fit them and still have a slope that oscillates. The paper's motivation is that a learned energy can have non-smooth derivatives, so the predicted stress oscillates 2001.04292 §4. Fitting the derivative as well as the value is the remedy. The paper calls the two-term version the H1 Sobolev loss 2001.04292 eq. 18.

The one card so far applies this to a stored-energy function ψ. The stress is the derivative of ψ with respect to the strain, so the derivative is a physical target, not an extra label 2001.04292 eq. 19. The loss is the mean of the value error and the squared error of ∂ψ/∂C 2001.04292 eq. 18. The original form, Sobolev training for neural networks, is due to Czarnecki et al. (2017), as the card's lineage notes 2001.04292 §4. No card exists for that paper, so it is named here only.

Variants

The cards give one model family, and its variants differ in input and in loss. The paper compares them on the same data 2001.04292 tab. 1.

H1 loss on a graph-input energy model

The energy network reads a grain-contact graph, passes the graph code and the strain C to an MLP, and outputs ψ. Both the value and the derivative with respect to C are fitted 2001.04292 §3.3 2001.04292 eq. 18. Card: Geometric deep learning for computational mechanics (Part I).

H1 loss on an MLP

The same loss on a feed-forward MLP that has no graph input. It is the paper's no-graph baseline for the fracture study 2001.04292 §7.3. On the blind 100-RVE set, its errors are reported lower than those of an L2-trained MLP 2001.04292 §7.1 fig. 12.

H1 loss with regularisation

The paper names a regularised H1 model, which narrows the train-test gap of the graph model on 150 polycrystals 2001.04292 fig. 13. The card does not describe the regulariser, so the form is not known from the cards.

L2 loss baseline

The value-only fit is the reference the paper compares against 2001.04292 §7.1 tab. 1. It is the variant to drop when no derivative labels exist.

When to use which

The paper evaluates on author-generated FFT data and reports the results as eCDF curves without numeric errors in the text 2001.04292 §7.1. The choices above therefore rest on one paper and on the paper's own comparisons.

Implementation sketch

The H1 loss needs the model's derivative with respect to its input. In an autodiff framework this is one extra gradient call per batch. The stress labels must match the convention of the energy: the card's stress is twice the derivative 2001.04292 eq. 19.

# H1 Sobolev loss for an energy model  [arxiv:2001.04292 eq. 18] [eq. 19]
# Input:  C (N, 6) strain inputs, psi_true (N,) energy labels,
#         S_true (N, 6) stress labels
# Output: scalar loss

def h1_loss(model, C, psi_true, S_true):
    # C must be differentiable with respect to its input  [§4]
    psi_pred = model(C)
    # Derivative of the prediction with respect to the input  [eq. 18]
    dpsi_pred = grad(sum(psi_pred), C)
    # Target derivative is half the stress label  [eq. 19]
    dpsi_true = 0.5 * S_true
    # Value and derivative errors with unit weight  [eq. 18]
    value_err = square(psi_true - psi_pred)
    slope_err = sum(square(dpsi_true - dpsi_pred), axis=-1)
    return mean(value_err + slope_err)

The two error terms have unit weight in the card's equation. The card does not say how the components of the derivative are normalised, so a scale mismatch between the two terms is possible.

Contradictions

None known. Only one paper with a card covers this concept. Within that paper, the dataset size is given as 150 RVEs in §7 and as 100 RVEs for the K-fold study in §7.1. The card notes that their relation is not stated 2001.04292 §7.1 fig. 13.

Papers

• Geometric deep learning for computational mechanics (Part I): applies the H1 loss to a graph-input hyperelastic energy and compares it with L2 and no-graph baselines. Author-reported; no numeric errors in the text.