---
kind: concept
slug: sobolev-training
title: Sobolev training
summary: >-
  Train a network to match a target's derivatives as well as its values, so the learned function has a smooth, correct slope. The H1 loss adds a squared error on the gradient to the usual value error. Only one card so far: a hyperelastic energy model whose stress is the energy's derivative. The gains are author-reported and qualitative in the text, and the derivative labels must come from a solver.
tags: [loss-function, derivatives, surrogate, hyperelasticity, graph-input]
papers: ["2001.04292"]
related: [surrogate-modeling, graph-neural-network, physics-informed-neural-network, automatic-differentiation, knowledge-distillation]
status: draft
updated_at: 2026-11-08
---

## Overview

Sobolev training adds the error on a target's derivative to the usual error on
its value. A network that is trained on values alone can fit them and still
have a slope that oscillates. The paper's motivation is that a learned energy
can have non-smooth derivatives, so the predicted stress oscillates
[arxiv:2001.04292 §4]. Fitting the derivative as well as the value is the
remedy. The paper calls the two-term version the H1 Sobolev loss
[arxiv:2001.04292 eq. 18].

The one card so far applies this to a stored-energy function ψ. The stress is
the derivative of ψ with respect to the strain, so the derivative is a
physical target, not an extra label [arxiv:2001.04292 eq. 19]. The loss is the
mean of the value error and the squared error of ∂ψ/∂C
[arxiv:2001.04292 eq. 18]. The original form, Sobolev training for neural
networks, is due to Czarnecki et al. (2017), as the card's lineage notes
[arxiv:2001.04292 §4]. No card exists for that paper, so it is named here only.

## Variants

The cards give one model family, and its variants differ in input and in
loss. The paper compares them on the same data [arxiv:2001.04292 tab. 1].

### H1 loss on a graph-input energy model

The energy network reads a grain-contact graph, passes the graph code and the
strain C to an MLP, and outputs ψ. Both the value and the derivative with
respect to C are fitted [arxiv:2001.04292 §3.3] [arxiv:2001.04292 eq. 18].
Card: [Geometric deep learning for computational mechanics (Part I)](/papers/2001.04292).

### H1 loss on an MLP

The same loss on a feed-forward MLP that has no graph input. It is the
paper's no-graph baseline for the fracture study [arxiv:2001.04292 §7.3]. On
the blind 100-RVE set, its errors are reported lower than those of an L2-trained
MLP [arxiv:2001.04292 §7.1] [fig. 12].

### H1 loss with regularisation

The paper names a regularised H1 model, which narrows the train-test gap of the
graph model on 150 polycrystals [arxiv:2001.04292 fig. 13]. The card does not
describe the regulariser, so the form is not known from the cards.

### L2 loss baseline

The value-only fit is the reference the paper compares against
[arxiv:2001.04292 §7.1] [tab. 1]. It is the variant to drop when no derivative
labels exist.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Derivative labels exist and the derivative matters downstream, e.g. stress from an energy | H1 loss | Fitting the slope keeps the derivative smooth [arxiv:2001.04292 §4] [eq. 18] |
| Only values are labelled | L2 loss | The H1 term needs a derivative target, which the paper took from its solver [arxiv:2001.04292 §6.1] |
| Microstructure is a graph and averaged state variables are too coarse | Graph-input H1 model | The graph encodes grain contacts that averages cannot capture [arxiv:2001.04292 §1] |
| Few samples and a graph input | H1 with regularisation | The unregularised hybrid overfits on 150 polycrystals; the regularised model narrows the gap [arxiv:2001.04292 fig. 13] |
| Output must be invariant to rotation | Input through the strain invariant C | Frame indifference holds by construction [arxiv:2001.04292 §5.1] |

The paper evaluates on author-generated FFT data and reports the results as
eCDF curves without numeric errors in the text [arxiv:2001.04292 §7.1]. The
choices above therefore rest on one paper and on the paper's own comparisons.

## Implementation sketch

The H1 loss needs the model's derivative with respect to its input. In an
autodiff framework this is one extra gradient call per batch. The stress
labels must match the convention of the energy: the card's stress is twice
the derivative [arxiv:2001.04292 eq. 19].

```pseudocode
# H1 Sobolev loss for an energy model  [arxiv:2001.04292 eq. 18] [eq. 19]
# Input:  C (N, 6) strain inputs, psi_true (N,) energy labels,
#         S_true (N, 6) stress labels
# Output: scalar loss

def h1_loss(model, C, psi_true, S_true):
    # C must be differentiable with respect to its input  [§4]
    psi_pred = model(C)
    # Derivative of the prediction with respect to the input  [eq. 18]
    dpsi_pred = grad(sum(psi_pred), C)
    # Target derivative is half the stress label  [eq. 19]
    dpsi_true = 0.5 * S_true
    # Value and derivative errors with unit weight  [eq. 18]
    value_err = square(psi_true - psi_pred)
    slope_err = sum(square(dpsi_true - dpsi_pred), axis=-1)
    return mean(value_err + slope_err)
```

The two error terms have unit weight in the card's equation. The card does not
say how the components of the derivative are normalised, so a scale mismatch
between the two terms is possible.

## Contradictions

None known. Only one paper with a card covers this concept. Within that paper,
the dataset size is given as 150 RVEs in §7 and as 100 RVEs for the K-fold
study in §7.1. The card notes that their relation is not stated
[arxiv:2001.04292 §7.1] [fig. 13].

## Papers

- [Geometric deep learning for computational mechanics (Part I)](/papers/2001.04292): applies the H1 loss to a graph-input hyperelastic energy and compares it with L2 and no-graph baselines. Author-reported; no numeric errors in the text.