Thin-plate spline (TPS) warp

Overview

A thin-plate spline is a smooth mapping of the plane, built from a set of control points. Among the mappings that pass exactly through a set of control-point pairs, it is the one that bends the plane the least. It has a closed form: an affine part plus a weighted sum of radial basis terms U(r) = r² log r² over the control points 2203.14367 §3.1 2203.14367 eq. 1 2203.14367 eq. 2 2203.14367 eq. 3.

Both papers on this page use the name for a smooth surface fitted over 2-D points, but for different jobs:

• Warping. The exact-interpolation form maps one image onto another. The paper on image animation uses it as a motion model: it predicts K groups of five keypoints for each image and fits one TPS per group, so the warp follows the keypoints' non-linear motion 2203.14367 §3.1. Card: Thin-Plate Spline Motion Model for Image Animation.
• Smoothing. A smoothing version fits noisy samples on a grid and is used as an estimator of the spatially correlated part of an error. The photogrammetry paper applies it to displacement nodes with 10-fold cross-validation 2610.01438 §2.4.3. Card: The Impact of Processing Parameters on High-Accuracy Measurements in UAV Photogrammetry.

The two uses differ in what they promise. The warp passes through its control pairs exactly, and the paper states this as its constraint [arxiv:2203.14367 eq. 1]. The photogrammetry paper uses a smoothing parameter of 1 and calls it "corresponding to a cubic spline interpolant". It does not say whether its fit passes exactly through the nodes 2610.01438 §2.4.3. Do not treat the smoothing variant as an exact warp.

Variants

TPS per keypoint group (the paper's motion model)

Each group of five source and driving keypoints defines one TPS. The paper states the fit as minimising bending energy under exact interpolation 2203.14367 eq. 1. The K TPS warps and one affine background warp are combined per pixel. A softmax over K+1 contribution maps sums to one, and the flow is the weighted sum of the K+1 transforms 2203.14367 eq. 5 2203.14367 eq. 6 2203.14367 §3.1.

The paper also drops TPS fields at random during early training. Each field is kept with probability 1 − P, and the kept terms are rescaled by 1/(1 − P) 2203.14367 eq. 7 2203.14367 eq. 8 2203.14367 §3.1. The paper gives no value for P and says dropout is removed after several epochs 2203.14367 §3.1. Card: Thin-Plate Spline Motion Model for Image Animation.

Measured on TaiChiHD, this variant replaces the affine motion of the baseline (MRAA). Its first ablation step gives AKD 4.96 against 5.41 for MRAA, with L1 unchanged at 0.048 and AED worse at 0.153 against 0.149 [arxiv:2203.14367 tab. 4]. Dropout, multi-resolution masks and the two auxiliary losses bring the full model to AKD 4.57 2203.14367 tab. 4.

Affine motion per keypoint (the baseline it replaces)

The comparison baseline, MRAA, moves each keypoint with a local affine piece. The paper argues that one affine piece per keypoint cannot represent a curved, non-linear motion such as a lip opening, so it proposes TPS instead 2203.14367 §1 2203.14367 §2. MRAA has no card in this wiki, so its numbers are used only as reported in the paper's tables 2203.14367 tab. 1.

Smoothing TPS with cross-validation (error decomposition)

The photogrammetry paper compares two processing settings by the errors of their derived displacement maps. To split each displacement error into a systematic and a random part, it uses thin-plate smoothing splines on the grid nodes, with smoothing parameter 1 and 10-fold cross-validation [arxiv:2610.01438 §2.4.3]. The nodes are divided at random into ten equal subsets. For each subset, the other nine are used to predict its nodes. The predicted value is the systematic part. The random part is the total error minus the systematic part 2610.01438 §2.4.3. The paper summarises systematic error by the median absolute error across time intervals, and random error by RMSE 2610.01438 §2.4.3.

The paper gives no formula or library for the smoothing spline. Its kernel and the exact meaning of the parameter value 1 are therefore not stated.

This is a diagnostic, not a warp. Its numbers compare the two processing settings, and say nothing about which smoothing method is best. The baseline setting has a median absolute systematic vertical error of 14 mm, and the enhanced setting 7 mm. The random part is 6 to 7 mm for both [arxiv:2610.01438 §3.2] 2610.01438 tab. 5. Card: The Impact of Processing Parameters on High-Accuracy Measurements in UAV Photogrammetry.

When to use which

Two papers stand behind this page. The table states only what their tables and text support. All numbers are author-reported.

Implementation sketch

The TPS fit and the blend of the K+1 warps for animation, then the smoothing split for error analysis. The paper's loss and training details are in the card and not repeated here. Points that differ from the text are covered under Contradictions.

# Fit one TPS per keypoint group, source and driving  [§3.1] [eq. 1-3]
# Input:  kp_source, kp_driving: (K, 5, 2) control points per group
# Output: K warp fields sampled on the grid

def fit_tps(kp_source, kp_driving):
    # Radial kernel between control points: U(r) = r^2 log r^2  [eq. 3]
    kernel = radial_kernel(pairwise_distance(kp_driving))
    # Solve for the affine part and radial weights that interpolate
    # the pairs with minimal bending energy  [eq. 1] [eq. 2]
    affine, weights = solve_linear_system(kernel, kp_driving, kp_source)
    return evaluate_tps(affine, weights, grid)    # (K, H, W, 2)

# Blend the K+1 warps per pixel into one flow  [eq. 5] [eq. 6]
# Input:  K TPS grids and one affine background grid
# Output: flow (H, W, 2) used to warp the source

def blend_warps(tps_grids, affine_grid, logits):
    # Softmax across the K+1 transforms, per pixel  [eq. 5]
    weights = softmax(concat([logits_bg, logits_tps]), axis=transforms)
    # Flow is the weighted sum of the transforms  [eq. 6]
    return sum(weights[j] * grids[j] for j in range(K + 1))

# Split one error component into systematic and random parts  [§2.4.3]
# Input:  node positions (N, 2), total error per node (N,)
# Output: systematic part (N,), random part (N,)

def split_error(nodes, total_error):
    # Ten random, equal subsets of the nodes  [§2.4.3]
    folds = random_partition(nodes, num_folds=10)
    systematic = zeros_like(total_error)
    for held_out in folds:
        # Smoothing TPS, smoothing parameter 1, fitted on nine subsets.
        # The paper gives no formula or library for this fit  [§2.4.3]
        model = fit_smoothing_tps(nodes[~held_out], total_error[~held_out],
                                  smoothing=1)
        # Prediction at the held-out nodes is the systematic part  [§2.4.3]
        systematic[held_out] = model.predict(nodes[held_out])
    # Random part is what the systematic part does not explain  [§2.4.3]
    return systematic, total_error - systematic

Contradictions

Within the first paper, the motion dimension it reports does not match its own formula. The formula K·(6 + 5·2) + 6 gives 38, 70 and 134 for K = 2, 4 and 8. The text gives 118 for K = 8 2203.14367 §4.3. The comparison in tab. 3 depends on this count 2203.14367 tab. 3.

Within the second paper, the smoothing parameter 1 is called a smoothing spline and, in the same sentence, "corresponding to a cubic spline interpolant" 2610.01438 §2.4.3. The paper does not say whether the fitted surface passes through the nodes. This is a wording conflict inside one paper, and it matters for whether the method is an exact warp. It is not a conflict between papers. The card's pitfalls list it.

The paper text and the released code of the first paper differ in several places. These are paper-versus-code differences, not disagreements between papers:

• TPS direction. The paper writes the constraint as the source keypoints mapped to the driving keypoints 2203.14367 eq. 1. The code fits the warp from driving to source and uses it directly as a sampling grid util.py dense_motion.py.
• Dropout rescaling. Equation 7 divides the kept TPS terms by 1 − P. The code divides the logits by 1 − P after the exponential, so the kept terms are not rescaled as equation 7 requires dense_motion.py.
• Loss weights and reduction. The paper's total is an unweighted sum of absolute-value terms 2203.14367 eq. 14. The code weights each term by 10 and takes means taichi-256.yaml.
• Ridge and epsilon. The paper's constraint is exact interpolation. The code adds 0.01 times the identity before the solve and adds 1e-9 inside the logarithm util.py util.py.

Implementers should pick either the text or the code for each point and say which one they chose. The card's pitfalls list these in full.

Papers

• Thin-Plate Spline Motion Model for Image Animation: replaces the affine motion of keypoints with one TPS per group of five keypoints, adds dropout on the TPS fields and multi-resolution occlusion masks, and reports lower keypoint error on TaiChiHD, TED-talks and VoxCeleb.
• The Impact of Processing Parameters on High-Accuracy Measurements in UAV Photogrammetry: uses a smoothing thin-plate spline with 10-fold cross-validation to split displacement error into systematic and random parts, and reports how eight processing parameters change the systematic and random errors.