wikillm

Concept · 1 paper · updated 2026-11-15

Perceptual loss

A perceptual loss compares images through the features of a pretrained CNN (usually VGG) instead of pixel by pixel. The matting paper applies VGG16 feature losses to the predicted alpha map and to the foreground colour, with no weights given between the terms. Its Composition-1K ablation shows the alpha feature loss raising SAD, so the gain is not established there.

loss-functionvggfeature-lossmatting

Markdown2 min read

AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

Overview

A pixel loss such as L1 or L2 compares two images value by value. A perceptual loss instead passes both images through a pretrained network and compares the intermediate activations, usually at several layers. The matting card uses a VGG16 network for this, at four layers 1909.09725 §3.1. The card does not say whether that network is frozen during training, and does not state the source of its weights 1909.09725 §3.1.

The invariant the variants share is the layer-wise distance between features, summed over layers. What differs is which images are compared and under which mask.

Variants

VGG16 feature loss on the alpha map

The predicted alpha is multiplied by the ground-truth foreground, and the ground-truth alpha is multiplied by the same foreground. The loss is the squared distance between VGG16 features of those two products, summed over conv1_2, conv2_2, conv3_3 and conv4_3 1909.09725 §3.1 1909.09725 eq. 3. Card: Context-aware matting.

Colour feature loss on the foreground

The same VGG16 layers compare the predicted foreground with the true one. Both are multiplied by the ground-truth alpha, so the loss only sees the visible part of the subject 1909.09725 §3.1 1909.09725 eq. 4. An ℓ1 term restricted to pixels with positive ground-truth alpha is a separate, non-perceptual loss for the same decoder 1909.09725 eq. 5. Card: Context-aware matting.

When to use which

SituationUseWhy
Alpha-only model with a trimap and a fixed pixel-loss recipeStart without the alpha feature lossIn the Composition-1K ablation, adding it to the Laplacian-plus-CE model raised SAD from 35.8 to 38.8 1909.09725 tab. 1
Model with a foreground decoder that must keep colour detailColour feature loss, if a gradient metric matters more than SADModel (6), which adds the alpha and colour feature losses, has Grad 16.9 against 17.3 for model (3), but its SAD is 38.0 against 35.8 1909.09725 tab. 1
Reproducing the total training lossCheck the source before copyingThe paper gives no weights between its Laplacian, feature and colour losses 1909.09725 §3.1. The card records that the released code does not include this loss

Implementation sketch

Language-agnostic pseudocode of the common core. The VGG weights and the per-layer weights are not stated in the paper, so the values below are inputs.

Pseudocode
# Pretrained feature extractor: VGG16, layers conv1_2 to conv4_3  [§3.1]
# The paper gives no weights for the layers or for the loss terms

def feature_distance(x, y, layers):
    # Sum over layers of the squared distance between activations  [eq. 3]
    total = 0
    for layer in layers:
        total += sum((phi(layer, x) - phi(layer, y)) ** 2)
    return total

# Alpha-feature variant: composite with the ground-truth foreground  [eq. 3]
def alpha_feature_loss(alpha_pred, alpha_gt, fg_gt, layers):
    x = alpha_gt * fg_gt       # ground truth, over the true foreground
    y = alpha_pred * fg_gt     # prediction, over the same foreground
    return feature_distance(x, y, layers)

# Colour-feature variant: both masked by the ground-truth alpha  [eq. 4]
def colour_feature_loss(fg_pred, fg_gt, alpha_gt, layers):
    x = alpha_gt * fg_gt
    y = alpha_gt * fg_pred
    return feature_distance(x, y, layers)

Contradictions

None known. Only one card in this set uses the concept.

Within the matting paper: Table 1 shows that model (4), which adds the alpha feature loss to model (3), has a higher SAD (38.8 against 35.8) and a higher MSE 1909.09725 tab. 1. The card records no statement from the paper that this loss improves SAD, so check the paper's discussion before citing one.

Papers

  • Context-aware matting: applies VGG16 feature losses to the predicted alpha and to the foreground colour, and reports ablations with and without them.

Source and license

Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.