---
kind: concept
slug: perceptual-loss
title: Perceptual loss
summary: >-
  A perceptual loss compares images through the features of a pretrained CNN (usually VGG) instead of pixel by pixel. The matting paper applies VGG16 feature losses to the predicted alpha map and to the foreground colour, with no weights given between the terms. Its Composition-1K ablation shows the alpha feature loss raising SAD, so the gain is not established there.
tags: [loss-function, vgg, feature-loss, matting]
papers: ["1909.09725"]
related: [image-matting]
status: draft
updated_at: 2026-11-15
---

## Overview

A pixel loss such as L1 or L2 compares two images value by value. A perceptual
loss instead passes both images through a pretrained network and compares the
intermediate activations, usually at several layers. The matting card uses a
VGG16 network for this, at four layers [arxiv:1909.09725 §3.1]. The card does
not say whether that network is frozen during training, and does not state the
source of its weights [arxiv:1909.09725 §3.1].

The invariant the variants share is the layer-wise distance between features,
summed over layers. What differs is which images are compared and under which
mask.

## Variants

### VGG16 feature loss on the alpha map

The predicted alpha is multiplied by the ground-truth foreground, and the
ground-truth alpha is multiplied by the same foreground. The loss is the
squared distance between VGG16 features of those two products, summed over
conv1_2, conv2_2, conv3_3 and conv4_3 [arxiv:1909.09725 §3.1]
[arxiv:1909.09725 eq. 3]. Card: [Context-aware matting](/papers/1909.09725).

### Colour feature loss on the foreground

The same VGG16 layers compare the predicted foreground with the true one. Both
are multiplied by the ground-truth alpha, so the loss only sees the visible
part of the subject [arxiv:1909.09725 §3.1] [arxiv:1909.09725 eq. 4]. An ℓ1
term restricted to pixels with positive ground-truth alpha is a separate,
non-perceptual loss for the same decoder [arxiv:1909.09725 eq. 5]. Card:
[Context-aware matting](/papers/1909.09725).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Alpha-only model with a trimap and a fixed pixel-loss recipe | Start without the alpha feature loss | In the Composition-1K ablation, adding it to the Laplacian-plus-CE model raised SAD from 35.8 to 38.8 [arxiv:1909.09725 tab. 1] |
| Model with a foreground decoder that must keep colour detail | Colour feature loss, if a gradient metric matters more than SAD | Model (6), which adds the alpha and colour feature losses, has Grad 16.9 against 17.3 for model (3), but its SAD is 38.0 against 35.8 [arxiv:1909.09725 tab. 1] |
| Reproducing the total training loss | Check the source before copying | The paper gives no weights between its Laplacian, feature and colour losses [arxiv:1909.09725 §3.1]. The card records that the released code does not include this loss |

## Implementation sketch

Language-agnostic pseudocode of the common core. The VGG weights and the
per-layer weights are not stated in the paper, so the values below are inputs.

```pseudocode
# Pretrained feature extractor: VGG16, layers conv1_2 to conv4_3  [§3.1]
# The paper gives no weights for the layers or for the loss terms

def feature_distance(x, y, layers):
    # Sum over layers of the squared distance between activations  [eq. 3]
    total = 0
    for layer in layers:
        total += sum((phi(layer, x) - phi(layer, y)) ** 2)
    return total

# Alpha-feature variant: composite with the ground-truth foreground  [eq. 3]
def alpha_feature_loss(alpha_pred, alpha_gt, fg_gt, layers):
    x = alpha_gt * fg_gt       # ground truth, over the true foreground
    y = alpha_pred * fg_gt     # prediction, over the same foreground
    return feature_distance(x, y, layers)

# Colour-feature variant: both masked by the ground-truth alpha  [eq. 4]
def colour_feature_loss(fg_pred, fg_gt, alpha_gt, layers):
    x = alpha_gt * fg_gt
    y = alpha_gt * fg_pred
    return feature_distance(x, y, layers)
```

## Contradictions

None known. Only one card in this set uses the concept.

Within the matting paper: Table 1 shows that model (4), which adds the alpha
feature loss to model (3), has a higher SAD (38.8 against 35.8) and a higher
MSE [arxiv:1909.09725 tab. 1]. The card records no statement from the paper
that this loss improves SAD, so check the paper's discussion before citing one.

## Papers

- [Context-aware matting](/papers/1909.09725): applies VGG16 feature losses to the predicted alpha and to the foreground colour, and reports ablations with and without them.
