---
kind: concept
slug: image-blending
title: Image blending
summary: >-
  Image blending pastes an object into a background so the seam and colours match while fine detail survives. Classical gradient-domain solvers keep detail but can look unrealistic. GP-GAN adds a learned low-resolution colour estimate and solves at full resolution. The trade-off is a trained network and paired training data.
tags: [image-editing, image-blending, gradient-domain, gan]
papers: ["1703.07195"]
related: [generative-adversarial-network, image-to-image-translation]
status: draft
updated_at: 2026-11-08
---

## Overview

Image blending places a source object into a target background so that the
result reads as one photo. The difficulty is the boundary: brightness, colour
and texture on either side of the mask edge should agree, while the object's own
fine detail should survive the edit [arxiv:1703.07195 §1]. Gradient-domain
methods keep the high-frequency detail but can leave colours that look wrong.
Learned colour models can produce realistic colour, but direct high-resolution
generation is costly [arxiv:1703.07195 §1] [arxiv:1703.07195 §3.4].

The idea that the variants share is an output at the target resolution, with
fine structure taken from the source inside the mask and the background outside
it. Colour is the part that may come from a different source.

## Variants

### Gradient-domain blending (classical baseline)

These methods solve for an image whose gradients follow the source inside the
mask and the background outside it [arxiv:1703.07195 §3.4]. The paper's
comparison set consists of Poisson image editing, modified Poisson editing and
multi-splines blending [arxiv:1703.07195 §4.3]. This page does not summarise
those methods beyond that; they have no card here.

### Learned colour plus Gaussian-Poisson solve (GP-GAN)

Card: [GP-GAN: Towards Realistic High-Resolution Image Blending](/papers/1703.07195).

- **Phase one, colour estimate.** A supervised encoder-decoder, the Blending
  GAN, maps the composite to a 64 × 64 colour image. It is trained with 0.999
  weight on an L2 term and 0.001 on an adversarial term [arxiv:1703.07195 eq. 2]
  [arxiv:1703.07195 §3.3]. Training pairs are two frames from the same fixed
  webcam, with the destination frame standing in for the ground truth
  [arxiv:1703.07195 §4.1] [arxiv:1703.07195 §3.3].
- **Phase two, full-resolution solve.** A closed-form Gaussian-Poisson objective
  (eq. 9) is solved at each level of a Laplacian pyramid, from coarse to fine.
  The colour target at the coarsest level is the network output, and each level
  passes its result on as the next target [arxiv:1703.07195 §3.4]
  [arxiv:1703.07195 alg. 1].
- **Reported result.** On 500 images, the paper reports a realism score of
  −0.069 against −0.140 for the best baseline (multi-splines blending)
  [arxiv:1703.07195 tab. 1]. In a crowd-sourced comparison GP-GAN received 947
  votes against 770 for that baseline [arxiv:1703.07195 tab. 2].

## When to use which

The paper tests one benchmark only, so the table below states constraints and
the evidence the paper gives for each.

| Situation | Use | Why |
|---|---|---|
| No training budget, or no paired scene data | Gradient-domain methods | GP-GAN needs a trained network. Training used 150K examples for 25 epochs [arxiv:1703.07195 §4.2]. The classical methods are not trained [arxiv:1703.07195 §1]. |
| Output must be high resolution and colour must look natural | GP-GAN-style two-phase blend | On the paper's benchmark it scores higher on realism and in the user study [arxiv:1703.07195 tab. 1] [arxiv:1703.07195 tab. 2]. |
| Training pairs come from fixed cameras | GP-GAN | Its pairs are two frames from one webcam [arxiv:1703.07195 §4.1]. The paper does not address unpaired data. |
| Need to compare with the paper's numbers | Same setup only | The realism score uses an external network, RealismCNN, on 500 annotated images from Transient Attributes [arxiv:1703.07195 §4.3] [arxiv:1703.07195 §4.1]. |
| Budget for run time or memory is tight | Measure first | The paper does not report run time or memory [arxiv:1703.07195 §3.4]. The solve runs over the full image at every pyramid level [arxiv:1703.07195 alg. 1]. |

## Implementation sketch

GP-GAN's two phases are the common core for that variant. Only the first phase
is learned; the second is a closed-form solve.

```pseudocode
# Two-phase blending  [arxiv:1703.07195 §3.3-§3.4] [alg. 1]
# Input:  source, background (H, W, 3) in [0, 1]; mask (H, W) in {0, 1}
# Output: blended (H, W, 3)

def blend(source, background, mask, G):
    # Phase one: learned low-resolution colour estimate  [§3.3] [eq. 2]
    small = composite_at_64px(source, background, mask)
    colour = G(small)

    # Phase two: coarse-to-fine closed-form solves on a Laplacian pyramid  [§3.4]
    for level in levels_coarse_to_fine(H, W):
        # Bring the previous estimate to this level's size  [alg. 1]
        colour = resize(colour, size_at(level))
        # Gradients: source inside the mask, background outside  [eq. 8]
        guidance = select_gradients(source[level], background[level], mask[level])
        # Solve eq. 9 at this level: gradient match plus colour match  [eq. 9]
        colour = solve_gaussian_poisson(guidance, colour, beta=1.0)
    return colour
```

Notes on the parts that differ between implementations:

- The colour weight is written as β in eq. 5 and as λ in eq. 9. The paper
  appears to use one weight under two symbols, and sets it to 1
  [arxiv:1703.07195 §3.4] [arxiv:1703.07195 eq. 5] [arxiv:1703.07195 eq. 9].
- The closed form follows Frankot and Chellappa [arxiv:1703.07195 §3.4]. The
  paper gives the discrete objective but not the transform basis or the
  Gaussian kernel size.
- The Blending GAN's encoder width is not stated in the paper. Its layer widths
  appear only in its figures [arxiv:1703.07195 fig. 3] [arxiv:1703.07195 fig. 4].

### Hyperparameters stated in the paper

| Name | Value | Where |
|---|---|---|
| L2 weight λ (phase one) | 0.999 | [arxiv:1703.07195 eq. 2] [arxiv:1703.07195 §3.3] |
| Colour-preserving weight β | 1 | [arxiv:1703.07195 §3.4] [arxiv:1703.07195 eq. 5] |
| Generator Adam learning rate, β1 | 0.002, 0.5 | [arxiv:1703.07195 §4.2] |
| Batch size, epochs, training examples | 64, 25, 150K | [arxiv:1703.07195 §4.2] |
| Network input size | 64 × 64 | [arxiv:1703.07195 fig. 5] |

Not stated in the paper: the critic's learning rate and update schedule, the
encoder bottleneck width, the Gaussian σ in eq. 7 and eq. 9, the Laplacian
smoothing, and the number of pyramid levels [arxiv:1703.07195 §3.3]
[arxiv:1703.07195 §3.4].

## Contradictions

- **Wording against the table.** The paper says most users preferred GP-GAN
  [arxiv:1703.07195 §4.4]. In its vote table, GP-GAN has 947 of 2,979 votes,
  about 32%, and the table does not show a majority over the other methods
  [arxiv:1703.07195 tab. 2]. Use the table's numbers, not the sentence.
- **Other papers.** None known. This page has one paper, and its benchmark
  results have no independent replication on this page.

## Papers

- [GP-GAN: Towards Realistic High-Resolution Image Blending](/papers/1703.07195):
  a supervised low-resolution GAN colour estimate, followed by a multi-scale
  closed-form Gaussian-Poisson solve.