---
kind: concept
slug: image-enhancement
title: Image enhancement
summary: >-
  Restoring the light, exposure or colour of dark, backlit or underwater photos. Four variants: a tiny supervised transformer for light and exposure, an unsupervised CLIP-guided illumination map for backlit photos, and two paired underwater models, a transformer GAN and a CNN that blends RGB and HSV. The trade-off is paired data and model size against unpaired data and CLIP compute.
tags: [image-enhancement, low-light, exposure-correction, backlit, underwater, unpaired-training, color-space]
papers: ["2205.14871", "2303.17569", "2111.11843", "2103.07138"]
related: [image-denoising, vision-transformer, depthwise-separable-convolution, object-queries, vision-language-pretraining, soft-prompt-tuning, generative-adversarial-network, channel-attention, convolutional-neural-network, attention-mechanism]
status: draft
updated_at: 2026-10-31
---

## Overview

The task is to take a photo whose light or colour is wrong and produce a
version that looks correct. The light cases are low light, backlight, over- or
under-exposure and uneven lighting [arxiv:2205.14871 §1] [arxiv:2303.17569 §1].
The colour case is underwater: water absorbs and scatters light, which causes
colour casts, colour artefacts and blur [arxiv:2111.11843 §1]. Underwater
images are often bluish or greenish, low in brightness and hazy
[arxiv:2103.07138 §IV-C]. In all four variants the fix is a learned mapping from
the input to a corrected image. The papers differ on where the correction comes
from and what it is made of.

Variants differ in two choices: where the correction comes from (paired
ground-truth photos, or a frozen vision-language model), and what the correction
is made of (per-pixel affine terms with a global colour matrix and gamma, a
single illumination map that the image is divided by, a full image-to-image
generator, or curves in two colour spaces blended per pixel).

## Variants

### Per-pixel and global adjustment with paired training data

Predicts a per-pixel multiply and add, a 3x3 colour matrix and a gamma for each
image, and is trained with a supervised loss on paired photos
[arxiv:2205.14871 §3.1] [eq. 4]. The local branch works at full resolution
with depth-wise convolutions, and the global branch uses ten learned queries
over a low-resolution encoding [arxiv:2205.14871 §3.2] [fig. 3]. The whole model
has about 0.09M parameters, and the authors report about 0.004 s per image
[arxiv:2205.14871 tab. 1] [tab. 2]. Card: [IAT](/papers/2205.14871).

Author-reported results: PSNR 23.50 and SSIM 0.824 on LOL-V2-real [arxiv:2205.14871 tab. 1]; PSNR 25.32 and SSIM 0.920 on MIT-Adobe FiveK, expert C targets [arxiv:2205.14871 tab. 2]; average PSNR 20.34 on the exposure-correction set, which has five expert targets [arxiv:2205.14871 tab. 3].

### Unsupervised illumination map guided by CLIP

Trains an enhancer with no paired targets. A frozen CLIP model scores outputs
against a positive and a negative prompt, and both prompts are learned
[arxiv:2303.17569 §3.1]. The enhancer predicts a one-channel illumination map,
and the output is the input divided by that map [arxiv:2303.17569 §3.1]. An
identity loss keeps the output close to the input's content [arxiv:2303.17569
§3.1]. Training alternates prompt refinement, which uses a margin ranking loss,
with enhancer tuning [arxiv:2303.17569 §3.2]. Card: [CLIP-LIT](/papers/2303.17569).

Author-reported results on the BAID backlit test set (368 images, photographer
retouched references): PSNR 21.579, SSIM 0.883, LPIPS 0.159 and MUSIQ 55.682,
against 19.740 PSNR for the best released baseline, Zero-DCE [arxiv:2303.17569
tab. 1]. Replacing the learned prompts with fixed words drops PSNR to 14.748
[arxiv:2303.17569 tab. 3]. Training uses 380 backlit and 384 well-lit images,
unpaired [arxiv:2303.17569 §4].

### Paired transformer GAN for underwater colour

A U-shaped generator trained as a GAN on pairs of underwater photos and
reference images. The skip connections are replaced by a channel-wise
transformer, and a spatial transformer sits at the bottleneck
[arxiv:2111.11843 §3.2.1] [arxiv:2111.11843 §3.2.2] [arxiv:2111.11843 §3.2.3].
The loss adds colour terms in LAB and LCH spaces to an RGB term, a perceptual
term and the adversarial term [arxiv:2111.11843 §3.3] [eq. 13]. Training data
come from LSUI, which the authors built from 18 existing enhancers, image-quality
scores and volunteer votes [arxiv:2111.11843 §3.1].

Author-reported results: PSNR 24.16 and SSIM 0.93 on Test-L400, and PSNR 22.91
and SSIM 0.91 on Test-U90, the best of the ten compared methods
[arxiv:2111.11843 tab. 3]. In the ablation, the pix2pix baseline reaches PSNR
19.34 on Test-L400 and adding only the channel transformer reaches 22.47
[arxiv:2111.11843 tab. 6]. Cost at 256 × 256: 65.6 M parameters, 66.2 GFLOPs and
0.07 s per image on the authors' hardware [arxiv:2111.11843 tab. 3]. Card:
[U-shape Transformer](/papers/2111.11843).

### Paired CNN with RGB and HSV branches for underwater colour

Three CNN blocks trained end to end on paired underwater photos, with no
adversarial term. An RGB block removes noise and colour cast. An HSV block
predicts four piecewise-linear curves, for value, saturation and hue, from
global pooling [arxiv:2103.07138 §III-A] [eq. 1]. An attention block takes the
input and both outputs and predicts six weight maps, which blend the RGB and
HSV outputs per pixel and per channel [arxiv:2103.07138 §III-A]. The loss sums
L1, SSIM, an HSV term and a VGG-19 perceptual term [arxiv:2103.07138 §III-B]
[eq. 2]. Training uses 800 real pairs from UIEBD and 1200 synthetic pairs
built from NYU-v2 RGB-D images [arxiv:2103.07138 §IV-A]. The paper says the
model has far fewer parameters than deep CNNs but gives no count, runtime or
memory figure [arxiv:2103.07138 §I]. Card: [UIEC^2-Net](/papers/2103.07138).

Author-reported results on 90 real and 900 synthetic test images: PSNR 24.5663
and SSIM 0.9346 on the real set, and PSNR 20.5442 and SSIM 0.8486 on the
synthetic set, the best full-reference scores in its tables [arxiv:2103.07138
tab. II] [tab. I]. On real images it also reports the best UCIQE, UISM and
UIQM among the compared methods [arxiv:2103.07138 tab. III]. Its own ablation
shows the RGB block alone reaches PSNR 23.8228 on the real set, against 24.5663
for the full model [arxiv:2103.07138 tab. IV].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Paired low-light or mis-exposed photos with targets exist | Per-pixel and global adjustment | It is supervised and needs the pairs [arxiv:2205.14871 §4] |
| Only unpaired photos, mainly backlit scenes | CLIP-guided illumination map | It trains without targets, from two unpaired sets [arxiv:2303.17569 §4] |
| Model must be very small (edge or mobile) | Per-pixel and global adjustment | About 0.09M parameters and about 1.44 GFLOPs as reported [arxiv:2205.14871 tab. 1] |
| Enhancement feeds a detector or segmenter | Per-pixel and global adjustment | The authors train it jointly with a detector; the gain depends on initialisation [arxiv:2205.14871 tab. 6] |
| Frozen CLIP is acceptable, and per-iteration cost is not a hard limit | CLIP-guided illumination map | Each update runs CLIP passes on the output and the input [arxiv:2303.17569 §3] |
| Underwater colour cast, paired references available, GPU budget, and a generator-style model is wanted | Paired transformer GAN | It reports the best scores on its own sets [arxiv:2111.11843 tab. 3]. It is about 65 M parameters [arxiv:2111.11843 tab. 3] |
| Underwater colour cast, paired references available, and no adversarial training wanted | Paired CNN with RGB and HSV branches | Its loss has no adversarial term [arxiv:2103.07138 §III-B] [eq. 2]. It reports the best full-reference scores on its own real and synthetic sets [arxiv:2103.07138 tab. II] [tab. I]. It gives no parameter count [arxiv:2103.07138 §I] |
| Underwater photos and no paired references | None of the four, as tested | Every underwater model here trains on pairs, real or synthetic [arxiv:2111.11843 §3.1] [arxiv:2103.07138 §IV-A]. The unpaired model is tested on backlit scenes [arxiv:2303.17569 §4] |
| Code must be reused as is | Check each licence first | The first paper's code is Apache-2.0 [arxiv:2205.14871 §1]. The U-shape code is MIT [code: LICENSE]. The CLIP-LIT code has no licence file, so implement from the paper only [arxiv:2303.17569 §1]. The UIEC^2-Net code has no licence file, so cite it by path only [code: README.md#L11] |

Compare numbers only on the same test set. The four variants report on
different datasets and metrics, so their scores do not rank against each other.

## Implementation sketch

The common core is one forward pass: predict a correction from the image, apply
it, and clamp to the valid range. The variants differ in how the correction is
built and applied.

```pseudocode
# Common core of image enhancement  [arxiv:2205.14871 §3.2] [arxiv:2303.17569 §3.1]
# Input:  sRGB image in [0, 1], shape (B, 3, H, W)
# Output: enhanced sRGB image in [0, 1]

def enhance(image):
    # Variant A: predict a per-pixel multiply and add, and a global colour
    # matrix and gamma  [arxiv:2205.14871 §3.2] [eq. 4]
    multiply_map, add_map = local_branch(image)
    colour_matrix, gamma = global_branch(image)
    adjusted = image * multiply_map + add_map
    coloured = apply_colour(colour_matrix, adjusted)
    out = clamp(coloured, 1e-8, 1.0) ** gamma
    # Variant B: predict one illumination map and divide by it  [arxiv:2303.17569 §3.1]
    # illumination = enhancer(image)           # (B, 1, H, W)
    # out = image / (illumination + epsilon)   # keep a small epsilon guard
    # Variant C: U-shaped generator, no division or gamma  [arxiv:2111.11843 §3.2.1]
    # out = generator(image)                   # trained with GAN and colour losses
    # Variant D: RGB block, HSV curves, then per-pixel blend  [arxiv:2103.07138 §III-A]
    # rgb_out = rgb_block(image)               # (B, 3, H, W)
    # hsv_out = hsv_curves(rgb_out)            # (B, 3, H, W), curves from pooling
    # w = attention_block(concat([image, rgb_out, hsv_out], dim=1))  # (B, 6, H, W)
    # out = w[:, :3] * rgb_out + w[:, 3:] * hsv_out
    return clamp(out, 0.0, 1.0)
```

Notes on the variants:

- Variant A's gamma needs a floor on the base before the power, otherwise the
  power is undefined at zero [arxiv:2205.14871 §3.2] [eq. 4].
- Variant B's division is sensitive near zero, so keep an epsilon in the
  denominator and clamp the result [arxiv:2303.17569 §3.1].
- Variant B's CLIP scoring depends on a logit scale that the paper's equations
  do not show. Without it, the margin used in refinement cannot be reached
  [arxiv:2303.17569 §3.2] [eq. 2] [eq. 6]. Keep the scale when you implement.
- Variant B's enhancer is not a standard encoder-decoder U-Net, despite the
  paper's wording. Do not build one from that phrase alone [arxiv:2303.17569 §3.1].
- Variant C's loss weights in [eq. 13] are tied to the reduction the released
  code uses. The colour cross-entropy sums over pixels there. With a per-pixel
  mean, the same weights would count that term far less [arxiv:2111.11843 §3.3]
  [eq. 13] [code: loss/LAB.py#L31].
- Variant C's released generator has no output activation, so the output is
  unbounded. Clamping it, or adding a sigmoid, changes what is learned
  [code: net/block.py#L420-L441].
- Variant D's curve sum in [eq. 1] needs a knot beyond the last one, which the
  paper does not define. The released code sums one step fewer, with M = 11
  knots [arxiv:2103.07138 §III-A] [eq. 1] [code: core/Models/UWModels/UIEC2Net.py#L173].
- Variant D's attention prose lists the HSV output first, but its channel
  description, and the released code, give the first three maps to the RGB
  output [arxiv:2103.07138 §III-A] [code: core/Models/UWModels/UIEC2Net.py#L160-L163].
- Variant D's RGB to HSV conversion divides by V and by V − min. The paper
  gives no grey-case handling, so keep a small epsilon in both denominators
  [arxiv:2103.07138 §III-C] [eq. 9] [eq. 10] [code: core/Models/blocks/RGB2HSV.py#L15].

## Contradictions

- **CLIP-LIT's PSNR claim.** Its main table says the method has the best PSNR on
  the BAID test set, at 21.579 [arxiv:2303.17569 tab. 1]. Its retrained
  supervised table gives Zhao et al. a BAID PSNR of 22.123, above that value
  [arxiv:2303.17569 tab. 7]. The same paper gives both numbers, and this page
  does not resolve which is the right comparison.
- **IAT's efficiency and quality claim.** The text says the method is best on
  both quality and efficiency on LOL [arxiv:2205.14871 §4.1]. The same paper's
  table gives MAXIM higher PSNR and SSIM on LOL-V1 (23.43 and 0.863, against
  23.38 and 0.809) [arxiv:2205.14871 tab. 1].
- **IAT's exposure average.** The table gives an average PSNR of 20.34 for IAT
  on the exposure set [arxiv:2205.14871 tab. 3]. The five per-expert values in
  the same paper average to 20.40 [arxiv:2205.14871 tab. 3]. This page quotes the
  table value.
- **The underwater paper's "more than 2 dB" claim.** The abstract says the
  method is more than 2 dB better [arxiv:2111.11843 abstract]. From its table,
  the gap is 2.13 dB on Test-U90 against Ucolor, but 1.25 dB on Test-L400
  [arxiv:2111.11843 tab. 3]. The claim holds only on the first set.
- **The underwater paper's efficiency claim.** The results text says the model
  has relatively few parameters, FLOPs and running time [arxiv:2111.11843 §4.5].
  Its own table gives 66.2 GFLOPs and 65.6 M parameters, above FUnIE and UIE-DAL.
  Its runtime is below WaterNet, Ucolor and FUnIE, but UGAN is faster
  [arxiv:2111.11843 tab. 3].
- **UIEC^2-Net's training loss, paper against code.** The paper trains with L1,
  SSIM, an HSV term and a perceptual term [arxiv:2103.07138 §III-B] [eq. 2]. The
  shipped training script computes only the SSIM loss on the network output
  [code: train.py#L165-L172]. The released config also uses other weights and
  a 1000-epoch schedule, against 50 epochs in the paper [arxiv:2103.07138 §IV-A].
  The page follows the paper, and the code is a different run.
- **The underwater dataset size.** The LaTeX source gives 4279 image pairs
  [arxiv:2111.11843 §3.1]. The arXiv abstract metadata gives 5004. This page
  uses 4279.
- **Across the four papers.** Their datasets, metrics and training data differ,
  so no result contradicts another. Do not read one as a ranking of the variants.

## Papers

- [IAT](/papers/2205.14871): per-pixel and global adjustment with about 0.09M parameters, trained on paired data.
- [CLIP-LIT](/papers/2303.17569): unpaired training with learned CLIP prompts and an illumination map, evaluated on backlit photos.
- [U-shape Transformer](/papers/2111.11843): paired transformer GAN for underwater colour, with the LSUI dataset.
- [UIEC^2-Net](/papers/2103.07138): paired CNN with RGB and HSV branches and per-pixel attention, for underwater colour.