---
kind: concept
slug: text-watermarking
title: Text watermarking
summary: >-
  Hides a statistical bias in the word choices of generated text so a detector can later find it with a simple test. Red-green watermarks need the preceding context, which diffusion language models often lack. One paper so far adapts the method by tilting each denoising step's logits with the expected green-count gradient. Author-reported: 99% TPR at 1% FPR on LLaDA and Dream.
tags: [watermarking, diffusion-language-models, detection]
papers: ["2509.24368"]
related: [ai-generated-text-detection, discrete-diffusion]
status: draft
updated_at: 2026-10-18
---

## Overview

A text watermark biases which tokens a model picks, following a secret rule,
so that a detector can later decide whether a text came from the model
[arxiv:2509.24368 §1]. The red-green family assigns each token to a "green"
list derived from its context hash and favours green tokens. Detection counts
green tokens and runs a binomial test [arxiv:2509.24368 §3.2].

The shared problem is that the green list needs the context of each token.
In a left-to-right model that context is fixed. In a diffusion language model,
tokens are filled in any order, so a token is often sampled before some of
its context is decoded [arxiv:2509.24368 §3.1]. The shared invariant is the
detector: a binomial test on the green count, which needs the hash and the
key, not the model [arxiv:2509.24368 §3.2].

## Variants

### Expected-gradient tilt for diffusion models

At each denoising step, the method computes the distribution of each
position's context hash from the current token distributions. It then
evaluates the expected green count on the top-k tokens and hashes and adds a
scaled gradient of that count to the logits [arxiv:2509.24368 §3.2] [arxiv:2509.24368 alg. 2].
The tilt has two parts: a booster that raises green tokens in the current
position, and a greenify term that favours tokens making the next position's
hash more likely to be green [arxiv:2509.24368 §3.3] [arxiv:2509.24368 eq. 9].
Using both beats either part alone in the authors' ablation [arxiv:2509.24368 §4.3].
Card: [Watermarking Diffusion Language Models](/papers/2509.24368).

### Hash choice: SumHash or MinHash

SumHash sums token ids at the context offsets; MinHash takes the minimum of
permuted token ids [arxiv:2509.24368 §3.2]. The main experiments use SumHash
[arxiv:2509.24368 §4.1]. The authors report no significant effect of the hash
choice [arxiv:2509.24368 §4.3].

### Strength by delta or by KL budget epsilon

The strength can be a constant delta, or it can be set by bisection so that
the KL divergence from the model stays within a budget epsilon
[arxiv:2509.24368 app. G.2]. The paper recommends the delta form for the main
experiments [arxiv:2509.24368 §4.3]. The authors report that the epsilon form
gives much weaker detection [arxiv:2509.24368 §4.3].

### Naive adaptation (baseline)

The baseline applies a red-green watermark only at positions whose context is
already fixed [arxiv:2509.24368 §4.1]. The paper uses it as the comparison
point for the tilt method above.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Diffusion language model that decodes tokens in any order | Expected-gradient tilt, delta form | At context offset {-1} and delta 4, the paper reports TPR@1 of 0.99 on both models, against 0.63 (LLaDA) and 0.49 (Dream) for the baseline [arxiv:2509.24368 tab. 1]. |
| Need a bound on how far the output moves from the model | Epsilon form, with the weaker detection the authors report [arxiv:2509.24368 §4.3] | The paper calls the KL constraint an imperfect proxy for quality [arxiv:2509.24368 §4.3]. |
| Text quality must stay close to the unwatermarked model | Calibrate the strength on your own texts | Average accuracy falls from 59.4 to between 53.7 and 56.0 on LLaDA and from 50.4 to between 47.0 and 50.1 on Dream, depending on the setting [arxiv:2509.24368 tab. 1]. The paper calls the effect model-dependent [arxiv:2509.24368 §4.1]. |
| Short outputs | Expect weaker detection; plan for longer texts | Detection grows with length [arxiv:2509.24368 §4.1] [arxiv:2509.24368 fig. 2]. |
| Text may be edited after generation | Plan for partial loss | Detection holds up to about 30% of words deleted or substituted in 300-token texts, and paraphrasing reduces it [arxiv:2509.24368 §4.2] [arxiv:2509.24368 fig. 3]. |
| Autoregressive model | Not covered by this paper's card | The paper states that for autoregressive models the method reduces to the red-green watermark [arxiv:2509.24368 app. E]. This page has no card for that original method. |

## Implementation sketch

The core of one watermarked step, in the paper's own terms. Strength
normalisation differs between the appendix and the released code; see
Contradictions before choosing a value.

```pseudocode
# Booster added to the logits at one denoising step  [arxiv:2509.24368 alg. 2]
# Input:  logits (B, L, V); delta (strength); k (top-k size)
# Output: booster (B, L, V), added to the logits before sampling

def watermark_booster(logits, delta, k):
    # Token distributions at each position  [arxiv:2509.24368 §3.2]
    probs = softmax(logits / T)                              # (B, L, V)
    # Distribution over the hash of each position's context  [arxiv:2509.24368 eq. 6]
    hash_probs = convolve_context_probs(probs, context_offsets)
    # Expected green count over the top-k tokens and hashes  [arxiv:2509.24368 eq. 2]
    energy = expected_green_count(probs, hash_probs, k)      # (B,)
    # Gradient of the energy with respect to each position's distribution  [arxiv:2509.24368 §3.1]
    alpha = gradient(energy, probs)                          # (B, L, V)
    # Logit tilt  [arxiv:2509.24368 eq. 4]
    return delta * alpha
```

Detection uses the same green matrix and does not run the model:

```pseudocode
# Detection  [arxiv:2509.24368 §3.2]
# Input:  tokens (L,); context offsets C; green matrix G; green rate gamma

def detect(tokens, C, G, gamma):
    # Keep one copy of each (context hash, token) pair
    pairs = unique(zip(context_hash(tokens, C), tokens))
    # Count the pairs that are green under G
    greens = sum(G[h, t] for h, t in pairs)
    # One-sided binomial test of the green count against gamma
    return binomial_p_value(greens, len(pairs), gamma)
```

## Contradictions

- **Strength scale.** The paper gives delta = 4 for context {-1} and 5 for
  larger contexts [arxiv:2509.24368 §4.1]. Its appendix divides delta by the
  context size [arxiv:2509.24368 app. A.1]. The released configs set delta to
  2.0. The paper does not say which value produced its table, and the
  released configs do not reproduce the stated delta.
- **Remasking for Dream.** The paper says random remasking for both models
  [arxiv:2509.24368 app. A.1]. The released Dream config uses a different
  strategy. The paper does not say which one produced the Dream results.
- **Remasking changes the setting.** With entropy remasking, the appendix
  chooses delta = 3 for context {-1} to reach about 1.0 TPR@1 at 300 tokens
  [arxiv:2509.24368 app. C.5]. The detection figures depend on the remasking
  strategy, so they do not transfer across strategies without recalibration.

Author-reported figures come from a single paper and have not been reproduced
here.

## Papers

- [Watermarking Diffusion Language Models](/papers/2509.24368): adapts
  red-green watermarks to diffusion language models by tilting each denoising
  step with the expected green-count gradient; detection stays the binomial test.