Langevin dynamics

Overview

Langevin dynamics draws samples from a target distribution that is defined through an energy: lower energy means higher probability. The state moves downhill on the energy, and Gaussian noise is added at each step so the chain does not collapse onto a single minimum 2202.11705 §2. The invariant the method relies on is that, with a small enough and slowly decreasing step, and with the noise vanishing, the chain's distribution converges to the target. The paper states this only in that limit 2202.11705 §3.1.

The update is one line. Take the gradient of the energy at the current state, step against it with a fixed size, and add noise 2202.11705 eq. 2. Everything else in a variant is a choice of energy, of how the noise shrinks, and of how the continuous state becomes an output.

Variants

Only one paper card covers this concept so far. Its variant is for text, and the state is a soft sequence rather than a token list.

Soft-sequence sampling for constrained text (COLD)

Each position holds a vector of logits over the vocabulary, not a token. The energy is a weighted sum of constraint scores: fluency under a left-to-right language model, fluency under a right-to-left model, the likelihood of fixed right-hand words, and an n-gram match to a reference 2202.11705 eq. 1 2202.11705 §3.2. The sample is then turned into tokens one position at a time, choosing among the language model's top-k next tokens by the largest soft logit 2202.11705 eq. 6 2202.11705 §3.3. The language model stays frozen, with no fine-tuning 2202.11705 §4. The paper applies the method to abductive reasoning, counterfactual rewriting and lexically constrained generation, each with its own energy 2202.11705 §4.1.1 2202.11705 §4.2 2202.11705 §4.3.

The card is COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics.

When to use which

With one variant on record, the choice is between this method and other ways to meet a constraint during decoding. The comparisons below use only the paper's own tables.

The per-sample cost is before sample-and-select. The paper draws 16 or 32 samples per input and picks one, so total cost scales with that count 2202.11705 app. B.

Implementation sketch

The core loop below follows the paper's equations. The step size and the noise schedule are not given in the paper's text, so they are parameters here. Values and the released code's differences are in the variant card.

# Langevin update on a soft sequence  [eq. 2] [§3.1]
# Input:  energy E (differentiable), initial soft state x0, step eta
# Output: soft state x after n_steps

def langevin_sample(E, x0, eta, sigma, n_steps):
    x = x0
    for n in range(n_steps):
        # Gradient of the energy at the current state  [eq. 2]
        grad = grad_E(E, x)
        # Gaussian noise; the scale is set by the schedule  [§3.4]
        eps = normal(0, sigma(n))
        # Step downhill on the energy, then add the noise  [eq. 2]
        x = x - eta * grad + eps
    return x

# Map the soft state to tokens, left to right  [eq. 6] [§3.3]
# candidates: the LM's top-k next tokens given the tokens so far
def discretise(x, lm, k):
    tokens = []
    for t in range(len(x)):
        candidates = lm_top_k(lm, tokens, k)
        tokens.append(argmax_over(x[t], candidates))
    return tokens

Contradictions

Within the one paper on record, the description and the equations disagree in two places:

• Where the noise goes. The text of the method section says the noise is added to the gradient 2202.11705 §3.4. The update equation adds it to the state 2202.11705 eq. 2. Both readings are given here; the equation is the one used in the pseudocode above.
• Update rule, paper against released code. The equation is a plain step of size eta 2202.11705 eq. 2. The released code does not follow it literally: it passes eta to an Adam optimiser. The difference is recorded in the paper card, not in the paper's text. An implementation that follows the equation will not match the code's behaviour, so pick one and say which.

No other paper in the corpus covers this concept yet, so there is nothing to compare across papers.

Papers

• COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics: applies Langevin sampling over soft token logits to constrained text generation with a frozen GPT2-XL.