---
kind: concept
slug: langevin-dynamics
title: Langevin dynamics
summary: >-
  Samples from a distribution given by an energy, by repeating a gradient step downhill plus Gaussian noise. Only one card so far: COLD decoding applies it to a soft text sequence with weighted constraint terms, then maps each position to a token. Author-reported: 94.50% keyword coverage on CommonGen, at higher perplexity than NeuroLogic.
tags: [sampling, decoding, energy-based-models]
papers: ["2202.11705"]
related: [constrained-decoding, controllable-text-generation, decoding-strategies, diffusion-models]
status: draft
updated_at: 2026-10-19
---

## Overview

Langevin dynamics draws samples from a target distribution that is defined
through an energy: lower energy means higher probability. The state moves
downhill on the energy, and Gaussian noise is added at each step so the
chain does not collapse onto a single minimum [arxiv:2202.11705 §2]. The
invariant the method relies on is that, with a small enough and slowly
decreasing step, and with the noise vanishing, the chain's distribution
converges to the target. The paper states this only in that limit
[arxiv:2202.11705 §3.1].

The update is one line. Take the gradient of the energy at the current state,
step against it with a fixed size, and add noise [arxiv:2202.11705 eq. 2].
Everything else in a variant is a choice of energy, of how the noise shrinks,
and of how the continuous state becomes an output.

## Variants

Only one paper card covers this concept so far. Its variant is for text, and
the state is a soft sequence rather than a token list.

### Soft-sequence sampling for constrained text (COLD)

Each position holds a vector of logits over the vocabulary, not a token. The
energy is a weighted sum of constraint scores: fluency under a left-to-right
language model, fluency under a right-to-left model, the likelihood of fixed
right-hand words, and an n-gram match to a reference [arxiv:2202.11705 eq. 1]
[arxiv:2202.11705 §3.2]. The sample is then turned into tokens one position at
a time, choosing among the language model's top-k next tokens by the largest
soft logit [arxiv:2202.11705 eq. 6] [arxiv:2202.11705 §3.3]. The language model
stays frozen, with no fine-tuning [arxiv:2202.11705 §4]. The paper applies the
method to abductive reasoning, counterfactual rewriting and lexically
constrained generation, each with its own energy [arxiv:2202.11705 §4.1.1]
[arxiv:2202.11705 §4.2] [arxiv:2202.11705 §4.3].

The card is [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](/papers/2202.11705).

## When to use which

With one variant on record, the choice is between this method and other ways
to meet a constraint during decoding. The comparisons below use only the
paper's own tables.

| Situation | Use | Why |
|---|---|---|
| Constraints are keywords or soft context, and some violation is acceptable | Langevin sampling over soft logits | Coverage reached 94.50% on CommonGen, above the 91.00% the paper reports for NeuroLogic [arxiv:2202.11705 tab. 3] |
| Fluency matters more than keyword coverage | Not this method on the paper's numbers | Perplexity was 54.98 against 28.61 for NeuroLogic, and human fluency was 2.07 against 2.53 [arxiv:2202.11705 tab. 3] |
| Output must always satisfy a hard rule, such as a grammar | Hard masking, see [constrained decoding](/concepts/constrained-decoding) | Langevin terms are preferences, and the paper's coverage is below 100% [arxiv:2202.11705 tab. 3] |
| Need to rewrite text while keeping most of it | This method with the n-gram term | The counterfactual energy keeps the original ending through an n-gram match, and the paper reports 56.84 min-edit overlap against 52.90 for DeLorean [arxiv:2202.11705 tab. 2] |
| Latency or compute is tight | Check the budget first | About 33.6 s per sample on GPT2-XL, and 22.7 s on GPT2-M [arxiv:2202.11705 tab. 5]. Each sample runs 2000 update steps [arxiv:2202.11705 §4.1.1] |

The per-sample cost is before sample-and-select. The paper draws 16 or 32
samples per input and picks one, so total cost scales with that count
[arxiv:2202.11705 app. B].

## Implementation sketch

The core loop below follows the paper's equations. The step size and the
noise schedule are not given in the paper's text, so they are parameters
here. Values and the released code's differences are in the variant card.

```pseudocode
# Langevin update on a soft sequence  [eq. 2] [§3.1]
# Input:  energy E (differentiable), initial soft state x0, step eta
# Output: soft state x after n_steps

def langevin_sample(E, x0, eta, sigma, n_steps):
    x = x0
    for n in range(n_steps):
        # Gradient of the energy at the current state  [eq. 2]
        grad = grad_E(E, x)
        # Gaussian noise; the scale is set by the schedule  [§3.4]
        eps = normal(0, sigma(n))
        # Step downhill on the energy, then add the noise  [eq. 2]
        x = x - eta * grad + eps
    return x

# Map the soft state to tokens, left to right  [eq. 6] [§3.3]
# candidates: the LM's top-k next tokens given the tokens so far
def discretise(x, lm, k):
    tokens = []
    for t in range(len(x)):
        candidates = lm_top_k(lm, tokens, k)
        tokens.append(argmax_over(x[t], candidates))
    return tokens
```

## Contradictions

Within the one paper on record, the description and the equations disagree in
two places:

- **Where the noise goes.** The text of the method section says the noise is
  added to the gradient [arxiv:2202.11705 §3.4]. The update equation adds it
  to the state [arxiv:2202.11705 eq. 2]. Both readings are given here; the
  equation is the one used in the pseudocode above.
- **Update rule, paper against released code.** The equation is a plain step
  of size eta [arxiv:2202.11705 eq. 2]. The released code does not follow it
  literally: it passes eta to an Adam optimiser. The difference is recorded in
  the [paper card](/papers/2202.11705), not in the paper's text. An
  implementation that follows the equation will not match the code's
  behaviour, so pick one and say which.

No other paper in the corpus covers this concept yet, so there is nothing to
compare across papers.

## Papers

- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](/papers/2202.11705): applies Langevin sampling over soft token logits to constrained text generation with a frozen GPT2-XL.