A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models

In plain words

Controllable text generation (CTG) means getting a language model to write text that meets a stated condition: a sentiment, a topic, a set of keywords, a persona, a storyline or a fixed format. This survey reviews how that is done with large pre-trained transformer models, which are first trained on huge amounts of text and then used for many tasks.

The authors sort the methods by where the control signal enters. One family adjusts the model's weights with extra modules, prompts or reinforcement signals. A second family retrains or redesigns the model so it takes the condition directly. A third family leaves the model's weights fixed and steers its word choices while it writes. The third family is cheap to train but can be slow when generating, and the text quality can drop.

The survey also lists the common tasks (dialogue, storytelling, data-to-text and others), the main pre-trained model families, and the ways people check whether the output met the condition and still reads well. Its main message is that the field is young: there are few dedicated benchmarks, and automatic evaluation is still weak.

TL;DR

A survey with no experiments of its own. It defines CTG as generating text conditioned on a control element, lists seven task families, and groups PLM-based methods into fine-tuning, retraining/refactoring and post-processing. It also reviews general and CTG-specific evaluation metrics and lists open challenges. Use it as a map of the field; for implementation details, go to the primary papers it cites.

Problem

Pre-trained language models (PLMs) generate fluent text, but they are black boxes and hard to steer toward a required content, style or structure §1. Applications need such constraints: story endings, emotion, persona, debiasing, data augmentation and format §1 §2.2. Fine-tuning all weights becomes resource-intensive as models grow, and retraining needs large labeled data §3.1. The paper aims to give what the authors say earlier CTG surveys lack: a systematic review of tasks, approaches and evaluation, and a tracking of recent large-PLM CTG work §1.

Scope assumptions: only Transformer-based PLMs §2.3; literature from roughly the last 3-4 years, preferring leading NLP venues and highly cited or widely discussed open-source work §1.

Method

This is a taxonomy, not an algorithm. The steps below are the paper's structure, in its order.

1. Formal definition. A CTG system generates a text Y from a control element C, aiming at P(Y|C) §2.1 eq. 1. Control conditions fall into three types: semantic, structural and lexical §2.1 fig. 3. Language modelling factorises a text token by token §2.3 eq. 2. The conditional form multiplies the per-token probabilities given the earlier tokens and C §2.3 eq. 3.

2. Tasks. The survey lists seven families §2.2 tab. 1:

   • attribute-based generation (topic, tense, politeness, sentiment, keywords);
   • dialogue generation (persona, politeness, emotion, templates, ground-truth references);
   • storytelling (story structure, ending, topic, persona);
   • data-to-text (structure from tables or graphs);
   • data augmentation (replaced slot values);
   • debiasing (political, gender, subjective, social and sentiment bias, toxicity);
   • format control (length, rhythm, classical poetry and couplets).

3. PLM families. Typical Transformer PLMs are grouped as auto-encoding (AE), auto-regressive (AR) and sequence-to-sequence (Seq2Seq). AE and Seq2Seq models use bidirectional attention; AR models use causal attention, and the authors prefer large AR models for complex reasoning, but AE and Seq2Seq for fill-in-the-blank and summarisation-like tasks §2.3. tab. 2 summarises the models:

   Name	Model type	Infrastructure	Pre-training task	Main application
   BERT	AE	Encoder	MLM+NSP	NLU
   XLNET	AR	Transformer-XL	PLM	NLU
   GPT2, GPT3	AR	Decoder	SLM	NLG
   T5	Seq2Seq	Encoder+Decoder	CTR	NLG+NLU
   mBART	Seq2Seq	Encoder+Decoder	FTR	NLG
   UniLM	AE+AR+Seq2Seq	Encoder+Decoder	SLM+CTR+NSP	NLG+NLU
   ERNIE-T	AE	Encoder	CTR+NSP	NLU
   XLM	Seq2Seq	Encoder+Decoder	TLM	NLG
   Bloom	AR	Decoder	SLM	NLG
   PaLM	AR	Decoder	SLM	NLG
   LLaMA	AR	Decoder	SLM	NLG

   For CTG, most methods treat a PLM as a conditional generator, with the control C entering the conditional probability §2.3 eq. 3.

4. Three approach families. Methods are grouped by how the PLM is used §3.1 fig. 4 tab. 3. Fine-tuning and retraining change the PLM's weights; post-processing keeps them fixed and acts at decoding time §3.1.

   4a. Fine-tuning §3.2:

   • Adapted modules. An extra module is trained around a PLM. Auxiliary Tuning adds the logits of a condition model to the PLM's logits §3.2. StructAdapter inserts adapters after each feed-forward sub-layer, freezes the PLM and trains only the adapters for graph-to-text §3.2. Adapter-Bot adds lightweight adapters on DialoGPT for dialogue §3.2.
   • Prompt-based. The method moves from manual templates to discrete token search and to continuous virtual tokens §3.2. Prefix-tuning freezes the PLM and optimises a small continuous prefix §3.2. P-tuning builds the template from continuous virtual tokens learned by gradient descent §3.2. Other variants train attribute-specific prefixes jointly with contrastive learning §3.2, use an attribute discriminator to assist prompt tuning (DisCup) §3.2, or combine single-attribute prompts for multi-attribute control (Tailor) §3.2. Inverse Prompt uses reverse prediction of the prompt during beam search §3.2. Non-residual prompting uses an auxiliary trainable GPT-2 as a prompt encoder §3.2.
   • Reinforcement learning. The reward says whether the control condition holds. The fine-tuning objective maximises the expected reward §3.2 eq. 4, and the reward is reduced by a KL penalty that keeps the tuned policy close to the original PLM §3.2 eq. 5. The survey gives examples: a reward model trained on human preferences §3.2; RL on GPT-2 for data augmentation §3.2; reward shaping for story plot points, shown on an LSTM model §3.2.
   • Instruction tuning. FLAN fine-tunes on a mixture of more than 60 NLP datasets written as natural-language instructions, and generalises to unseen tasks §3.2. Chung et al. scale the number of tasks and models and add chain-of-thought data §3.2. InstructGPT collects labelled demonstrations to fine-tune GPT-3, then applies RLHF §3.2.

   4b. Retraining or refactoring §3.3:

   • CTRL trains a language model conditioned on control codes prepended to the corpus, so that p(x_i | x_<i) becomes p(x_i | x_<i, c). It has 1.63 billion parameters and was trained on a 140 GB corpus §3.3. It also proposes a top-k sampling variant that penalises already-generated tokens §3.3 eq. 6.
   • POINTER is an insertion-based model for hard lexical constraints, trained from scratch; it fills in constrained words first and then inserts finer words between them §3.3.
   • CBART adds a token-level classifier to BART's encoder to predict where to replace and insert, then decodes the edits in parallel §3.3.
   • CoCon injects a control block into GPT and trains with four self-supervised losses (self-reconstruction, null content, cycle reconstruction, adversarial) §3.3.
   • Mention Flags add a mention-status embedding to the decoder so that the model is pushed to satisfy all lexical constraints §3.3.
   • Two personalised or multi-task dialogue models are also covered §3.3.

   4c. Post-processing §3.4. The PLM stays fixed; the control signal reshapes the next-token distribution at decoding time. This extends standard decoders (greedy, constrained beam search, top-k, nucleus) §3.4.

   • Guided strategies keep the PLM separate from the guide, which acts only at inference. PPLM takes gradients of an attribute model and nudges the PLM's hidden activations §3.4. MEGATRON-CNTRL predicts keywords, retrieves knowledge, ranks candidate sentences and feeds GPT-2 §3.4. FAIR builds a content plan with BERT, fills templates with BART and refines iteratively §3.4. Discriminator-guided methods include Adversarial Search (DAS), GeDi, DExperts, FUDGE, Plug-and-Blend, K2T (shifts the vocabulary toward words similar to a keyword), Mix and Match (Metropolis-Hastings with BERT proposals) and COLD decoding (Langevin dynamics) §3.4.
   • Trainable strategies train a module jointly with the frozen PLM. The energy-based model P(x) proportional to P_LM(x) exp(-E(x)) is trained with noise-contrastive estimation §3.4 eq. 10. The GDC method defines a target distribution from constraint functions and fits the policy to it with a KL-adaptive distributional policy gradient §3.4 eq. 11 eq. 12 eq. 13 eq. 14.

   The GeDi step is the most reusable piece of the post-processing family. Its contrast between a desired and an anti-control class is given by eq. 7 and eq. 8, and it is combined with the PLM by Bayes' rule eq. 9:

   P(c \mid x_{1:t}) = \frac{P(c)\prod_{j=1}^{t} P_\theta(x_j \mid x_{<j}, c)}{\sum_{c' \in \{c,\bar{c}\}} P(c') \prod_{j=1}^{t} P_\theta(x_j \mid x_{<j}, c')} eq. 8

   P_w(x_t \mid x_{<t}, c) \propto P_{LM}(x_t \mid x_{<t})\, P_\theta(c \mid x_{1:t}) eq. 9

   PseudocodeCopy

   # Guided next-token distribution, GeDi-style  [§3.4] [eq. 7-9]
   # Input:  prefix (token ids), desired code c, anti-code c_bar,
   #         priors P(c), P(c_bar) (not stated in the paper)
   # Output: distribution over the vocabulary, shape (V,)
   
   def guided_next_token_probs(prefix, lm, cc_lm, code, anti_code, prior):
       # Fixed PLM next-token distribution  [§3.4]
       lm_probs = lm.next_token_probs(prefix)              # (V,)
       # Extend the prefix by every candidate token, one batch  [eq. 8]
       extended = append_all_tokens(prefix)                # (V, t)
       # Joint likelihood of each extended prefix under each code  [eq. 7]
       log_like_code = cc_lm.sequence_log_prob(extended, code)
       log_like_anti = cc_lm.sequence_log_prob(extended, anti_code)
       # Log-scores with the class priors  [eq. 8]
       score_code = log_like_code + log(prior[code])
       score_anti = log_like_anti + log(prior[anti_code])
       # Desired-code posterior; sigmoid form is equal to eq. 8  [eq. 8]
       class_post = sigmoid(score_code - score_anti)       # (V,)
       # Bayes-rule combination with the PLM distribution  [eq. 9]
       combined = lm_probs * class_post
       return combined / sum(combined)

5. Evaluation §4 fig. 7. General metrics judge fluency, factuality, grammar and diversity, by human, automatic or semi-automatic methods §4.1. Human-centric methods include direct judgments (binary, Likert, RankME) and indirect judgments through downstream task success §4.1.1 fig. 8. Automatic metrics compare generated text to references: lexical (BLEU-n eq. 15, ROUGE-n eq. 16, Self-BLEU, Distinct-n eq. 18, PPL eq. 17, Reverse PPL), syntactic (TESLA variants) and semantic (BERTScore, BLEURT, MAUVE) §4.1.2. Semi-automatic methods train a scorer against human judgments or a discriminator over human and model outputs §4.1.3. CTG-specific metrics check the control: accuracy of a classifier for semantic conditions, rule-based coverage or success rate for lexical and structural conditions, and human relevance ratings; CTRLEVAL is a reference-free PLM-based score §4.2.

   \text{BLEU-n} = \frac{\sum_{t \in G} \sum_{g \in t} Count_{match}(g)}{\sum_{t \in G} \sum_{g \in t} Count(g)} eq. 15

   \text{ROUGE-n} = \frac{\sum_{t \in G} \sum_{g \in t} Count_{match}(g)}{\sum_{t \in R} \sum_{g \in t} Count(g)} eq. 16

   Both use the same numerator, and differ only in the denominator: BLEU-n uses the generated n-grams (precision), ROUGE-n the reference n-grams (recall) §4.1.2 eq. 15 eq. 16.

Hyperparameters

No training hyperparameters exist: the survey trains nothing. The table lists the numeric values the survey states for the cited systems, and the parameters the paper refers to but does not give.

Name	Value	Where	Notes
CTRL model size	1.63 billion parameters	§3.3	trained on a 140 GB corpus §3.3
PPLM attribute model size	"100,000 times less" than the PLM	§3.4	attribute model is a bag of words or a single layer §3.4
Mix and Match sequence length	20	§3.4	timing example only
RL penalty coefficient β	not stated	§3.2 eq. 5	"regular coefficient"; no value
CTRL penalty parameter θ	not stated	eq. 6	I(c) = θ if the condition holds, else 1
GeDi class priors P(c), P(c')	not stated	§3.4 eq. 8	"learnt or set manually"
GDC coefficients λ_i	estimated by Self-Normalized Importance Sampling	§3.4 eq. 13	no step size or sample count
Literature window	roughly the last 3-4 years	§1	selection rule, not a hyperparameter

Results

The survey reports no results of its own. Numbers below are quoted from the works it cites, or are its authors' own timing checks. Author-reported, setups not described here.

Setting	Metric	Baseline	Reported value	Source
MEGATRON-CNTRL, controllable story generation	Human judgement: share of stories controlled by the keywords	not stated	up to 91.5%	§3.4
Inverse Prompt, seven-word rhyming poem	Generation time, authors' test with the provided code	not stated	up to about 10 minutes	§3.2
Mix and Match, length-20 sequence	Generation time	not stated	"almost 11 seconds"	§3.4
CTRL	Model scale	not applicable	1.63 B parameters, 140 GB corpus	§3.3
FLAN	Instruction-tuning mixture	not applicable	more than 60 NLP datasets	§3.2

Paper vs code

No official code. The paper is a survey and links no repository. A GitHub repository search on the title and "controllable text generation survey" returned no repository identified as the authors'. The closest match, IAAR-Shanghai/CTGSurvey, is a different survey by an unrelated owner. The paper does mention running "the provided source code" for Inverse Prompt §3.2. That is a third-party method's code, not this paper's.

Pitfalls

• No experiments, no common benchmark. Every number is quoted from a cited work or from one timing check. Mix and Match's 11 seconds §3.4 and Inverse Prompt's 10 minutes §3.2 give no hardware, batch size or decoding settings.
• "Systematic" is a selection rule, not a search protocol. The abstract calls the review systematic. The only stated method is a recency window, a venue preference and citation or community attention §1. No databases, queries or screening steps are given.
• Some methods sit in two categories. POINTER appears under both "Retrain" and "Refact" in tab. 3, and the RL and instruction-tuning entries overlap. Do not treat the categories as disjoint.
• Eq. 6 (CTRL top-k penalty) is malformed as typeset. The \left( and \right. pair leaves one parenthesis unclosed in both the numerator and the denominator. θ has no value and I(c) is defined by a condition that the text does not name eq. 6.
• Eq. 14 is typeset with a missing θ. The log term reads π_(x), while the policy is written π_θ elsewhere eq. 14.
• Eq. 17 (PPL) conditions on earlier words, but the prose defines p(w_i) without context. Implement the conditional form, which is what the equation writes eq. 17.
• Eq. 8 and the GeDi step are the costly part. The paper says eq. 8 needs "two parallel forward passes" of the CC-LM §3.4. Scoring every vocabulary candidate per step, as the pseudocode does, is our reading of eq. 8, not a stated batching scheme. The priors P(c) and P(c') are not given values.
• Decoding-time trade-off. The survey notes that post-processing keeps PLM weights fixed but can be slow at inference and can lower fluency. PPLM loses some fluency §3.4; Mix and Match and COLD need many iterations §3.4 §5.1.
• Equation numbers are inferred. The LaTeX source has no printed equation numbers; we counted them in source order, which matches the paper's own labels (equ15 and equ16 for BLEU and ROUGE). If in doubt, cite the section rather than the equation.
• Newer than its title. The source we compiled from cites LLaMA tab. 2 and GPT-4 §2.3, both from 2023, while the arXiv first version is dated 2022. The fetched source is the one used here; check the arXiv version history for the exact text.
• Recency. The survey is a snapshot of 2022-2023 work. For any method, check its later versions before implementing.

Lineage

• Builds on: none stated. The paper cites earlier CTG surveys §1 but names no arXiv id.
• Compared against: an earlier CTG survey, which the authors say lacks a systematic review of tasks, approaches and evaluation, and a tracking of large-PLM approaches §1.
• Superseded by: not known from wiki content.

Related

ConceptControllable text generationControllable text generation makes a model's output meet a condition, such as a parse tree, a keyword or a speaker's gender. Seven method cards cover seven routes, from diffusion latent steering and frozen-model energy decoding to a moral-direction filter and control tokens, and a survey card maps the field. Results are author-reported.ConceptDecoding strategiesHow a model picks each next step from its scores: greedy, temperature, top-k and top-p sampling, CoT-decoding, majority voting, beam search, contrastive search, contrastive decoding, MBR over robot action chunks, toxicity filtering and tree-based speculative verification. Twenty paper cards; numbers are author-reported.ConceptSoft prompt tuningReplace hand-written prompt tokens with trainable continuous vectors, usually while the model stays frozen, and fit them by gradient descent. Variants differ in where the vectors go, what they are fitted on, and whether several prompts are mixed. Reported gains are author-reported, on different tasks, and not comparable.ConceptReinforcement learning from human feedback (RLHF)Post-training that fits a reward model to human preference judgments, then optimises the model with RL after supervised fine-tuning. DPO and KTO drop the reward model and RL loop; LIMA drops RLHF; Constitutional AI swaps human harm labels for AI feedback. Extensions cover data, images and video; sycophancy and red-team studies test side effects. Results are author-reported.Paper · 2026-07-07Ad Headline Generation using Self-Critical Masked Language ModelGenerates one short advertising headline for a set of products with a BERT-Large masked language model, then fine-tunes it with self-critical policy gradient, using ROUGE-L F1 against approved headlines as reward. Author-reported: ROUGE-L is 6.33 above a bi-LSTM baseline, and 6.53% more headlines get the top audit rating than human-submitted ones.Paper · 2026-01-12Stochastic CHAOS: Why Deterministic Inference Kills, and Distributional Variability Is the Heartbeat of Artifical CognitionArgues that greedy, deterministic LLM inference hides capability, brittleness and tail risk. Compares greedy decoding with T=0.7, top-p 0.9 sampling and majority vote on GLUE paraphrases, few-shot ICL, InstruSum, GSM8K/SVAMP/StrategyQA chain-of-thought and jailbreak safety. Sampling raises author-reported accuracy and robustness; several safety and reasoning tables are simulations.

Source and license

Adapted from “A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models” by Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, Dawei Song (arXiv:2201.05337), used under CC0 1.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.