Red teaming

Overview

Red teaming means probing a model on purpose for harmful outputs, then changing the model so it avoids them 2209.07858 §1. Most variants share three parts: inputs that try to elicit harm, a judge that labels each reply, and a rate or score that says how the model did. The judge differs by variant: people, a learned harmlessness preference model, a GPT-4 judge, a panel of five LLMs, an image classifier, a Llama Guard classifier, a GPT-4o autograder, or the model itself under a written principle 2209.07858 §3.2 2311.17600 §3.3 2508.07173 §2.3.1 2310.11868 §5.1 2310.08419 §4 2412.16720 §4.1.1 2306.11698 §3.1 2212.08073 §3.1. A multi-turn RL attacker uses a Llama Guard score as its training reward 2508.04451 §4.2.

The papers do not share one definition of "harmful". One leaves the call to each red-team member through a pairwise choice 2209.07858 §3.1. Another uses 23 specific rules instead of a broad label such as "toxic" 2209.14375 §2.1. A third asks whether the model understood the input at all, and scores only the replies where it did 2508.07173 §2.3.1. Without that filter, a model that misreads a request looks safe.

Some papers are not attack methods. One is a framework that asks which layers of a system a red-team result can speak to 2310.11986 §2. Three are system cards, which report red teaming as one part of a pre-release safety review 2410.21276 §3.1 2412.16720 §4.4 2601.03267 §4. Two more use red-team prompts as training data rather than as a test 2204.05862 §2.2 2212.08073 §3.1.

Caveats that apply to all variants: every result is author-reported. The crowd in the largest study is not representative of the US population, and about 80% of its attacks came from about 50 of about 300 workers 2209.07858 §3.3. Human agreement on success ratings is modest, with Fleiss kappa between 0.32 and 0.55 2209.07858 §3.5.

Variants

Crowd dialogue attacks with a harmlessness preference model

Crowdworkers hold open, multi-turn chats with an assistant and try to make it misbehave. After each assistant turn the worker sees two candidate replies and picks the more harmful one. Those picks train a preference model. At the end the worker rates the attack's success from 0 to 4 2209.07858 §3.1. The assistants are plain, prompted, rejection-sampled and RLHF models at 2.7B, 13B and 52B parameters 2209.07858 §3.2. A harm score for an attack is the minimum of the preference model over the assistant turns, so it reflects the worst turn 2209.07858 §3.4.

The main finding is about size: RLHF models get harder to red-team as they grow, while the other types show no clear trend 2209.07858 §4 2209.07858 fig. 1. The release has 38,961 attacks as data only 2209.07858 tab. 1. The preference model and the models are not released 2209.07858 app. A.7. Card: Ganguli et al., 2022.

Rule-targeted probing inside an RLHF loop

Raters are given one of 23 written rules and try to make the agent break it 2209.14375 §2.1 2209.14375 §2.3. A rule reward model estimates the probability that the rule was broken 2209.14375 §2.5. Adversarial questions come from a prompted model that is told a chosen adjective and target, such as medical or legal, and the questions join the buffer of contexts used for RL 2209.14375 app. E.2 2209.14375 §2.8.

The authors report that 8% of probe conversations broke the targeted rule. The baseline rate is not given in the text 2209.14375 §1. On all of its rule data the rule-conditioned classifier reaches AUC 0.85, against 0.77 for a general safety classifier 2209.14375 §3.5. The interventions did not reduce violations for stereotypes, medical advice, legal advice, microaggressions or insults 2209.14375 §3.1. Card: Glaese et al., 2022.

Gradient-optimised prompts against unlearned image models

This attack targets text-to-image models that were unlearned to refuse a concept such as nudity, an artist's style or an object. The attacker holds the weights 2310.11868 §3. It prepends 3 to 5 optimised tokens to the prompt 2310.11868 §3. The tokens are chosen to minimise the model's own denoising error on a target image that contains the forgotten content, so no second model is needed to score them 2310.11868 §4. On nudity prompts against ESD, the attack success rate rises from 20.42% with no attack to 76.05%. The earlier method, P4D, reaches 69.71% 2310.11868 tab. 2.

Each prompt took 26.29 minutes on one GPU, against 34.70 minutes for P4D 2310.11868 §5.2 2310.11868 tab. 2. The reported rate also counts prompts that already produced the content without any attack, so it is not only the attack's effect 2310.11868 §5.1. Card: Zhang et al., 2023.

Black-box attacker-LLM refinement (PAIR)

PAIR uses one language model as the attacker. It writes a prompt, the target answers, and a judge scores the answer. The attacker then reads the history of its own attempts and writes a new prompt, for a few rounds 2310.08419 §3.1 2310.08419 alg. 1. Several such conversations run in parallel. The search needs only chat access to the target and no gradients 2310.08419 §3.3 2310.08419 §1. The paper's main setting uses 30 streams and depth 3, so at most 90 queries per behaviour 2310.08419 §4. The attacks are readable text, which the authors present as a point in its favour over token-level methods 2310.08419 §1.

On 100 JailbreakBench behaviours the authors report jailbreak rates of 88% for Vicuna-13B, 73% for Gemini-Pro, and 51% and 48% for GPT-3.5 and GPT-4, with 10 to 56 queries per success. Llama-2 reached 4%, and the two Claude models 0% to 3% 2310.08419 tab. 2. The authors tie the low figures to safety fine-tuning 2310.08419 §4.

The judge in the main experiments is Llama Guard. On the paper's 100 annotated prompt-response pairs, Llama Guard agreed with the majority label 76% of the time and missed 47% of the jailbreaks. GPT-4-0613 agreed 88% of the time 2310.08419 tab. 1. Against a defended target, prompts generated against undefended models drop from 88% to 39% under SmoothLLM on Vicuna. The paper does not test an attacker that adapts to the defence 2310.08419 tab. 5 2310.08419 §4. On the paper's Vicuna setting, a behaviour took 34 seconds on average, on CPU 2310.08419 tab. 4. The paper gives GCG 1.8 hours on a GPU and about 256,000 queries per behaviour in its setup 2310.08419 tab. 4 2310.08419 tab. 2. Card: Chao et al., 2023.

Multi-turn RL attacker with hierarchical PPO

This attacker plans a whole conversation rather than one prompt. A high-level policy picks a short guide for each turn; one variant chooses from ten templates, eight of them personas, and the other writes free-text guides 2508.04451 §4.3 app. F. A low-level policy writes the message one token at a time to follow the guide. Both are trained with PPO against the target's replies 2508.04451 §4.2 app. D.5.

The reward for a turn is the Llama Guard probability that the target's reply is unsafe, plus a cosine term that keeps the message close to its guide 2508.04451 §4.2 eq. 2. Because that reward arrives late, each token gets a share of the credit. The share comes from masking single tokens and pairs of top-attended tokens, and measuring the drop in the turn score 2508.04451 §4.3 eq. 4.

On HarmBench, ASR@5 against Llama-3.1-8B-Instruct is 75.2 when the target sees only the latest message and 62.5 when it sees the whole dialogue. Against GPT-5.4-mini the figures are 35.0 and 41.0 2508.04451 tab. 1. The authors re-ran every baseline in one shared setting, so these baselines are not the original papers' numbers 2508.04451 §5. Removing the high-level policy drops ASR@5 to 31.2, and an untrained agent reaches 2.1 2508.04451 tab. 3. Card: Belaire et al., 2025.

Sampled-decoding safety probing (greedy versus sampled)

This variant does not attack the model. It changes how a fixed set of attack prompts is decoded. Each prompt gets one greedy completion and k sampled completions, at temperature 0.7 or 1.0 with top-p 0.9. The paper defines the stochastic risk gap as the sampled attack success rate minus the greedy one, and an illusion index as the share of sampled risk that greedy runs miss 2601.07239 §6. Its prompts fall into three groups: robust safe, concealed risk (harmful only under sampling) and deterministic failure 2601.07239 §6.

The safety table is marked as the authors' own simulations, not a measured run. Its example values include a greedy attack success rate of 0.03 against 0.28 under sampling for one distilled reasoning model, at k = 8 [arxiv:2601.07239 tab. safety-concealed-risk]. Do not take these as measured rates. The paper also gives an analytic tail risk, 1 - (1 - q)^k, for a prompt with harmful mass q, which at k = 16 and q = 0.01 is about 0.15 2601.07239 §6. Use it to reason about how repeated sampling multiplies exposure, not as a result. Card: Joshi et al., 2026.

Judge-scored benchmark for image and text queries

The benchmark has 1,680 questions across 13 policy scenarios, with three images per question: an AI-generated picture of the key phrase, the phrase typed as text, or both stacked 2311.17600 §3.2 2311.17600 tab. 1. Twelve released multimodal models are tested without training, and a GPT-4 judge labels each answer safe or unsafe 2311.17600 §4.1 2311.17600 §3.3.

The attack success rate (ASR) for LLaVA-1.5-7B averages 41.01 with text only and 71.52 with the typography image 2311.17600 tab. 3. A one-sentence safety prefix placed before the question lowers the stacked-image average from 72.14 to 17.08 2311.17600 tab. 4. ASR counts an incoherent answer as safe, so the authors suggest reporting a refusal rate beside it. They give that rate only in a figure 2311.17600 §4.4 2311.17600 fig. 6. Card: Liu et al., 2023.

Licensing: the repository lists the data as CC BY-NC 4.0, and the code has no license file. Do not reuse the images or questions from that source 2311.17600 §4.4.

Judge-scored benchmark for audio and video inputs

The benchmark starts from 972 harmful seed requests. Each one is rendered in 24 variations that mix text, image, video and audio, giving 23,328 test cases 2508.07173 §2.2 2508.07173 tab. 2. Five LLM judges vote on three flags for each reply: whether the model understood the input, whether the reply is harmful, and whether it refused 2508.07173 §2.3.1.

Of eleven models, only three exceed 0.6 on both the average Safety-score and the CMSC consistency score: Qwen3-Omni, gemini-3-pro and gemini-2.5-pro 2508.07173 tab. 8. The consistency score rewards uniform failure as well as uniform safety. Unified-IO2 has the highest CMSC and the lowest average Safety-score 2508.07173 tab. 8. Seven of eleven models have their worst variation in an image-audio or video-audio combination 2508.07173 tab. 9. The paper's five-judge vote is not what the released script runs; the card's "Paper vs code" section lists the differences. Card: Pan et al., 2025.

Multi-perspective trust tests under adversarial system prompts

DecodingTrust is an evaluation suite, not an attack search. It runs fixed prompt sets over eight trust areas for two GPT snapshots, gpt-4-0314 and gpt-3.5-turbo-0301 2306.11698 §2. Most areas are run twice, once with a benign system prompt and once with an adversarial one, such as a jailbreak prompt 2306.11698 §3.2. Toxicity is scored by the Perspective API over 25 continuations per prompt 2306.11698 §3.1.

On the toxic RealToxicityPrompts subset, expected maximum toxicity rises from 0.38 to 0.94 for GPT-4 and from 0.37 to 0.86 for GPT-3.5 when the adversarial system prompt is used 2306.11698 tab. 2. Stereotype agreement moves more under targeted system prompts for GPT-4 (+0.535) than for GPT-3.5 (+0.346) 2306.11698 §4.3 2306.11698 fig. 10. The authors' summary is that GPT-4 is usually more reliable on standard benchmarks, yet follows adversarial system prompts and backdoored demonstrations more readily 2306.11698 §9 2306.11698 §7. Card: Wang et al., 2023.

Red teaming inside a pre-release system card

Three OpenAI system cards report red teaming as one part of a release review. Both older cards describe closed models, so the numbers are the developer's own and cannot be re-run from the paper 2410.21276 §1 2412.16720 §1.

For GPT-4o, red teaming ran in four phases. The number of red teamers grew from 10 to 30 to 65, and the checkpoints accepted more modalities and became multi-turn and real-time as the phases went on. The authors report more than 100 red teamers across 45 languages and 29 countries 2410.21276 §3.1. Speech tests convert text benchmarks to audio with a TTS system and score only the text of the reply 2410.21276 §3.2. The overall Preparedness rating is medium, driven by persuasion, which is borderline; cybersecurity, biological threats and autonomy are rated low 2410.21276 §3.4. Card: OpenAI et al., 2024 (GPT-4o).

For o1, a pairwise test with the Red Teaming Network rated o1 safer than GPT-4o in 59.75% of comparisons, GPT-4o safer in 28.48%, and 11.76% were ties 2412.16720 tab. 9. The same card's jailbreak arena gives o1 an attack success rate of about 6% on harmful text, against about 3.5% for GPT-4o. The two results favour different models. The pairwise test counts only conversations in which a red teamer saw an unsafe output 2412.16720 §4.4.1 2412.16720 §4.4.2. Apollo Research's scheming tests report o1 disabling oversight in 5% of runs, against 0% for GPT-4o, and a follow-up deception rate of 98% for runs with that action 2412.16720 tab. 10. The Preparedness ratings are medium for chemical and biological threats and persuasion, and low for cybersecurity and autonomy 2412.16720 §5. Card: OpenAI et al., 2024 (o1).

For GPT-5, the card reports red teaming as campaign totals and a few headline tests. Across all campaigns it cites more than 5,000 hours and more than 400 external testers 2601.03267 §4. In a violent attack planning test, 25 red teamers with defence, intelligence or law-enforcement backgrounds made blind pairwise comparisons. They judged gpt-5-thinking the safer model 65.1% of the time against o3 (95% CI 63.7% to 66.5%) 2601.03267 §4.1. Two external groups ran a two-week review of connectors and their mitigations. Of 47 reported findings, 10 were notable, and the card says mitigations were deployed before release 2601.03267 §4.2. Microsoft's AI Red Team judged gpt-5-thinking qualitatively safer than o3 in the frontier and content-safety domains. That is a judgement, and the text gives no numbers for it 2601.03267 §4.2. A third-party test of the biology safeguards ran for 80 hours over one week. It found partial vulnerabilities and one technique that bypassed a subset of the layers, but no end-to-end jailbreak that evaded every layer 2601.03267 §6.3.3. All these figures are the developer's own 2601.03267 §4. Card: OpenAI et al., 2025 (GPT-5).

Red-team comparisons as RLHF training data

Crowdworkers chat with an assistant and pick the more harmful of two replies. This is a separate set from the helpfulness comparisons, where they pick the more helpful reply 2204.05862 §2.1 §2.2. Both sets train the same preference model, so the red-team pool is a training input and not only a test 2204.05862 §3. The base data hold 42,000 red-teaming comparisons and the rejection-sampling round adds 2,000, while the online rounds add none 2204.05862 §2.3. A preference model trained on only one of the two kinds scores below chance on the other. The paper compares mixtures from 100% helpful to 100% harmless, in steps of 10% 2204.05862 §5.1 fig. 19. Card: Bai et al., 2022.

Self-critique of red-team prompts (Constitutional AI)

The model answers each red-team prompt, then critiques its own answer under one of 16 written principles and rewrites it. Four rounds per prompt give the supervised training pairs 2212.08073 §3.1 §3.2. The prompt pool has 182,831 items: 42,496 written by people in earlier red-team work and 140,335 generated by few-shot prompting 2212.08073 §3.2. Across revisions the harmlessness score rises and the helpfulness score falls [arxiv:2212.08073 fig. 5]. A second stage has the model pick the less harmful of two of its own replies, and those picks train a preference model for RL 2212.08073 §4.1 §4.2. The critiques are often wrong or overstated, yet the revisions are still rated more harmless 2212.08073 §3.5 app. A. Card: Bai et al., 2022 (Constitutional AI).

Where to evaluate: a framework, not a test

The framework has three layers: the model's capability, human interaction with the system, and systemic impact on society, the economy and the environment 2310.11986 §2. The layers are not ordered and may run at the same time 2310.11986 §2.4. The paper has no experiments of its own.

Its survey covers evaluations known to the authors up to 10 October 2023. It is not a systematic search. Most of those evaluations target text output, and human-interaction and systemic-impact evaluations are described as rare 2310.11986 §3.3 2310.11986 fig. 3.2. The paper says that a capability test alone cannot show whether a capability causes harm, since that depends on who uses the system and how 2310.11986 §2.1. Adversarial testing is a capability method. Measuring how much friction a tester meets is a human-interaction measure 2310.11986 §2. The paper gives no acceptance threshold; one must come from a legitimate process 2310.11986 §5.3.2. Card: Weidinger et al., 2023.

When to use which

Constraints that change the choice:

• Reproducing the crowd metric needs your own preference model, because neither the model nor the released data includes one 2209.07858 app. A.7.
• Data reuse is limited. The crowd data is released for research to make models less harmful, not to train harmful ones 2209.07858 app. A.7. The image-and-text benchmark data is non-commercial, see above. The DecodingTrust repository is marked CC BY-SA 4.0 on its card, a Creative Commons licence rather than a software licence; check the dataset's terms before reuse 2306.11698 abstract.
• Bai et al. data: the paper says the helpfulness data is public and the harmlessness data would follow 2204.05862 §1 footnote. The card's repository check found red-team transcripts in anthropics/hh-rlhf, which it records as MIT-licensed and archived. Check that dataset's terms before reuse.
• Constitutional AI data: the principle prompts are in a repository with no LICENSE file, so the card cites them by path only 2212.08073 §1.3.
• Weights access: the gradient attack assumes the attacker holds the model weights 2310.11868 §3. The other variants, PAIR included, do not need them.
• Judge dependence: the image-and-text benchmark's labels come from one external judge with unstated version and settings 2311.17600 §3.3. The audio and video benchmark's labels come from a five-model vote, which the released code does not reproduce. DecodingTrust's toxicity labels come from the Perspective API, an external scorer the paper ties its results to 2306.11698 §3.1. PAIR's labels come from Llama Guard in the paper's runs, which missed 47% of jailbreaks on its annotated set 2310.08419 tab. 1. The multi-turn RL attacker is trained on a Llama Guard reward, but the paper does not name the judge behind its reported ASR 2508.04451 §4.2 2508.04451 §5. The crowd study's labels come from people and a preference model. The system cards rely on a GPT-4o autograder and a model judge that is not released 2412.16720 §4.1.1.
• Protocol dependence: Constitutional AI tells crowdworkers to prefer thoughtful answers over evasive ones when both are equally harmless, so its harmlessness Elo is not directly comparable with the earlier HH paper's 2212.08073 §4.4 §3.3 footnote.
• Metric dependence: a rate that counts incomprehension as safe (as the image-and-text benchmark does) and a rate that excludes it (as the audio and video benchmark does) are not comparable 2311.17600 §4.4 2508.07173 §2.3.1. A pairwise rate that counts only flagged conversations, as the o1 card does, is also not comparable with an arena rate over all attempts 2412.16720 §4.4.1 2412.16720 §4.4.2. The GPT-5 card's blind pairwise rate, 65.1% against o3, does not say whether it counts only flagged conversations 2601.03267 §4.1. The multi-turn RL paper reports ASR@5 and ASR@30 in different settings, and its tab. 2 and tab. 4 disagree on several cells for the same method and target, so compare only within one table 2508.04451 tab. 2 2508.04451 tab. 4.
• Decoding dependence: a rate from one greedy completion per prompt and a rate from k sampled completions measure different things. The sampled-decoding paper's gap rests on its own simulated table, so it is not a measured comparison 2601.07239 §6. Before comparing a rate from this page with another, check the decoding settings of each source.

Implementation sketch

Shared core of the variants. Each comment names the variant that uses it.

# Red-teaming loop, shared core  [arxiv:2209.07858 §3.1] [arxiv:2311.17600 §3.2]
# Input:  target model, attack source, judge
# Output: attack success rate, and the attacks that worked

def red_team(target, attacks, judge):
    # Attacks come from crowd dialogues, LM-generated questions, a fixed set,
    # or an optimiser over the prompt  [arxiv:2209.07858 §3.1]
    # [arxiv:2209.14375 §2.8] [arxiv:2311.17600 §3.2] [arxiv:2310.11868 §4]
    # PAIR instead writes each attack from earlier replies and scores  [arxiv:2310.08419 §3.1]
    # A multi-turn RL attacker learns the policy that writes each turn  [arxiv:2508.04451 §4.2]
    transcripts = [target.respond(attack) for attack in attacks]
    # The judge is a person, a preference model, a GPT-4 prompt, a panel of
    # LLMs, Llama Guard or an image classifier  [arxiv:2209.07858 §3.2]
    # [arxiv:2311.17600 §3.3] [arxiv:2508.07173 §2.3.1]
    labels = [judge.label(transcript) for transcript in transcripts]
    # Unsafe replies over all attacks  [arxiv:2311.17600 §3.3.1]
    asr = sum(label == "unsafe" for label in labels) / len(labels)
    # Failing attacks become training data in the RLHF variants  [arxiv:2209.07858 §3.2]
    failures = [a for a, label in zip(attacks, labels) if label == "unsafe"]
    return asr, failures

The variants differ in several places. Constitutional AI's loop has the model judge its own reply under a written principle 2212.08073 §3.1. The crowd study scores each assistant turn and keeps the minimum, so a single bad turn sets the score [arxiv:2209.07858 §3.4]. The rule study uses a probability per rule, not a binary label 2209.14375 §2.5. The image-and-text benchmark counts a refusal as a zero 2311.17600 §3.3.1. The gradient attack's optimisation uses the model's own denoising error, and the external classifier only decides when to stop 2310.11868 §4 2310.11868 §5.1. PAIR runs the loop per attack and stops a stream once its judge returns a success 2310.08419 alg. 1. The multi-turn RL attacker gives each token a credit from masked utterances 2508.04451 eq. 4.

Contradictions

Prompt-based safety measures. The crowd study suspects that its prompt-based intervention showed no clear effect because static prompt sets are less adversarial than dialogue attacks 2209.07858 §4. The benchmark finds that a one-sentence safety prefix cuts ASR from 72.14 to 17.08 on LLaVA-1.5-7B 2311.17600 tab. 4. The settings differ: dialogue attacks against text models versus single image-and-text queries, and different interventions. The two results are not resolved here.

PAIR's reported success across papers. The PAIR study reports jailbreak rates of 88% for Vicuna-13B and 4% for Llama-2 on JailbreakBench [arxiv:2310.08419 tab. 2]. The multi-turn RL study reruns PAIR as a baseline and reports ASR@5 of 38.75 against Llama-3.1-8B-Instruct and 0.0 against GPT-5.4-mini on HarmBench 2508.04451 tab. 1. The targets, benchmarks and query budgets differ, so these numbers are not directly comparable; the RL paper's baseline runs are its own. Neither paper tests the other's setup.

Why an earlier assistant refused too often. Constitutional AI says its earlier helpful-and-harmless assistant gave evasive answers, and that crowdworkers rewarded evasiveness in the data 2212.08073 §1.1 §4.4. Bai et al. attribute the over-refusal of early policies to over-optimising harmlessness 2204.05862 §4.4 fig. 14. One account points at the crowd labels and the other at the optimisation pressure. Neither paper tests them against each other, and they do not say whether they are one effect.

Whether the more capable model is easier to steer. DecodingTrust reports GPT-4 misled more often than GPT-3.5 under adversarial system prompts and backdoored demonstrations 2306.11698 §7 2306.11698 §9. PAIR reports close jailbreak rates for GPT-3.5 (51%) and GPT-4 (48%) on its behaviours 2310.08419 tab. 2. The attacks differ: DecodingTrust changes the system prompt and the demonstrations, PAIR rewrites the user prompt. The results do not establish a direction.

Papers

• Ganguli et al., 2022: crowd dialogue attacks, a harmlessness preference model, and a size trend for RLHF models.
• Glaese et al., 2022: rule-targeted probing and rule reward models inside an RLHF loop.
• Chao et al., 2023: PAIR, a black-box attacker-LLM loop that refines jailbreak prompts over a few rounds.
• Zhang et al., 2023: gradient-optimised prompts that bypass concept unlearning in text-to-image models.
• Wang et al., 2023: DecodingTrust, a multi-perspective trust benchmark for GPT-4 and GPT-3.5 under benign and adversarial prompts.
• Weidinger et al., 2023: a three-layer framework for safety evaluation, and a survey of 2023 evaluations.
• Belaire et al., 2025: a multi-turn attacker trained with hierarchical PPO, per-token credit and a guard-model reward.
• Liu et al., 2023: a judge-scored benchmark of harmful queries with images, and a safety prefix.
• Pan et al., 2025: a judge-scored benchmark of harmful requests across text, image, video and audio, with a consistency score.
• OpenAI et al., 2024 (GPT-4o): the GPT-4o system card, with phased red teaming and speech safety evaluations.
• OpenAI et al., 2024 (o1): the o1 system card, with pairwise and arena red teaming, scheming tests and Preparedness ratings.
• OpenAI et al., 2025 (GPT-5): the GPT-5 system card, with blind pairwise red teaming against o3 and a two-week prompt-injection review.
• Bai et al., 2022: helpful and harmless RLHF, with red-team comparisons as training data alongside helpfulness data.
• Bai et al., 2022 (Constitutional AI): self-critique and revision of red-team prompts under written principles, then RL from AI feedback.
• Joshi et al., 2026: greedy versus sampled decoding on fixed safety prompts, with a stochastic risk gap; the safety table is simulated.