Unleashing the potential of prompt engineering for large language models

In plain words

A language model answers whatever text you give it, and a small change in wording can change the answer a lot. Prompt engineering is the practice of writing and arranging that text on purpose: stating instructions clearly, asking the model to play a role, showing it a few examples, or asking it to think in steps before answering.

This review goes through those techniques, from the simple ones to more elaborate ones. It also covers the same idea for models that handle both images and text, where small learned prompts can be tuned instead of written by hand. Finally, it looks at how prompts can be abused to attack these systems, and how people try to defend against that. The review does not add a new method of its own. It is a map of the field.

TL;DR

A survey, not a method. It organises prompt engineering for LLMs into basic prompting, advanced reasoning methods, automatic prompt optimization and retrieval or tool use, and adds the vision-language prompt methods CoOp, CoCoOp and MaPLe §1. It also covers evaluation metrics and benchmarks §5, application areas §6, and security risks such as adversarial examples, backdoors, prompt injection and model stealing §7. Use it as an index into the literature; check any number against the original paper it cites.

Problem

Model output depends strongly on the prompt: its length, the arrangement of examples, its phrasing and its directives §1. Writing good prompts by hand takes effort and expertise, and the review frames prompt design as the lever for getting accuracy, relevance and coherence out of a fixed model §1 §3.10. For complex analysis or reasoning, the accuracy of plain prompting is seen as still improvable §3.

Method

The paper is a review. Its sections, as numbered in the compiled paper:

• Basics §2. Background on GPT-4 as the model behind the paper's own examples, and the decoding settings temperature and top-k §2.1. Basic methods: giving detailed instructions §2.2, being clear and precise §2.3, role prompting §2.4, delimiters such as triple quotes §2.5, resampling ("try several times") §2.6, one-shot and few-shot prompting §2.7, and temperature and top-p settings §2.8. The review notes that zero-shot prompts can outperform few-shot ones in some settings §2.7.
• Advanced methods §3.
  • Chain-of-thought (CoT): intermediate reasoning steps, by a cue such as "let's think step by step" or by worked examples §3.1. Zero-shot CoT uses only the cue §3.1.1. Golden CoT supplies ground-truth reasoning inside the prompt §3.1.2.
  • Self-consistency: sample several reasoning paths and keep the answer they agree on most §3.2 (see pseudocode below).
  • Generated knowledge: have the model first write relevant facts, then answer with them in context §3.3 fig. 8 fig. 9.
  • Least-to-most: split a problem into sub-problems solved in sequence §3.4 fig. 10.
  • Tree of thoughts and graph of thoughts: explore several thoughts with look-ahead, backtracking and self-evaluation §3.5; or connect thoughts as nodes of a graph §3.6.
  • DECOMP: a decomposer writes a program of sub-queries, each handled by a sub-task function, run by a controller §3.7 fig. 12 fig. 13.
  • Active prompt: pick the most uncertain training questions by disagreement, entropy or variance, have humans annotate them, and use them as examples §3.8 fig. 14.
  • Prompt pattern catalog: a set of reusable prompt patterns in five groups (input semantics, output customisation, error identification, prompt improvement, interaction) §3.9.
  • Prompt optimization: ProTeGi (textual gradients with beam search and a bandit selection) §3.10.1; black-box prompt optimization with a trained rewriter §3.10.2; model-adaptive optimization with a warm-up set, SFT and RL §3.10.3; PromptAgent (Monte Carlo tree search over prompts) §3.10.4; RL-based prompt search §3.10.5; and GPT-store prompt helpers §3.10.6 fig. 15.
  • Retrieval augmentation: put retrieved external text into the prompt to reduce hallucination §3.11.
  • Reasoning and acting: ART (CoT plus tool calls) §3.12.1; ReAct (interleaved reasoning traces and actions) §3.12.2 fig. 16.
• Vision-language models §4. Zero- and few-shot prompting in VLMs §4.1; continuous prompt vectors for video tasks on CLIP §4.2; CoOp, which learns context vectors for the text side §4.3; CoCoOp, which generates an input-conditioned prompt with a small network §4.4; and MaPLe, which learns prompts for both vision and language branches at several transformer layers §4.5 fig. 17.
• Evaluation §5. Subjective (human) and objective (automatic) evaluation §5.1, with BLEU, ROUGE, METEOR and BERTScore named as automatic metrics §5.1.2. Benchmark families: math word problems, QA, language understanding and multimodal referring-expression tasks §5.1.2. Cross-method comparisons and the InstructEval framework §5.2.
• Applications §6. Teaching and assessment §6.1 fig. 18, content creation §6.2, programming §6.3, reasoning tasks §6.4, and synthetic dataset generation §6.5.
• Security §7. Adversarial attacks §7.1 with data poisoning §7.1.1, backdoors §7.1.2 fig. 20 and prompt injection and leaking §7.1.3; prompt hacking §7.2; model stealing §7.3; and defences such as adversarial training §7.4 fig. 19.
• Outlook §8. Understanding model structure §8.1 and AI agents for AI-generated content tools §8.2.
• Conclusion §9.

Display equations in the source are unnumbered. The DECOMP definitions are given as a program P of (function, sub-query, answer) triples and as in-context examples E_j built from them §3.7. Cite them as §3.7.

Self-consistency, as the review describes it §3.2:

# Self-consistency  [§3.2]
# Input:  question; number of sampled paths n
# Output: the final answer with the most support

def self_consistency(question, n):
    # 1. Prompt with CoT, but sample instead of greedy decoding  [§3.2]
    paths = []
    for i in range(n):
        paths.append(sample_reasoning(question))  # (reasoning, answer)
    # 2. Marginalise over the reasoning paths; keep the most frequent answer  [§3.2]
    answers = [answer for (reasoning, answer) in paths]
    return most_common(answers)

Hyperparameters

The review has no experiments and no hyperparameters of its own. The settings it mentions are listed below. None of them has a value stated in the text.

Results

No new results. The review quotes figures from the works it cites. These are author-reported, were not re-run here, and the setup is the one in the cited work.

Other claims are qualitative only, for example that zero-shot prompts can beat few-shot ones in some scenarios §2.7, that CoT improves logical reasoning §3.1, and that VLM methods outperform baselines on image recognition and VQA §4.3 §4.5. The review states that no single benchmark or model is best for every task §5.1.2, and that InstructEval finds generic prompts often match expert or automated prompts in few-shot settings §5.2.

Paper vs code

No official code. The paper text contains no code or repository link. The abstract and arXiv metadata carry no code link either. A GitHub repository search on the title words returned no results. Searches tried: a GitHub repository search on "prompt engineering for large language models review" via the API (no results). A general web search was not permitted in this session.

Pitfalls

• No experiments of its own. Every number comes from another paper, and the review does not restate the setup for most of them. Check the cited paper before relying on a figure.
• Citation mismatches. In §4.1 the zero-shot CLIP claim is cited to the GPT-3 reference, and the few-shot claim to the CLIP reference. Check the CLIP paper for the zero-shot claim. In §3.5 the single-prompt tree-of-thought variant (sample prompt fig. 11) is cited to a Bard reference, while the tree-of-thoughts method is cited to Yao et al. Use the method's own paper.
• Security incidents are vague. §7.3 describes extraction of a projection matrix from a production model and a customer-service model replication with financial losses, without enough detail to check. The card does not verify them; treat them as unverified.
• Model names do not match. The abstract mentions GPT-4o and Claude-3; the body uses Claude 3.5 Sonnet §1 and GPT-4 for all its examples §2.1. The examples have no stated temperature, sampling settings or model date, so they cannot be reproduced exactly §2.1.
• Some claims lack a specific source. For example, the hallucination claim for GPT-3 in §3.11 and the claim about least-to-most performance on GSM8K and SVAMP through PAL §3.4 are given without the numbers behind them.
• Heuristic prompt rules. Most basic-method advice (clarity, role, retrying, delimiters) is shown with single examples §2.2–§2.6. Those examples do not test how much each rule helps.
• Self-consistency cost. It needs several sampled outputs per question, and the review notes that controlling the sampling settings may need API access §3.2. The sample count and aggregation rule beyond "most consistent" are not stated §3.2.
• Security section is mostly generic. The defences in §7.4 and the prompt hacking guidance in §7.2 are described in general terms, with no evaluation.
• Version. The source fetched is the arXiv v6 record (updated 2025-05-11). The date line in the LaTeX source is a compile date, not a version marker.
• Figures. There are 20 figures in the compiled paper. They are described in words here and cited by number, not reproduced.

Lineage

• Builds on: the works it reviews in each section; no single method is the base.
• Compared against: the review compares methods within each section, not in a single table. Cross-method comparisons are summarised in §5.2.
• Superseded by: not known from content in this wiki.