---
kind: concept
slug: zero-shot-learning
title: Zero-shot learning
summary: >-
  Answering for classes or tasks with no labelled examples. Routes: attribute or word-vector scores, generated pseudo-features, prompted LLMs and option scoring, CLIP-space 3D and adapters, text-to-region matching, event extraction, sketch, composed and video retrieval, robot policies, navigation agents, and attacks. Main trade-off: what supervision and compute you have.
tags: [zero-shot, prompting, attributes, joint-embedding, open-vocabulary, segmentation, clip, multi-agent, text-to-3d, sketch-retrieval, composed-retrieval, point-cloud, evaluation, multitask, scaling-laws, finance, navigation, robotics, imitation-learning, remote-sensing, video, targeted-attack, adversarial, embodied-navigation, vision-language-model, neuro-symbolic, multilingual, event-extraction, travel-behavior, transportation, generative-model, pseudo-features, multiple-choice, scoring-functions, perplexity, prompt-selection, web-data, domain-generalization, text-only-training, adapter]
papers: ["1709.00663", "1812.00420", "1904.03451", "2104.05919", "2104.08315", "2110.02624", "2110.08207", "2110.14378", "2111.03930", "2112.12143", "2202.02005", "2204.08790", "2207.04429", "2208.02816", "2212.04037", "2212.05171", "2212.07143", "2212.09196", "2302.03084", "2302.06476", "2304.05613", "2305.08275", "2306.11029", "2311.10537", "2311.16445", "2311.16863", "2312.00819", "2401.11641", "2411.05755", "2412.10402", "2509.19994"]
related: [few-shot-learning, in-context-learning, chain-of-thought-prompting, prompt-engineering, prompt-optimization, language-model-surprisal, likelihood-based-text-evaluation, calibration, self-consistency, multi-agent-systems, llm-evaluation, multilingual-evaluation, emergent-abilities, llm-serving, continual-learning, open-vocabulary-segmentation, vision-language-pretraining, image-to-3d-generation, parameter-efficient-fine-tuning, metric-learning, linear-probing, transfer-learning, domain-adaptation, domain-generalization, instruction-tuning, multi-task-learning, neural-scaling-laws, contrastive-learning, sentiment-analysis, topological-map-navigation, visual-grounding, foundation-model, adversarial-robustness, adversarial-evaluation, anomaly-detection, imitation-learning, learning-from-demonstration, visual-prompting, multimodal-language-model, llm-agent, tool-use, discrete-choice-modeling, text-embedding, variational-autoencoder]
status: draft
updated_at: 2026-11-01
---

## Overview

Zero-shot means the model answers for a class or task it has no labelled
examples of. The thirty-one cards here share that goal and little else. One route
describes every class by a vector of attributes and scores an image against
each class description [arxiv:1812.00420 §5]. Another trains a conditional
generator on seen classes, then samples feature vectors for unseen classes and
fits a classifier on them [arxiv:1709.00663 §3] [arxiv:1709.00663 alg. 1]. Another maps each class name to
a word vector, so an unseen category can be matched from a sketch
[arxiv:1904.03451 §4.2] [arxiv:1904.03451 §4.3]. Another gives a pretrained
language model a plain-language instruction and the input, with no
demonstrations [arxiv:2302.06476 §3]. A third labels pixels with class names
given at test time, using region features aligned to words
[arxiv:2112.12143 §3.2]. Several rely on CLIP's shared image-text space: one
generates 3D shapes from text without any text-shape pairs [arxiv:2110.02624
§3], one adds a training-free cache of few-shot examples on top of zero-shot
CLIP [arxiv:2111.03930 §3.2], one starts a trainable head from zero-shot
class vectors [arxiv:2204.08790 §4.2], one turns a reference image into a
pseudo word for composed queries [arxiv:2302.03084 §3.2], and one aligns 3D
point clouds with CLIP-space images and captions [arxiv:2305.08275 §3.3].
Another keeps a SLIP image-text model frozen and aligns 3D point clouds to it
using category names rather than generated captions [arxiv:2212.05171 §3.2].
The last runs several prompted LLM "experts" over a medical question with no
examples [arxiv:2311.10537 §2]. Five more move the idea elsewhere. One
fine-tunes a T5 model on many prompted datasets and then tests it on task
types held out of training [arxiv:2110.08207 §3]. Another fits power laws to
the zero-shot ImageNet accuracy of CLIP-style models trained on open image-text
data [arxiv:2212.07143 §4.1]. A third keeps training CLIP on remote-sensing
image-text pairs and reports zero-shot classification on 12 datasets
[arxiv:2306.11029 §4.2.3]. A fourth chains a frozen LLM and CLIP to route a
ground robot from a text instruction, with no robot-language training
[arxiv:2207.04429 §3]. A fifth trains one robot policy on 100 manipulation
tasks, conditioned on an embedding of a language command, and tests it on
tasks held out of training [arxiv:2202.02005 §3] [arxiv:2202.02005 §6.2]. The
last prompts GPT-4 on six finance datasets with no examples or one example
[arxiv:2401.11641 §4].

Two more cards use the idea in new settings. One adapts a CLIP-style
image-text model to video, so a clip can be matched to action names it was
not trained on [arxiv:2208.02816 §1]. The other is a security study: it
attacks multimodal embedding models so that an image matches a target concept
named only in part, and it tests whether the attack holds on unseen wordings
[arxiv:2509.19994 §1] [arxiv:2509.19994 §5.1].

Two more cards let a pretrained model act as a zero-shot embodied agent. One
lets a vision-language model choose a numbered arrow at each step, and the
arrows are drawn on the camera image from depth-based free directions
[arxiv:2411.05755 §3.1] [arxiv:2411.05755 §3.4]. The other has an LLM write a
short program that calls pretrained navigation, detection and question-answering
modules [arxiv:2412.10402 §3].

Three more cards study the zero-shot setting itself rather than a method. One
asks whether GPT-3, given no examples, solves analogy puzzles built for
students [arxiv:2212.09196 §1]. Another measures the energy of 1,000
inferences for zero-shot generative models and for task-specific models
[arxiv:2311.16863 §3.2]. The third tests ChatGPT with one prompt per example,
and no shown examples, on seven NLP tasks in 34 to 37 languages
[arxiv:2304.05613 §3] [arxiv:2304.05613 §1].

Two more cards apply the idea to text and transport data. One detects event
triggers from a few keywords per type, with no mention-level labels for the
new types, and then generates the arguments of each event as a filled
template [arxiv:2104.05919 §2.1] [arxiv:2104.05919 §2.2]. The other prompts an
LLM with a structured trip description and no examples, to predict travel mode
or trip purpose [arxiv:2312.00819 §3.2].

Four more cards change how a zero-shot answer is chosen or built. One scores
each LLM answer option against a short task-specific premise, so that common
wordings of one answer do not take probability from another
[arxiv:2104.08315 §3.2]. Another ranks hand-written prompts, and their
paraphrases and back-translations, by perplexity on task inputs, with no labels
involved [arxiv:2212.04037 §7]. A third pretrains a two-tower image-text model
on loosely matched Chinese web pairs, with no object detector, and reports
zero-shot classification on remote-sensing and news data [arxiv:2110.14378
fig. 4a] [arxiv:2110.14378 fig. 5a]. The fourth trains a small residual network
on frozen CLIP text features, using only sentences whose style words have been
dropped, and then applies it to image features as well [arxiv:2311.16445 §4.2].

The first card uses "0-shot" in a narrower sense: LCA_0 is the average
accuracy on a new task before any training on it [arxiv:1812.00420 §3]. That
measure comes from a continual-learning study, so its zero-shot numbers are
secondary to its main contribution.

## Variants

### Attribute-based joint embedding

A feature extractor maps the input to a D-dimensional vector. Each class is
represented by a linear combination of its attribute vector, so an unseen
class gets an embedding without any image of it. Training uses cross-entropy
over the task's classes, scored by the dot product of the two embeddings
[arxiv:1812.00420 §5] [arxiv:1812.00420 eq. 12] [arxiv:1812.00420 eq. 13].
Card: [Efficient Lifelong Learning with A-GEM](/papers/1812.00420).

The paper uses this model only on the two attribute datasets, Split CUB and
Split AWA [arxiv:1812.00420 §6]. Attribute descriptors raise average accuracy
A_T and LCA for every method it tests [arxiv:1812.00420 §6.1]. For example,
A-GEM with attributes reaches 71% on CUB against 62% without, and 50% on AWA
against 44% [arxiv:1812.00420 tab. 5] [arxiv:1812.00420 tab. 6]. Those are
accuracies after the whole task sequence, not zero-shot numbers. Zero-shot
accuracy of the attribute models rises as more tasks are learned, most for
A-GEM [arxiv:1812.00420 §6.1]; the card gives this only in figures
[arxiv:1812.00420 fig. 4] [arxiv:1812.00420 fig. 5].

### Generated pseudo-features for unseen classes

This route does not learn a mapping from descriptions to images. A conditional
VAE models the distribution of image features given a class embedding, trained
on seen classes only [arxiv:1709.00663 §3]. The encoder reads the feature
concatenated with the embedding, and the decoder rebuilds the feature from a
latent sample and the embedding [arxiv:1709.00663 §3.1] [arxiv:1709.00663 §3.2].
For each unseen class, latent samples from a standard normal prior are decoded
with that class's embedding. The results are pseudo-training features for a
linear SVM, with N = 300 samples per class and cost C = 100 [arxiv:1709.00663
alg. 1] [arxiv:1709.00663 §5.3]. Card: [A Generative Model For Zero Shot
Learning Using Conditional Variational Autoencoders](/papers/1709.00663).

The paper's disjoint results are per-class accuracy on AwA1, AwA2, CUB and SUN.
Its AwA1 score is 71.4 against 65.6 for SJE [arxiv:1709.00663 tab. 2]. The
generalised setting, where seen and unseen classes are both tested, is where
the gains are largest. There the harmonic mean is 47.2 on AwA1 and 51.2 on
AwA2, against 27.5 and 27.8 for the best baselines [arxiv:1709.00663 tab. 3].
On ImageNet with 360 unseen classes the top-5 score is 24.7 against 16.8
[arxiv:1709.00663 tab. 4]. The generated features are unimodal, which the
authors note as a limit [arxiv:1709.00663 fig. 3] [arxiv:1709.00663 §5.6]. All
numbers are author-reported.

### Word-vector sketch retrieval

The task is zero-shot sketch-based image retrieval: a sketch query ranks
photos from categories that do not appear in training [arxiv:1904.03451
§4.1]. Two VGG-16 encoders with attention map sketches and photos into one
space [arxiv:1904.03451 §4.2]. Training combines a triplet ranking loss, a
domain loss through a gradient-reversal layer, and a semantic loss that
rebuilds each category's word vector from its embedding [arxiv:1904.03451
§4.3] [arxiv:1904.03451 eq. 4]. The word vectors are word2vec, with GloVe and
fastText named as alternatives [arxiv:1904.03451 §4.3]. Card: [Doodle to
Search: Practical Zero-Shot Sketch-based Image Retrieval](/papers/1904.03451).

The paper adds QuickDraw-Extended, with 330,000 sketches and 204,000 photos
[arxiv:1904.03451 §3.1]. Its full-gallery mAP is 0.3691 on Sketchy, 0.1094 on
TU-Berlin and 0.0752 on QuickDraw [arxiv:1904.03451 tab. 2]. Against the
triplet-only model on Sketchy, each added part raises mAP: attention, the
domain loss and the semantic loss, reaching 0.3691 [arxiv:1904.03451 tab. 3].
The QuickDraw score is low, and the authors say the problem is still unsolved
on that set [arxiv:1904.03451 §5.2]. All numbers are author-reported.

### Language-initialised linear head

CLIP is zero-shot on a new task by scoring the image against prompts built
from class names. This variant keeps that idea as a starting point for
adaptation. Each class column of a linear head is set to the averaged text
embedding of its prompts, with bias zero [arxiv:2204.08790 §4.2]. The paper
compares it with a random head in linear probing and full fine-tuning
[arxiv:2204.08790 tab. 2]. Card: [ELEVATER: A Benchmark and Toolkit for
Evaluating Language-Augmented Visual Models](/papers/2204.08790).

The benchmark averages 20 image-classification datasets. CLIP ViT-B/32 scores
56.64% zero-shot, and 5-shot linear probing reaches 65.35% with the text
initialisation against 58.09% with a random head [arxiv:2204.08790 tab. 2].
Full fine-tuning from a random head drops to 29.75% at 5 shots, against 63.29%
with the text initialisation [arxiv:2204.08790 tab. 2]. With the text
initialisation, full fine-tuning passes linear probing from about 20 shots
[arxiv:2204.08790 §5.1] [arxiv:2204.08790 tab. 2]. The paper also tries
knowledge text from WordNet, Wiktionary and GPT-3 for the class names. For a
UniCL checkpoint, zero-shot accuracy rises from 27.15% to 33.95% with
Wiktionary and GPT-3 together [arxiv:2204.08790 fig. 5].

The authors' own IC protocol trains for 10 search epochs and 50 final epochs
[arxiv:2204.08790 §4.1]. The released code reports the best per-epoch test
accuracy in its final run, so the IC numbers are not fixed-epoch results; see
the card's Pitfalls before comparing.

### Prompted zero-shot LLM

The model receives one prompt: a task instruction followed by the test input,
with no examples [arxiv:2302.06476 §3]. The paper compares ChatGPT
(gpt-3.5-turbo) with GPT-3.5 (text-davinci-003) on 20 datasets in seven task
categories [arxiv:2302.06476 §4.1] [arxiv:2302.06476 tab. 1]. Card:
[Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](/papers/2302.06476).

Results depend strongly on the task. On arithmetic word problems ChatGPT
improves on GSM8K from 59.5% to 78.9% [arxiv:2302.06476 tab. 11]. It also
beats GPT-3.5 on several entailment and dialogue sets, and on SST-2 sentiment
(93.7% against 88.8%) [arxiv:2302.06476 tab. 4] [arxiv:2302.06476 tab. 12].
It is weaker on summarization, where its mean ROUGE is 31.0 against 32.4
[arxiv:2302.06476 tab. 9], and on named entity recognition, where F1 is about
equal at 53.2 against 53.5 but far below fine-tuned models [arxiv:2302.06476
tab. 10]. On the Coin Flip set it drops from 97.8% to 65.8% [arxiv:2302.06476
tab. 11].

For reasoning, the paper adds a two-stage zero-shot chain-of-thought step: a
"think step by step" instruction first, then the rationale fed back with an
answer trigger [arxiv:2302.06476 §3]. This helps arithmetic but lowers
CommonsenseQA from 73.7% to 71.5% and StrategyQA from 61.1% to 55.5%
[arxiv:2302.06476 tab. 3].

The same no-example prompting is applied to analogy puzzles. GPT-3
(text-davinci-003) is given each puzzle as text, and the answer is either
written by the model or chosen by its mean token log-probability over the
options. The temperature is 0 with one run per item [arxiv:2212.09196 §4.2]. On
Digit Matrices the model beats the average undergraduate, with odds ratio 1.88
for generated answers and 6.27 for multiple choice [arxiv:2212.09196 §2.1]. It
matches or beats the human average on letter-string analogies, odds ratio 1.76
[arxiv:2212.09196 §2.2]. On story analogies it is weaker than students, with
odds ratio 0.37 [arxiv:2212.09196 §2.4]. Card: [Emergent Analogical Reasoning
in Large Language Models](/papers/2212.09196). The numbers are author-reported,
and the paper states no model snapshot date, so the results cannot be checked
against current models [arxiv:2212.09196 §4.2].

A domain case study comes from a finance survey, not a method paper. The
authors put a role description, an output format and one example in the system
message of GPT-4, then score six finance datasets [arxiv:2401.11641 §4.2]
[tab. 2]. The reported scores are 0.78 accuracy on Financial PhraseBank
sentiment, 0.79 on FiQA-SA, 0.81 entity F1 for financial NER, 0.64 and 0.73
exact match on FinQA and ConvFinQA, and 0.53 accuracy on stock movement
[arxiv:2401.11641 tab. 2]. No baseline is reported. The prompt text, model
snapshot, decoding settings and per-dataset sample counts are not stated, and
which of the paper's prompting strategies produced each score is unclear
[arxiv:2401.11641 §4.2] [tab. 2]. Card: [Revolutionizing Finance with LLMs: An
Overview of Applications and Insights](/papers/2401.11641).

### Premise-normalised scoring of multiple-choice options

Here the model is not asked to choose; each option is scored and the highest
score wins. The usual score, the raw probability of an option after the
question, is penalised when an answer has many valid wordings, because the
probability is split across them [arxiv:2104.08315 §1] [arxiv:2104.08315 §2].
Domain Conditional PMI divides the option's probability after the question by
its probability after a short domain premise, such as "the answer is:". The
premise is written by hand for each task [arxiv:2104.08315 §3.2] [arxiv:2104.08315
tab. 7]. No model is trained. Card: [Surface Form Competition: Why the Highest
Probability Answer Isn't Always Right](/papers/2104.08315).

On GPT-3 175B the paper's table gives 89.2 on COPA, against 85.2 for raw
probability and 82.8 for length-normalised likelihood [arxiv:2104.08315 tab. 1].
Across 16 splits and every model size, the method is best or tied on 62.5% of
splits for GPT-3 175B, against 18.75% for length-normalised likelihood
[arxiv:2104.08315 tab. 2]. The paper also reports that it loses on HellaSwag,
ARC-Easy and BoolQ in some settings [arxiv:2104.08315 §6]. On COPA Flipped, where the
answer string is fixed, the rules that do not correct for surface form give the
same score as the corrected one [arxiv:2104.08315 tab. 5]. All numbers are
author-reported.

### Perplexity-ranked prompt selection

Prompt performance can differ a lot between paraphrases of one instruction. The
paper's measure is the perplexity of the whole prompt under the model, averaged
over about 1,000 task inputs, without the label [arxiv:2212.04037 §2]
[arxiv:2212.04037 §4.2]. Across OPT and Bloom, prompts with lower perplexity
usually score better, with negative correlations on most classification tasks
[arxiv:2212.04037 tab. 5]. SPELL uses that finding as a selection rule. It
paraphrases hand-written seeds with GPT-3, back-translates each through
languages, ranks the candidates by mean perplexity, and keeps the three lowest
[arxiv:2212.04037 §3] [arxiv:2212.04037 §7]. Card: [Demystifying Prompts in
Language Models via Perplexity Estimation](/papers/2212.04037).

The average gain over the manual prompts is 1.8 accuracy points for OPT and 2.3
for Bloom [arxiv:2212.04037 §7.1]. The per-task spread is wide. AG News drops
by 12.5 points with Bloom, and DBpedia drops by 5.7 with OPT
[arxiv:2212.04037 tab. 10]. The positive correlations on IMDB at OPT 175B and
on Newspop at OPT 1.3B are not discussed in the text [arxiv:2212.04037 tab. 5]. The study
uses two model families [arxiv:2212.04037 §4.1] and English prompts only.

### Zero-shot prompting of structured trip data

The prompt for a traveller has six parts, in this order: a task description
that names the allowed options, travel time and cost per option as structured
data, traveller attributes written as plain sentences, rules and comparison
sentences about cost and time, and a JSON answer with a prediction and a
reason [arxiv:2312.00819 §3.3]. It contains no training examples, and the model
is called at temperature 0 [arxiv:2312.00819 §4.3]. The same serialised text
can instead be embedded, and a supervised head (multinomial logit, random
forest or neural network) fitted on those vectors [arxiv:2312.00819 §3.4].
Card: [Large Language Models for Travel Behavior Prediction](/papers/2312.00819).

On the Swissmetro mode-choice survey, zero-shot GPT-3.5 reaches 0.586
accuracy. That is above a multinomial logit trained on 10 samples (0.469), and
above the same logit with embedding features (0.494). A neural network trained
on 1,000 samples reaches 0.635 [arxiv:2312.00819 tab. 3]. On the 2017 US
travel survey for trip purpose, zero-shot GPT-3.5 reaches 0.402, against 0.313
for the 10-sample logit [arxiv:2312.00819 tab. 3]. GPT-4 and Llama 3.1 70B
give 0.570 and 0.540 on mode, and 0.401 and 0.408 on purpose
[arxiv:2312.00819 tab. 3]. All numbers are averages over five random samples
of balanced 200-row test sets [arxiv:2312.00819 §4.1]. The authors read this
as zero-shot LLMs beating small-sample supervised models, and as large-sample
supervised models doing best [arxiv:2312.00819 §4.4.1]. Removing the
step-by-step guidance from the prompt lowers GPT-3.5's mode accuracy by 5.8%,
and GPT-4's by under 1% [arxiv:2312.00819 §4.4.3]. The ablation covers mode
choice only.

### Multilingual prompting with one prompt per example

The same no-example setup is tested across languages. The paper treats
ChatGPT as a black box and builds one prompt per example: a task description,
then the input [arxiv:2304.05613 §3]. The description is written in English or
in the target language, and the model's reply is parsed with task-specific
rules that the paper does not publish [arxiv:2304.05613 §3]. The seven tasks
are POS tagging, NER, relation extraction, NLI, QA, commonsense reasoning and
summarization, with languages grouped by their share of CommonCrawl
[arxiv:2304.05613 §3] [arxiv:2304.05613 §4]-[§10]. Card: [ChatGPT Beyond
English: Towards a Comprehensive Evaluation of Large Language Models in
Multilingual Learning](/papers/2304.05613).

Results depend on the task. With English descriptions, average POS accuracy
is 84.5% against 79.3% for the supervised XLM-R baseline [arxiv:2304.05613
§4]. The gap is wide elsewhere: average NER macro-F1 is 29.9 against 88.4 for
DAMO [arxiv:2304.05613 §5], average NLI accuracy is 57.0 against 87.1 for
mT5-XXL [arxiv:2304.05613 §7], and average QA F1 is 53.5 against 85.2
[arxiv:2304.05613 §8]. Commonsense reasoning is the one case where the
target-language description has the higher average, 51.9 against 47.8, while
the TRT baseline scores 59.0 [arxiv:2304.05613 §9]. Summaries are often too
long, and the share of non-empty replies is 42% for Bengali and 43% for
Burmese with English descriptions [arxiv:2304.05613 §10]. The model snapshot
is not named, and responses were gathered between 1 March and 5 April 2023
[arxiv:2304.05613 §3]. Every number is author-reported.

### Keyword-seeded event extraction with generated arguments

This route covers two steps for whole documents. Triggers, the words that
mark an event, are tagged by a sequence labeller whose class vectors come from
a few seed keywords per event type, so a type with no labelled mentions can
still be detected [arxiv:2104.05919 §2.2]. Its zero-shot training uses
pseudo-labels from similarity between token and class vectors, and tokens with
uncertain similarity are left out of the loss [arxiv:2104.05919 §2.2]. For
each trigger, a BART encoder-decoder then reads an event template together
with the document and writes the template back with its slots filled. Its
output is limited to words that appear in the input [arxiv:2104.05919 §2.1]
[arxiv:2104.05919 eq. 2]. Card: [Document-Level Event Argument Extraction by
Conditional Generation](/papers/2104.05919).

On the ACE 2005 data, the keyword-based trigger model with no seen event types
reaches 55.19 trigger identification F1 [arxiv:2104.05919 tab. 8]. Paired with
the argument generator trained on the 10 most frequent types, the pipeline
reaches 72.31 trigger identification F1 and 48.18 argument identification Head
F1. BERT-QA, trained on the same 10 types, reaches 39.37 argument
identification Head F1 [arxiv:2104.05919 tab. 8]. The
supervised reference is 74.36 and 55.22 [arxiv:2104.05919 tab. 8]. Argument
rows are not given for the zero-shot setting, only for models trained on
some types [arxiv:2104.05919 tab. 8]. On WikiEvents, the argument generator
scores 64.57 argument classification Head F1 against 56.16 for BERT-QA
[arxiv:2104.05919 tab. 5]. The paper's gains depend on which column is
compared, so check the metric before quoting one. All numbers are
author-reported. The released code contains the argument generator only, so
the trigger model and its zero-shot training cannot be reproduced from it
[arxiv:2104.05919 §2.2].

### Multitask prompted training

This variant trains the model, rather than only prompting it. The T0 model is
a T5-style encoder-decoder trained by maximum likelihood to produce the target
text [arxiv:2110.08207 §5]. Each training dataset is written as several prompt
templates, and the training mixture draws examples from all of them
[arxiv:2110.08207 §4]. Evaluation scores each answer option by its summed
log-probability, with no length normalisation [arxiv:2110.08207 §5]. The
held-out tests cover four task types, namely natural language inference,
coreference, sentence completion and word sense disambiguation. Datasets of
those types are excluded from the training mixture [arxiv:2110.08207 §3].
Card: [Multitask Prompted Training Enables Zero-Shot Task Generalization](/papers/2110.08207).

On the 11 held-out datasets, the 11B model matches or exceeds every GPT-3
model on 9, with Winogrande and HellaSwag as the two exceptions
[arxiv:2110.08207 §6.1]. Per-dataset medians are in [arxiv:2110.08207 tab. 7];
for example CB reaches 78.57% and RTE 81.23%. These are medians across all
prompts, while GPT-3 is reported with one prompt per dataset, so the two are
not like for like [arxiv:2110.08207 §6.1]. The paper selects the checkpoint by
validation score on training datasets [arxiv:2110.08207 §5]; the released
evaluation uses a fixed step, which the card's Paper vs code section records.
Prompt count matters: moving from one prompt to 5.7 raises the median on 8 of
11 datasets [arxiv:2110.08207 §6.2]. The prompts themselves are in a separate
collection and tool, so reproducing the training mixture needs both
[arxiv:2110.08207 §4] [arxiv:2110.08207 app. G]. One T0 run took about 27 TPU
v3-512 hours [arxiv:2110.08207 app. B.1].

### Multi-agent prompting for medical multiple choice

MedAgents needs no training. The model names question experts and option
experts for each case, and each writes an analysis [arxiv:2311.10537 §2.1]
[arxiv:2311.10537 §2.2]. The analyses are merged into one report
[arxiv:2311.10537 §2.3]. Every expert votes YES or NO on the report, and a NO
comes with a revision proposal; this repeats for up to five rounds
[arxiv:2311.10537 §2.4] [arxiv:2311.10537 alg. 1]. A final prompt picks one
option letter from the agreed report [arxiv:2311.10537 §2.5]. Card:
[MedAgents: Large Language Models as Collaborators for Zero-shot Medical
Reasoning](/papers/2311.10537).

The paper reports 86.7% average accuracy for GPT-4 over nine datasets, against
80.6% for zero-shot GPT-4 and 85.4% for few-shot chain-of-thought with
self-consistency [arxiv:2311.10537 tab. 2]. For GPT-3.5 the average is 72.1%
against 67.8% zero-shot [arxiv:2311.10537 tab. 2]. The authors call the
few-shot self-consistency baseline comparable, and the gap is small on
average [arxiv:2311.10537 §3.2]. In a MedQA ablation with the model not
stated, direct prompting scores 49.0% and the full pipeline 67.0%
[arxiv:2311.10537 tab. 3]. The paper gives a cost of about $1.41 per 100
question-answer examples and about 40 seconds per example [arxiv:2311.10537
§3.1]. Its temperature is 1.0 for every call [arxiv:2311.10537 §3.1]. The
released code uses a different setting; see the card's Paper vs code section.

Author-reported throughout. The authors' own error review of 40 sampled cases
attributes most errors to missing or misapplied medical knowledge, with a
smaller share to the rationale [arxiv:2311.10537 §4.6].

### Text-queried region matching (open-vocabulary segmentation)

OpenSeg represents an image as N = 128 mask proposals, each with its own
feature vector [arxiv:2112.12143 §3] [arxiv:2112.12143 app. E]. The mask
branch is trained on class-agnostic COCO masks with a Dice loss, and the
alignment branch on image-caption pairs with a grounding loss
[arxiv:2112.12143 §3.1] [arxiv:2112.12143 §3.2]. At inference, each class
name is embedded by a frozen text encoder, compared with the region features,
and combined with the masks to give per-pixel labels [arxiv:2112.12143 §3.4].
Card: [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](/papers/2112.12143).

The paper calls its PASCAL-Context and PASCAL-20 results zero-shot evaluation
[arxiv:2112.12143 tab. 2]. Adding Localized Narratives caption data raises
PASCAL-20 mIoU from 60.0 to 63.8 with ResNet-101 [arxiv:2112.12143 tab. 2].
The best row, with an EfficientNet-B7 backbone and prompt engineering, reaches
72.2 [arxiv:2112.12143 tab. 2]. The ablation shows that inference with the
predicted masks beats pixel-wise labelling without masks on PASCAL-Context
(42.1 against 32.1 mIoU on PC-59) [arxiv:2112.12143 tab. 4]. These are
author-reported numbers. The 72.2 row is compared with an LSeg score copied
from its own paper, not re-run [arxiv:2112.12143 §4.3] [arxiv:2112.12143 tab. 2].

### Text-to-shape through CLIP features

The model is trained only on rendered views, their CLIP image features and
shape data without text labels [arxiv:2110.02624 §3]. A voxel autoencoder
squeezes each 32^3 grid into a 128-dimensional shape embedding
[arxiv:2110.02624 §3.1] [arxiv:2110.02624 §4]. A conditional RealNVP flow then
learns to produce those embeddings from the CLIP image features
[arxiv:2110.02624 §3.2]. At test time the CLIP text features take the place of
the image features. The flow is inverted from Gaussian noise, so one prompt
can give several shapes and nothing is optimised per request
[arxiv:2110.02624 §3.3] [arxiv:2110.02624 §1]. Card:
[CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](/papers/2110.02624).

On 224 prompts over ShapeNet categories, the author-reported FID is 2425.25
and category accuracy is 83.33% [arxiv:2110.02624 tab. 1]. The two text2shape
baselines score 14881.96 and 16078.05 FID, with 6.84% and 4.27% accuracy
[arxiv:2110.02624 tab. 1]. Those baselines were trained on chairs and tables
only, while the main model uses 13 classes. Trained on chairs and tables
alone, the model scores 14746.90 FID and 30.77% accuracy
[arxiv:2110.02624 app. B]. The prompt wording alone moves FID between about
2400 and 3140 [arxiv:2110.02624 tab. 2]. The paper does not measure how well
the text features match the image features the flow was trained on
[arxiv:2110.02624 §3.3].

### Training-free cache on zero-shot CLIP

Zero-shot CLIP scores an image against text prompts built from class names.
Tip-Adapter keeps a cache of CLIP features and labels from a few examples per
class. For a test image, it finds how similar each cached example is, and
adds the labels of similar examples to the zero-shot scores
[arxiv:2111.03930 §3.2] [arxiv:2111.03930 eq. 5] [arxiv:2111.03930 eq. 6].
Its weights are set from the cache, so the cache needs no gradient steps
[arxiv:2111.03930 §3.2]. An optional step fine-tunes the cache keys for 20
epochs, giving Tip-Adapter-F [arxiv:2111.03930 §4.1]. Card:
[Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language
Modeling](/papers/2111.03930).

On ImageNet with ResNet-50, zero-shot CLIP scores 60.33%
[arxiv:2111.03930 tab. 2]. Training-free Tip-Adapter scores 62.03% with 16
shots per class and 60.70% with one [arxiv:2111.03930 tab. 2]. CLIP-Adapter
scores 63.59% at 16 shots, after 200 epochs that take 50 minutes on one GPU;
Tip-Adapter-F scores 65.51% after 20 epochs and 5 minutes
[arxiv:2111.03930 tab. 3]. The training-free variant is below CLIP-Adapter at
every shot count in that table [arxiv:2111.03930 tab. 2]. The abstract calls
the two comparable [arxiv:2111.03930 §1]. The paper's eq. 6 has no scale on
the zero-shot term, while the card's Paper vs code section records that the
released code adds one. Results on other datasets at 16 shots appear only in a
figure, with zero-shot CLIP, Tip-Adapter and Tip-Adapter-F all reported
[arxiv:2111.03930 fig. 4].

### Text-only adapter for style-invariant CLIP features

CLAP keeps CLIP frozen and learns one small network on top of it. The network
is a residual two-layer MLP, and it starts as the identity because its last
layer is zero-initialised [arxiv:2311.16445 §4.1.2]. Training uses text only.
Each class has many template prompts, and each prompt gets a changed copy in
which size, colour or order words are dropped or moved [arxiv:2311.16445
§4.2.1] [arxiv:2311.16445 tab. 2]. A contrastive loss pulls a prompt towards
its changed copy, and the changed copy towards the bare class name, so the
network learns to keep content and drop style [arxiv:2311.16445 eq. 4]. At
inference the same network is applied to CLIP's image features and text
features, with a scalar weight α on its residual branch [arxiv:2311.16445 §4.1.3]
[arxiv:2311.16445 app. A.1]. Card: [CLAP: Isolating Content from Style through
Contrastive Learning with Augmented Prompts](/papers/2311.16445).

Averaged over PACS, VLCS, OfficeHome and DomainNet, zero-shot top-1 with a
bare class-name prompt is 79.9% for ViT-B/16, against 77.4% for CLIP and 76.1%
for the image-augmented baseline [arxiv:2311.16445 tab. 3]. With a noised
prompt the gap is larger: 76.1% against 70.4%, and the drop from the clean
prompt falls from 7.0 to 3.8 points [arxiv:2311.16445 tab. 3] [arxiv:2311.16445
tab. 4]. One-shot results under adversarial attacks are also higher, 29.5%
against 22.0% [arxiv:2311.16445 tab. 5]. The α values come from the paper's
appendix, and the card's Pitfalls section records that the released code picks
α on the test environments. All numbers are author-reported.

### Zero-shot classification at scale

The model is CLIP-style. Each class embedding is the mean of its prompt
embeddings, normalised, and an image takes the class with the highest cosine
similarity [arxiv:2212.07143 §4.1]. The paper trains OpenCLIP dual encoders
with InfoNCE on LAION subsets, from ViT-B/32 to ViT-g/14, and varies model
size, data size and samples seen [arxiv:2212.07143 §3.2] [tab. 18]. It fits a
line in log-log space to the lowest-error runs in each compute bin
[arxiv:2212.07143 §4.1]. The fitted ImageNet zero-shot exponent is -0.11 for
OpenCLIP on LAION and -0.16 for OpenAI CLIP [arxiv:2212.07143 §4.1]. Card:
[Reproducible scaling laws for contrastive language-image learning](/papers/2212.07143).

The best measured run, ViT-H/14 on LAION-2B with 34B samples seen (a nominal
label), reaches 78.0% zero-shot ImageNet top-1, against 75.5% for the OpenAI
ViT-L/14 baseline [arxiv:2212.07143 tab. 1]. Predictions beyond the measured
runs, such as 81.92% for ViT-G/14 at 68B samples, are extrapolations and are
not measured [arxiv:2212.07143 tab. 4]. The paper also reports linear-probe
and fine-tuning gains with scale [arxiv:2212.07143 §4.3] [§4.4]. Author-reported
throughout.

### Two-tower pre-training on loosely matched web pairs

BriVL trains an image encoder and a text encoder with a contrastive loss, over
about 650M Chinese image-text pairs from the web [arxiv:2110.14378 §Results:
Pre-Training Data Collection]. The pairs are weakly correlated, so a caption
may only loosely relate to its picture. The image side is a CNN with multi-scale
patch pooling and a small self-attention block, rather than an object
detector [arxiv:2110.14378 §Methods: Image Encoder]. To get many negatives
without a huge batch, it keeps two momentum-updated queues of recent keys
[arxiv:2110.14378 §Methods: Contrastive Loss]. Card: [Towards artificial general
intelligence via a multimodal foundation model](/papers/2110.14378).

On zero-shot remote-sensing scene classification with UCM and 5 unseen classes
against 16 seen ones, BriVL reaches 82.41%, against 76.02% for CLIP with
ResNet-50x4 [arxiv:2110.14378 fig. 4a]. On AID with 30 unseen classes it reaches
58.12% against 50.96% for the same CLIP model [arxiv:2110.14378 fig. 4b]. On
Toutiao News, title-only zero-shot accuracy is 62.38% with RoBERTa-large, against
42.55% for RoBERTa-large alone [arxiv:2110.14378 fig. 5a]. The class names for
BriVL were Chinese, while CLIP was given English names, so the comparison is
confounded [arxiv:2110.14378 §Pitfalls]. The released model and inference code
are licensed for non-commercial use without derivatives, as the paper states
[arxiv:2110.14378 §Code availability]. Treat the code as unusable for a
commercial product. All numbers are author-reported.

### Composed retrieval with a pseudo word token

Composed image retrieval takes a reference image plus a text edit, such as
"in watercolor", and ranks target images. Pic2Word needs no labelled triples
of reference, edit and target [arxiv:2302.03084 §1]. A three-layer MLP maps
the frozen CLIP ViT-L/14 image embedding to one pseudo token, which is placed
in a prompt such as "a photo of [*], [text]" [arxiv:2302.03084 §3.2]
[arxiv:2302.03084 §3.3] [arxiv:2302.03084 §4.1]. The mapper is trained on
unlabelled CC3M images with a symmetric contrastive loss, and the CLIP encoders
stay frozen [arxiv:2302.03084 §3.2] [arxiv:2302.03084 §4.1]. Card:
[Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image
Retrieval](/papers/2302.03084).

On the CIRR test split, Pic2Word scores 65.3 R@10, above TIRG (64.1) and
ARTEMIS (61.3), and below Combiner trained on CIRR (73.2) [arxiv:2302.03084
tab. 3]. On Fashion-IQ validation it averages 24.7 R@10 [arxiv:2302.03084
tab. 4]. Text alone is a strong baseline on both sets: 56.7 R@10 on CIRR test
and 17.3 on Fashion-IQ validation [arxiv:2302.03084 tab. 3] [arxiv:2302.03084
tab. 4]. On COCO object composition Pic2Word reaches 33.4 R@10 against 32.2
for Combiner trained on CIRR [arxiv:2302.03084 tab. 2]. The paper reports
Pic2Word ahead of Combiner when fewer than 1,000 labelled triples are
available for training Combiner [arxiv:2302.03084 §4.3] [arxiv:2302.03084
fig. 7]. The mapper's own variants change CIRR R@10 by up to about 7 points
[arxiv:2302.03084 app. B].

### Tri-modal 3D pre-training with generated captions

ULIP-2 aligns three things: a 3D point cloud, rendered images of the same
object, and text. The image and text encoders are a pretrained OpenCLIP
ViT-G/14 model, which the paper treats as frozen [arxiv:2305.08275 §3.3]. Only
the 3D encoder learns. Each rendered view gets ten candidate captions from
BLIP-2, and the caption that CLIP ranks highest is kept [arxiv:2305.08275
§4.1]. Card: [ULIP-2: Towards Scalable Multimodal Pre-training for 3D
Understanding](/papers/2305.08275).

For zero-shot 3D classification, the paper reports 50.6% top-1 on
Objaverse-LVIS and 84.7% on ModelNet40 with the Point-BERT encoder, both with
no manual captions [arxiv:2305.08275 tab. 1]. The table lists OpenShape at
46.8% top-1 on Objaverse-LVIS [arxiv:2305.08275 tab. 1]. On ModelNet40, a
holistic BLIP-2 caption gives 69.7% top-1 against 60.4% for manual captions,
in an ablation with ShapeNet pre-training [arxiv:2305.08275 tab. 5]. Both
figures are author-reported.

Two caveats matter for use. The pre-training recipe is not fully stated: the
paper gives no optimiser, learning rate or epoch count [arxiv:2305.08275 §4.1].
The paper does not say whether its "Objaverse + ShapeNet" pre-training includes
the Objaverse-LVIS evaluation shapes [arxiv:2305.08275 tab. 1]. The scaling
study trains on Objaverse without LVIS and reports 35.7% on Objaverse-LVIS for
the chosen encoder size, so the two Objaverse-LVIS numbers come from different
training sets [arxiv:2305.08275 tab. 9]. The triplets are costly: the
Objaverse set alone has about 10 million images, each needing ten caption
generations [arxiv:2305.08275 tab. 2] [arxiv:2305.08275 §4.1]. See the card's
Paper vs code section for what the public release contains.

### Frozen-encoder 3D alignment with category names

ULIP keeps the image and text encoders of SLIP frozen and trains only a 3D
point-cloud encoder [arxiv:2212.05171 §3.2]. Each training object is a
triplet of a rendered view, its category name and its point cloud, built from
ShapeNet55 CAD models without manual captions [arxiv:2212.05171 §3.1]. The
3D feature is projected to a 512-dimensional space and trained with symmetric
contrastive losses against the image and text features [arxiv:2212.05171 §3.2]
[arxiv:2212.05171 eq. 4]. The paper's default weights turn the image-text term
off, so the 3D encoder is pulled toward the text and image features
[arxiv:2212.05171 eq. 5]. For zero-shot classification, each class name is
filled into 64 prompts, the embeddings are averaged, and the object takes the
nearest class [arxiv:2212.05171 §4.5]. Card: [ULIP: Learning a Unified
Representation of Language, Images, and Point Clouds for 3D
Understanding](/papers/2212.05171).

On ModelNet40 with all 40 classes, PointBERT with ULIP reaches 60.4% top-1
against 20.2% for PointCLIP [arxiv:2212.05171 tab. 4]. On the Medium and Hard
subsets it reaches 40.4% and 37.1%. Those subsets remove categories that also
appear in the pre-training list [arxiv:2212.05171 §4.5]. On ScanObjectNN,
zero-shot top-1 is 48.5% for PointBERT with ULIP [arxiv:2212.05171 tab. 5].
The two-modality ablation favours aligning both pairs: for PointBERT on
ModelNet40, text only gives 44.7%, image only 35.5%, and both 60.4%
[arxiv:2212.05171 tab. 3]. All numbers are author-reported. The paper does not
say how its reported checkpoint is chosen; the card's Paper vs code section
records that the released code keeps the best epoch on the ModelNet40 test
split, which makes the headline numbers likely optimistic.

Pre-training starts from rendered views: each CAD model is rendered from 60
viewpoints [arxiv:2212.05171 §3.1], and the frozen SLIP encoders run in every
training step [arxiv:2212.05171 §3.2]. The paper describes averaging 64 prompts
during pre-training as well as at test time [arxiv:2212.05171 §3.1]. The
released code averages them only at test time, so the paper's pre-training
recipe is not reproduced exactly; see the card's Paper vs code section.

### Frozen LLM, CLIP and a map graph for navigation

LM-Nav trains nothing for the task. A frozen GPT-3 turns a route instruction
into an ordered list of landmark descriptions [arxiv:2207.04429 §4.2]. CLIP
scores each landmark against each node of a topological map of images
collected on an earlier drive, using the prompt "This is a photo of" followed
by the landmark [arxiv:2207.04429 §4.3] [arxiv:2207.04429 §5.2]. A dynamic
program then picks the walk that visits the landmarks in order and penalises
travel distance [arxiv:2207.04429 §4.4] [arxiv:2207.04429 alg. 1]. The robot
follows that walk with a visual navigation model, ViNG, which predicts the
temporal distance between two images [arxiv:2207.04429 §3]. Card:
[LM-Nav](/papers/2207.04429).

The paper reports 0.8 net success on 20 outdoor instructions, over more than
6 km, in two environments on a small wheeled robot [arxiv:2207.04429 tab. 1]
[arxiv:2207.04429 §5.2]. A GPS-only baseline without the navigation model
reaches 0.23 in one environment [arxiv:2207.04429 tab. 1]. The planner beats a
max-likelihood baseline on the larger environment, 0.8 against 0.2 success
[arxiv:2207.04429 tab. 4]. The main failure is landmark detection: a fire
hydrant or cement mixer may go unseen [arxiv:2207.04429 §5.2] [arxiv:2207.04429
fig. 4]. The language and vision models ran on a remote workstation, and the
paper reports no latency during execution [arxiv:2207.04429 §5.1]. All numbers
are author-reported.

### Visual prompting for VLM navigation

The vision-language model is not fine-tuned. At each step, a depth-based
test finds directions the robot can drive in without hitting a wall, and each
direction is drawn as a numbered arrow on the RGB image [arxiv:2411.05755 §3.1]
[arxiv:2411.05755 §3.3]. Candidate arrows are spaced so that unexplored
directions are favoured [arxiv:2411.05755 §3.2]. One model call picks an arrow
number, and a separate call on the un-annotated image decides when to stop
[arxiv:2411.05755 §3.4] [arxiv:2411.05755 §3.5]. Card:
[End-to-End Navigation with Vision Language Models: Transforming Spatial
Reasoning into Question-Answering](/papers/2411.05755).

On Habitat ObjectNav with Gemini Flash, the paper reports 50.4% success and
0.210 SPL. PIVOT scores 24.6% and 0.106, and the same model without the
navigability step scores 33.2% [arxiv:2411.05755 tab. 1]. Turning off wall
sliding drops success to 12.9% [arxiv:2411.05755 tab. 1]. The navigability step
ignores the robot's size and shape, and the authors say the method should not
be assumed to transfer to a real robot without a collision-aware controller
[arxiv:2411.05755 §4.1] [arxiv:2411.05755 §5]. On GOAT, success is 16.3% with sliding on, against
8.3% for PIVOT, but below two specialised systems at 29.5% and 24.9%
[arxiv:2411.05755 tab. 2] [arxiv:2411.05755 tab. 3]. All numbers are
author-reported, and each step costs two model calls.

### LLM-written programs over pretrained skills

The paper treats an LLM as a planner. It receives a task instruction and a
fixed prompt of 15 worked examples, and writes a short program with one line
per module call [arxiv:2412.10402 §3]. The modules are a pretrained point-goal
navigation policy, a value map that stores a feature per pixel so that targets
seen earlier can be revisited, and object detectors with a CLIP-based filter
[arxiv:2412.10402 §3] [arxiv:2412.10402 §4.1]. The planner is GPT-4o
[arxiv:2412.10402 §3]. No module is trained for the target tasks
[arxiv:2412.10402 §4.1]. Card: [TANGO: Training-free Embodied AI Agents for
Open-world Tasks](/papers/2412.10402).

On HM3D-OVON val unseen, the paper reports 35.5% success and 19.5 SPL, against
35.2% and 19.6 for VLFM [arxiv:2412.10402 tab. 1]. On OpenEQA the score is
37.2, against 38.1 for a Socratic-LLM baseline with frame captions
[arxiv:2412.10402 tab. 2]. On GOAT-Bench it reaches 32.1% success against 29.5%
for the best prior success row, with SPL 16.5 against 17.2 for Modular GOAT
[arxiv:2412.10402 tab. 3]. The text calls the method on par with VLFM on
OVON and second on OpenEQA [arxiv:2412.10402 §4.2]. The released evaluator
does not call the LLM; it runs fixed programs per task, so its numbers do not
test the planning step. See the card's Paper vs code section before using it.

### Language-conditioned visuomotor policy

BC-Z is imitation learning for a real arm. One policy takes a camera image and
a task embedding, and outputs motions of the end effector [arxiv:2202.02005
§3] [arxiv:2202.02005 §5.2]. The embedding comes from a pretrained sentence
encoder, with 512 dimensions, and it conditions a ResNet-18 through FiLM layers
in all four blocks [arxiv:2202.02005 §5.1] [arxiv:2202.02005 §5.3]. The
training data are demonstrations of 100 tasks, collected with a VR rig at 10 Hz.
Some of the data are human corrections made while a learned policy ran, which
the paper calls HG-DAgger [arxiv:2202.02005 §4]. Zero-shot means the policy is
given a command for a task that has no demonstrations in training
[arxiv:2202.02005 §1]. Each action is the offset from the current state to a
pose N steps ahead. The policy predicts ten future actions but executes only
the first [arxiv:2202.02005 §5.2]. Card: [BC-Z: Zero-Shot Task Generalization
with Robotic Imitation Learning](/papers/2202.02005).

On 28 held-out task rows, the language command reaches 32% average success, and
the video command 4% [arxiv:2202.02005 tab. 3]. Per-task success in the
one-distractor column runs from 0% to 87%, and its overall row is 38%; the
four-to-five-distractor overall row is 32% [arxiv:2202.02005 tab. 2]. With 21
training tasks, language conditioning scores 40% and one-hot task IDs 42%
[arxiv:2202.02005 tab. 3]. The authors read the near-equal numbers as evidence
that the language space is adequate, so the held-out gap sits in the policy
[arxiv:2202.02005 §6.2]. A single-task policy trained on more than 300 demonstrations
of each of four held-out tasks scores 0% on all four [arxiv:2202.02005 §6.2].
The numbers come from one real robot setup with 12 robots, 7 operators and 125
hours of robot time [arxiv:2202.02005 §1]. All numbers are author-reported. The
released code covers only the language-conditioned model, and its training
settings differ from the paper's; see the card's Paper vs code section.

### Continued CLIP pretraining on remote-sensing pairs

This variant trains the model. It starts from CLIP and keeps training with the
symmetric contrastive loss on aerial and satellite image-text pairs
[arxiv:2306.11029 §3.1] [arxiv:2306.11029 eq. 1]. The text is generated from
existing labels. Box-to-caption writes five captions for each image from its
object boxes, and mask-to-box turns segmentation masks into boxes first
[arxiv:2306.11029 §3.2.1] [arxiv:2306.11029 §3.2.2]. The training set has
165,745 images, each with five captions, so 828,725 pairs [arxiv:2306.11029
§4.1.2]. Zero-shot classification fills the prompt "a satellite photo of
{class name}." [arxiv:2306.11029 §4.2.3]. Card: [RemoteCLIP: A Vision Language
Foundation Model for Remote Sensing](/papers/2306.11029).

Averaged over 12 classification datasets, zero-shot accuracy rises from 56.02%
to 62.41% for ViT-B-32, from 65.67% to 71.30% for ViT-L-14, and from 50.61% to
53.46% for ResNet-50 [arxiv:2306.11029 tab. 3]. The ViT-L-14 model beats CLIP
on 9 of 12 datasets [arxiv:2306.11029 §4.2.3]. The gain is not uniform: the
largest drops are on EuroSAT, at −11.15 points for ViT-B-32 and −24.83 for
ResNet-50 [arxiv:2306.11029 tab. 3]. The authors attribute the drops to input
resolution and image distribution [arxiv:2306.11029 §4.2.3]. The largest model
took 233.4 hours on four GPUs [arxiv:2306.11029 §4.1.3]. Author-reported
throughout.

### Cross-frame video encoder with video-specific prompts

X-CLIP reuses a pretrained CLIP image encoder for video, and does not add
video-text pretraining [arxiv:2208.02816 §1]. Each frame's [class] token
gives a message token, and the message tokens exchange information across
frames inside every transformer block [arxiv:2208.02816 §3.2.1]. A small
temporal transformer then merges the frame vectors into one video vector
[arxiv:2208.02816 §3.2]. On the text side the class name is embedded by the
CLIP text encoder, and a prompt generator adds a term built from the video's
patch tokens [arxiv:2208.02816 §3.3.1]. A video and a class are scored by
cosine similarity [arxiv:2208.02816 eq. 3]. Card: [Expanding Language-Image
Pretrained Models for General Video Recognition](/papers/2208.02816).

The fully supervised result on Kinetics-400 is 87.1% top-1 for ViT-L/14 at 8
frames, against 84.8% for ViViT-H/16×2 [arxiv:2208.02816 tab. 1]. In the
zero-shot setting, trained on Kinetics-400 with 32 frames and one view,
X-CLIP-B/16 scores 72.0% on UCF-101 against 58.3% for ActionCLIP, and 65.2%
on 220 unseen Kinetics-600 classes against 42.1% for ER-ZSAR
[arxiv:2208.02816 tab. 3] [arxiv:2208.02816 tab. 4]. The unseen-class score
averages draws of 160 of those 220 classes [arxiv:2208.02816 app. C]. In the
paper's own ablation, video-specific prompts raise zero-shot Kinetics-400
accuracy from 63.9% to 70.0% over a CLIP prompt ensemble [arxiv:2208.02816
tab. 9]. Freezing the text branch costs points in zero-shot, where the paper
reports 70.0% against 72.9% with both branches tuned [arxiv:2208.02816 tab. 7].
All numbers are author-reported. The zero-shot and few-shot results cannot be
reproduced from the released code alone; see the card's Paper vs code section.

### Targeted attack on zero-shot matching

This variant is a security evaluation, not a method for recognising new
classes. Image-to-text matching in multimodal embedding models works by
zero-shot classification and retrieval against text, so an attacker can push
an image toward a chosen text concept [arxiv:2509.19994 §1]. The earlier
single-target attack matched one exact wording, and its fooling dropped for
close synonyms [arxiv:2509.19994 §1] [arxiv:2509.19994 fig. 1]. This paper
instead matches a set of target-side proxies for the concept, and adds a term
that keeps the adversarial embedding near embeddings of benign source-class
images [arxiv:2509.19994 §4.1] [arxiv:2509.19994 eq. 4]. The weight α trades off
generalisation against undetectability: a small α favours generalisation and
a large α favours undetectability [arxiv:2509.19994 §5.5]. Card: [Improving
Generalizability and Undetectability for Targeted Adversarial Attacks on
Multimodal Pre-trained Models](/papers/2509.19994).

On classification, the average success rate is 98.31% with no defence,
against 93.18% for Illusion Attack, and 84.17% under an anomaly-detection
defence, against 20.04% for Illusion Attack [arxiv:2509.19994 tab. 2]. On
retrieval the average R@1 success rate is 77.72% with no defence and 61.80%
with the defence [arxiv:2509.19994 tab. 3]. The main results use proxies and
targets from the same dataset. With cross-domain proxies, PTA's retrieval
success falls from 55.78% to 43.59%, and the paper says both methods degrade
under the shift [arxiv:2509.19994 tab. 21] [arxiv:2509.19994 §5.7.5]. The
anomaly detectors' settings and the proxy-generation prompts are not fully
stated, so the defence numbers cannot be reproduced from the text
[arxiv:2509.19994 §5.1] [arxiv:2509.19994 §5.7.1]. All numbers are
author-reported.

## When to use which

| Situation | Use | Why |
|---|---|---|
| You can train a network and describe each class with attributes, and classes arrive without examples | Attribute joint embedding | Unseen classes are scored from their attribute vectors [arxiv:1812.00420 §5]. Evidence covers only CUB and AWA [arxiv:1812.00420 §6]. |
| Sketch queries against a photo gallery, with class names that are unseen in training and word vectors for every class | Word-vector sketch retrieval | Photos are ranked by distance to the sketch, with the word vector as a training signal [arxiv:1904.03451 §4.2] [arxiv:1904.03451 §4.3]. Absolute scores are low on TU-Berlin and QuickDraw [arxiv:1904.03451 tab. 2]. |
| You have a CLIP model, a few labelled examples per class, and want a better starting point than a random head | Language-initialised linear head | 5-shot linear probing reaches 65.35% against 58.09% with a random head [arxiv:2204.08790 tab. 2]. Needs the text encoder and class names. |
| No training data, no training budget, and a capable instruction-following LLM is available | Prompted zero-shot | One prompt per item, no model training [arxiv:2302.06476 §3]. Accuracy varies widely by task [arxiv:2302.06476 tab. 11]. |
| Multi-step arithmetic or symbolic reasoning | Zero-shot chain-of-thought | Helps arithmetic [arxiv:2302.06476 tab. 3], but hurts commonsense QA [arxiv:2302.06476 tab. 3]. Check the target task first. |
| Medical multiple-choice questions, no training data, GPT-4 or GPT-3.5 access, and budget for many calls per question | Multi-agent prompting | Zero-shot GPT-4 averages 80.6%, MedAgents 86.7% [arxiv:2311.10537 tab. 2]. Costs about $1.41 per 100 questions [arxiv:2311.10537 §3.1]. Evidence covers nine multiple-choice sets only. |
| Mode choice or trip purpose from survey-style records, with few or no labelled samples and a hosted LLM API | Zero-shot prompting of structured trip data | GPT-3.5 reaches 0.586 mode accuracy with no examples, against 0.469 for a logit on 10 samples [arxiv:2312.00819 tab. 3]. With 1,000 samples a neural network reaches 0.635 [arxiv:2312.00819 tab. 3]. Evidence covers two survey datasets, and the card's prompt is fixed text [arxiv:2312.00819 §3.3]. |
| Few labelled trip samples, access to a text-embedding model, and a supervised head you can train | Embedding features with a supervised head | Embedding features raise the 10-sample logit from 0.469 to 0.494 on mode, a gain of 2.5 points against a printed spread of ±0.075 [arxiv:2312.00819 tab. 3]. Large-sample models still do best [arxiv:2312.00819 §4.4.1]. |
| Event triggers with unseen types, few keywords per type, and mention-level labels only for seen types | Keyword-seeded trigger detection with a generative argument extractor | Zero-seen-type trigger F1 is 55.19 [arxiv:2104.05919 tab. 8]. Argument results exist only for models trained on some types [arxiv:2104.05919 tab. 8]. The trigger code is not released [arxiv:2104.05919 §2.2]. |
| Named entity recognition, summarization or other tasks where fine-tuned models lead | Collect labels and fine-tune, or test zero-shot against your baseline first | The paper finds ChatGPT below GPT-3.5 on summarization and NER and below fine-tuned models in most cases [arxiv:2302.06476 §4.3]. |
| Pixel labels for class names chosen at test time, with image-caption data and a mask model available | Text-queried region matching | Class names are embedded and matched to region features, with no per-class training [arxiv:2112.12143 §3.4]. Training needs 64 TPU v3 cores [arxiv:2112.12143 §4.1]. |
| 3D shape from text, with image-shape training data but no text-shape pairs | Text-to-shape through CLIP features | Only images are used in training, and text features replace them at inference [arxiv:2110.02624 §3.3]. The paper does not measure how well the two feature types match [arxiv:2110.02624 §3.3]. |
| A few labelled images per class, a CLIP model, and no budget for fine-tuning | Training-free cache | Gains over zero-shot CLIP are small on ImageNet, 62.03% against 60.33% at 16 shots [arxiv:2111.03930 tab. 2]. If you can afford fine-tuning, CLIP-Adapter and Tip-Adapter-F score higher [arxiv:2111.03930 tab. 3]. |
| Image plus text edit as the query, CLIP ViT-L/14 available, and fewer than about 1,000 labelled triples | Composed retrieval with a pseudo token | Pic2Word is ahead of Combiner in that regime [arxiv:2302.03084 §4.3] [arxiv:2302.03084 fig. 7]. With more triples Combiner trained on CIRR scores higher on CIRR [arxiv:2302.03084 tab. 3]. |
| 3D object recognition with no 3D text labels, and compute for rendering and captioning every object | Tri-modal 3D pre-training | 50.6% zero-shot top-1 on Objaverse-LVIS [arxiv:2305.08275 tab. 1]. The triplet cost is high [arxiv:2305.08275 tab. 2] [arxiv:2305.08275 §4.1]. |
| Zero-shot on task types that training did not cover, with a labelled training mix of many prompted datasets and a large accelerator budget | Multitask prompted training | T0 (11B) matches or beats GPT-3 on 9 of 11 held-out datasets, with a different prompt protocol [arxiv:2110.08207 §6.1]. One run took about 27 TPU v3-512 hours [arxiv:2110.08207 app. B.1]. |
| Zero-shot image classification where you choose model and data size, with open image-text data | Scaling contrastive pre-training | The fitted ImageNet exponent is -0.11 for OpenCLIP [arxiv:2212.07143 §4.1]. The ViT-H/14 run took 279 hours on 824 GPUs [arxiv:2212.07143 tab. 18]. Larger-scale predictions are not measured [arxiv:2212.07143 tab. 4]. |
| Domain text tasks such as finance, with a closed LLM API and little or no labelled data, for a first estimate | Domain prompting, then a small check against your own labels | The reported GPT-4 scores have no baseline and no stated prompts or sample counts, so they only show what is possible [arxiv:2401.11641 tab. 2] [§4.2]. |
| A robot must follow a route given in free text, in a site it has already driven and mapped, with no robot-language data | Frozen LLM, CLIP and map graph search | 0.8 net success on 20 outdoor instructions [arxiv:2207.04429 tab. 1]. Needs a visual navigation model and a map drive first [arxiv:2207.04429 §3] [§5.2]. Evidence covers one robot and two outdoor sites. |
| A robot must follow new task commands in language, with teleoperated demonstrations of many tasks, and partial success on held-out tasks is acceptable | Language-conditioned visuomotor policy | Held-out language success averages 32% [arxiv:2202.02005 tab. 3] and ranges from 0% to 87% by task [arxiv:2202.02005 tab. 2]. Video commands reached 4% [arxiv:2202.02005 tab. 3], so do not rely on them. |
| Text or zero-shot classification on aerial or satellite images, a CLIP model, and budget for continued training on image-text pairs | Continued CLIP pretraining on remote sensing | +6.39 points over CLIP on average for ViT-B-32, across 12 datasets [arxiv:2306.11029 tab. 3]. Some datasets get worse, so test on yours [§4.2.3]. The largest model took 233.4 hours on four GPUs [§4.1.3]. |
| Video clips with action names not seen in training, a CLIP-style model, and budget for several frames per view | Cross-frame video encoder | Zero-shot UCF-101 is 72.0% against 58.3% for ActionCLIP, a different baseline [arxiv:2208.02816 tab. 3]. Multi-view testing adds 1.5 points on Kinetics-400 [arxiv:2208.02816 tab. 6], at extra FLOPs per view [arxiv:2208.02816 tab. 1]. |
| Security review of a zero-shot classifier or image search index, asking whether a targeted image can be made to match a chosen concept | Targeted attack with proxies, with anomaly detection as a candidate defence | Gives a stronger attack than the single-target baseline, and a defence to test against [arxiv:2509.19994 tab. 2] [arxiv:2509.19994 tab. 3]. Results depend on proxies drawn from the same dataset [arxiv:2509.19994 tab. 21]. |
| Energy or carbon per query is the binding constraint, and a task-specific model exists for the task | Task-specific model, or a zero-shot generative model only where the task is open-ended | On SciQ QA the best task-specific model emits about 0.3 g CO2eq per 1,000 inferences, against about 10 g for the multi-purpose models [arxiv:2311.16863 §4.2.1] [fig. 3]. Measured on A100 GPUs, one input at a time, in one region [arxiv:2311.16863 §3.2]. |
| A robot must find a named object in an unmapped building with a depth sensor and a hosted vision-language model, and can afford two model calls per step | Visual prompting for VLM navigation | ObjectNav success 50.4% against 24.6% for PIVOT, both with Gemini Flash [arxiv:2411.05755 tab. 1]. Evidence is simulation only, and success falls to 12.9% without wall sliding [arxiv:2411.05755 tab. 1]. Two calls run every step [arxiv:2411.05755 §3.5]. |
| An embodied task in simulation with a pretrained point-goal policy and detectors already available, and an LLM you can call yourself | LLM-written programs over skills | OVON success 35.5% against 35.2% for VLFM [arxiv:2412.10402 tab. 1]. The LLM step is not exercised by the released evaluator, so test your own LLM call first. |
| Tagging or classification across many languages, no labels in the target languages, and a chat-model API | Prompted zero-shot, checked against a supervised multilingual baseline | POS average 84.5% against 79.3% for XLM-R [arxiv:2304.05613 §4]. NER, NLI and QA trail supervised models by wide margins [arxiv:2304.05613 §5] [§7] [§8]. |
| Generation or reasoning in low-resource languages, where the reply must be usable | Test on your own data first, or use a model trained for the language | Summary replies were non-empty for 42% of Bengali items with English descriptions [arxiv:2304.05613 §10]. Commonsense scores fall to 24.5 for Urdu with a target-language description, against 34.7 with English [arxiv:2304.05613 §9]. |
| 3D object recognition with rendered views and category names, a frozen image-text encoder, and no 3D labels for the target classes | Frozen-encoder 3D alignment | 60.4% zero-shot top-1 for PointBERT with ULIP, against 20.2% for PointCLIP [arxiv:2212.05171 tab. 4]. Pre-training needs rendered views of every CAD model [arxiv:2212.05171 §3.1]. |
| Image classification over unseen classes with attribute or text descriptions, no unseen images, and a training budget for a generator | Generated pseudo-features | Generalised-setting harmonic mean 47.2 on AwA1 against 27.5 for the best baseline [arxiv:1709.00663 tab. 3]. The gain is small on CUB, where it leads by 0.1 points [arxiv:1709.00663 tab. 3]. |
| A causal LLM or other language model scores multiple-choice options, answers have many valid wordings, and you can write one short domain premise per task | Premise-normalised scoring (Domain Conditional PMI) | Best or tied on 62.5% of GPT-3 175B splits, against 18.75% for length-normalised likelihood [arxiv:2104.08315 tab. 2]. Needs a hand-written premise per task, and two model passes per option [arxiv:2104.08315 §3.2]. |
| Same task and model, a handful of hand-written prompts, and a need to choose one prompt without labels | Perplexity-ranked prompt selection | Average gain of 1.8 points on OPT and 2.3 on Bloom [arxiv:2212.04037 §7.1]. Can lose 12.5 points on a task such as AG News with Bloom [arxiv:2212.04037 tab. 10]. Evaluated on OPT and Bloom only [arxiv:2212.04037 §4.1]. |
| A frozen CLIP model, sentences as the only training data, and a need for more robust zero-shot or adversarial behaviour | Text-only adapter on CLIP (CLAP) | ViT-B/16 zero-shot 79.9% against 77.4% for CLIP [arxiv:2311.16445 tab. 3]. Training takes minutes on one GPU [arxiv:2311.16445 §5.1]. The α weight is tuned on test environments in the released code [arxiv:2311.16445 §5.1]. |
| Zero-shot retrieval or classification in one domain, a large image-text corpus for pretraining, and a non-commercial research setting | Two-tower pre-training on web pairs (BriVL) | UCM zero-shot 82.41% against 76.02% for CLIP [arxiv:2110.14378 fig. 4a]. The released code and weights are non-commercial and no-derivatives, so they cannot be used in a commercial product [arxiv:2110.14378 §Code availability]. |
| Latency, cost or throughput is the binding constraint | Check the cards before choosing | MedAgents reports about 40 seconds per question [arxiv:2311.10537 §3.1]. The program card reports timing only for its navigation policy [arxiv:2412.10402 §3]. The VLM navigation card gives no latency [arxiv:2411.05755 §3.5]. The segmentation card reports training cost only [arxiv:2112.12143 §4.1]. The attribute, prompting, shape, 3D, multitask, scaling and finance cards report no inference timing. The navigation card reports no execution latency [arxiv:2207.04429 §5.1]. The BC-Z card gives a 10 Hz control rate, with no inference latency [arxiv:2202.02005 §4]. The video card reports FLOPs per view, not time [arxiv:2208.02816 tab. 1]. The attack card reports training time per epoch only [arxiv:2509.19994 tab. 16]. The energy card reports energy, not latency [arxiv:2311.16863 tab. 2]. The travel card gives no token counts, prices or latency for its prompts [arxiv:2312.00819 §4]. The event card notes that a document is encoded once per event mention, so cost grows with the event count [arxiv:2104.05919 §4]. |

## Implementation sketch

Attribute route, scoring with the paper's joint embedding. The code in the
A-GEM repository adds a learned bias to these logits; the paper's equation
does not [arxiv:1812.00420 eq. 13].

```pseudocode
# Attribute-based zero-shot scoring  [arxiv:1812.00420 §5]
# Input:  image x, attribute matrix A (C x a) for the candidate classes,
#         feature extractor phi, attribute embedding W (a x D)
# Output: index of the predicted class

def zero_shot_attribute_predict(x, A, phi, W):
    # Map the input to a D-dimensional feature  [arxiv:1812.00420 §5]
    f = phi(x)                  # shape D
    # Each class embedding is a linear combination of its attributes  [§5]
    class_emb = A @ W           # shape C x D
    # Score each class by a dot product, as in eq. 13  [eq. 13]
    scores = class_emb @ f      # shape C
    return argmax(scores)
```

Prompted route, following the paper's two-stage prompt [arxiv:2302.06476 §3].
The paper does not state how the two parts are joined or what decoding settings
are used, so the separators here are an assumption.

```pseudocode
# Two-stage zero-shot chain-of-thought  [arxiv:2302.06476 §3]
# Input:  test input x, reasoning instruction p1, answer trigger p2
# Output: answer string

def zero_shot_cot(x, p1, p2):
    # Stage 1: elicit a free-text rationale  [§3]
    rationale = model(p1 + x)
    # Stage 2: feed the rationale back, then ask for the answer  [§3]
    return model(p1 + x + rationale + p2)
```

## Contradictions

No two papers report conflicting numbers for the same task. Within papers and
across comparisons:

- The CVAE paper's conclusion says it outperforms all baselines in the
  generalised setting on all four datasets [arxiv:1709.00663 §6]. Its CUB
  table gives 34.5 against 34.4 for ALE, a 0.1-point lead [arxiv:1709.00663
  tab. 3].
- Tip-Adapter's abstract calls the training-free variant comparable to
  CLIP-Adapter, but its ImageNet table shows it below CLIP-Adapter at every
  shot count [arxiv:2111.03930 §1] [arxiv:2111.03930 tab. 2].
- The shape paper's headline comparison pits a model trained on 13 classes
  against baselines trained on chairs and tables. Its appendix gives the
  chairs-and-tables result [arxiv:2110.02624 tab. 1] [arxiv:2110.02624 app. B].
- MedAgents states a temperature of 1.0 [arxiv:2311.10537 §3.1]. The released
  code uses a different value (see the card's Paper vs code section), and the
  paper does not say which setting produced its tables.
- Doodle to Search reports Sketchy mAP of 0.3691 on its own split, and 0.3521
  for its own model under the random 25-class partition that ZSIH used
  [arxiv:1904.03451 tab. 2]. ZSIH's TU-Berlin split is not reported by its
  authors [arxiv:1904.03451 tab. 2]. Compare across papers only under the same
  split.
- The finance survey's Table 2 gives 0.53 accuracy for stock movement, but the
  caption of its 1,470-point run gives 51% [arxiv:2401.11641 tab. 2]
  [arxiv:2401.11641 fig. 8]. The paper does not say whether these are the same
  run.
- The scaling paper says both exponents come from fits on the Pareto frontier
  of available models [arxiv:2212.07143 §4.1]. The released notebook fits the
  OpenAI CLIP points without binning, so the -0.11 and -0.16 values are not
  from one procedure (see the card's Paper vs code section).
- LM-Nav's text says 85% of experiments followed the instruction without
  collisions or disengagements [arxiv:2207.04429 §5.3]. Its Table 1 reports
  0.8 net success in both environments [arxiv:2207.04429 tab. 1]. The paper
  does not say how the two figures relate.
- RemoteCLIP's text says it is best on all three retrieval benchmarks
  [arxiv:2306.11029 §4.2.1]. On UCM, the FBCLM row reaches 61.60 mean recall,
  against 56.36 for the best RemoteCLIP row [arxiv:2306.11029 tab. 2]. Its
  ResNet-50 zero-shot average is 53.46 in the backbone table [arxiv:2306.11029
  tab. 3] but 55.06 in the pretraining ablation [arxiv:2306.11029 tab. 5].
- TANGO's abstract calls the method state-of-the-art. Its text says on par
  with VLFM on OVON and second on OpenEQA, and its tables show higher rows
  there [arxiv:2412.10402 §4.2] [arxiv:2412.10402 tab. 1] [arxiv:2412.10402
  tab. 2].
- VLMnav's ObjectNav success (50.4%, HM3D Habitat) and TANGO's OVON success
  (35.5%, HM3D-OVON val unseen) come from different benchmarks and target
  definitions, so they are not comparable [arxiv:2411.05755 tab. 1]
  [arxiv:2412.10402 tab. 1]. VLMnav's text gives a 1.2 m success radius, while
  its config uses 0.3 m, so its own numbers depend on which is used
  [arxiv:2411.05755 §4.1].
- BC-Z's text gives 29 held-out tasks [arxiv:2202.02005 §4] [arxiv:2202.02005
  §6.2]. Its table has 28 rows, and the overall row matches a mean over 28
  [arxiv:2202.02005 tab. 2]. This page uses the table's count.
- ChatGPT's multilingual paper gives 34 languages in its method and 37 in its
  introduction and conclusion [arxiv:2304.05613 §3] [arxiv:2304.05613 §1].
  Its claim that ChatGPT beats XLM-R on 13 of 17 POS languages [arxiv:2304.05613
  §4] needs the better of the two prompts per language. Counting the wins in
  the paper's POS table gives 12 with English descriptions and 9 with
  target-language ones.
- ULIP's ModelNet40 zero-shot number (60.4% top-1, all classes) and ULIP-2's
  (84.7%) come from different pre-training triplets. ULIP uses ShapeNet55
  category names, and ULIP-2 uses generated captions on Objaverse and ShapeNet
  [arxiv:2212.05171 §3.1] [arxiv:2305.08275 §4.1] [arxiv:2305.08275 tab. 1].
  Compare them only with care.

The cards use "zero-shot" for different setups: attribute descriptors on a
new task inside a continual-learning stream, instruction prompting of a
pretrained LLM, class names matched to pixel regions, CLIP features for
text-to-3D generation, a training-free cache on top of zero-shot CLIP,
multi-agent prompting for medical questions, unseen sketch categories matched
through word vectors, a language-initialised head for adaptation, composed
queries with a pseudo token, 3D classification with generated captions,
multitask prompted training, zero-shot ImageNet accuracy under scaling, GPT-4
prompting on finance tasks, landmark matching for robot navigation, VLM action choice for navigation, LLM-written programs over skills,
language-conditioned robot policies, and zero-shot classification after
remote-sensing pretraining, zero-shot action
recognition on video, targeted attacks on text-matched classifiers, analogy
puzzles with no examples, per-query energy of zero-shot models, category-name
3D alignment with a frozen image-text encoder, per-language prompting of
ChatGPT, keyword-seeded trigger detection with generated event arguments, and
zero-shot prompting for travel behaviour, generated pseudo-features for
unseen classes, premise-normalised scoring of LLM options, perplexity-ranked
prompt selection, a text-only adapter on CLIP features, and two-tower
pre-training on web pairs. They report
different metrics and tasks, so their numbers are not comparable.

## Papers

- [A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoders](/papers/1709.00663): a conditional VAE on image features and class embeddings that generates pseudo-features for unseen classes before an SVM is fitted.
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](/papers/2104.08315): Domain Conditional PMI for zero-shot multiple-choice scoring with GPT-2 and GPT-3, using a hand-written domain premise per task.
- [Demystifying Prompts in Language Models via Perplexity Estimation](/papers/2212.04037): low-perplexity zero-shot prompts for OPT and Bloom, and SPELL, which paraphrases and back-translates seeds and keeps the lowest-perplexity prompts.
- [Towards artificial general intelligence via a multimodal foundation model](/papers/2110.14378): BriVL, a two-tower image-text model with momentum queues, pretrained on weakly correlated Chinese web pairs and evaluated zero-shot on remote-sensing and news data.
- [CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts](/papers/2311.16445): a text-only residual MLP on frozen CLIP features, trained so that style-word-dropped prompts keep their content, then applied to images.
- [Efficient Lifelong Learning with A-GEM](/papers/1812.00420): attribute joint-embedding models and the LCA metric, which measures early accuracy on new tasks; the main contribution is continual learning.
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](/papers/1904.03451): sketch-to-photo retrieval for unseen categories, with word-vector and domain losses and the QuickDraw-Extended dataset.
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](/papers/2302.06476): zero-shot prompting of ChatGPT and GPT-3.5 across 20 datasets, with a two-stage chain-of-thought variant.
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](/papers/2304.05613): zero-shot ChatGPT on seven NLP tasks in 34 to 37 languages, with English and target-language task descriptions.
- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](/papers/2110.02624): text-to-3D-shape generation trained without text-shape pairs, using CLIP image features and a conditional flow.
- [Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling](/papers/2111.03930): a training-free key-value cache added to zero-shot CLIP logits, with an optional fine-tuned variant.
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](/papers/2112.12143): OpenSeg, which segments images for text-named classes using mask proposals trained with COCO masks and caption grounding.
- [ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models](/papers/2204.08790): a transfer benchmark for language-augmented models, with a text-initialised linear head and knowledge sources for class names.
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](/papers/2302.03084): zero-shot composed retrieval, mapping a reference image to one pseudo word token trained on unlabelled images.
- [ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding](/papers/2212.05171): a 3D point-cloud encoder aligned to frozen SLIP image and text encoders with category names, evaluated for zero-shot 3D classification.
- [ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding](/papers/2305.08275): tri-modal 3D pre-training with BLIP-2 captions ranked by CLIP, evaluated for zero-shot 3D classification.
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](/papers/2311.10537): zero-shot medical multiple-choice answering by role-played experts, a report, and voting rounds.
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](/papers/2110.08207): T5 fine-tuned on many prompted datasets, evaluated zero-shot on held-out task types.
- [Reproducible scaling laws for contrastive language-image learning](/papers/2212.07143): OpenCLIP scaling study with fitted power laws for zero-shot ImageNet accuracy and extrapolations.
- [Emergent Analogical Reasoning in Large Language Models](/papers/2212.09196): GPT-3 prompted with no examples on four analogy task families, compared with student samples.
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](/papers/2401.11641): a finance survey with a small GPT-4 zero- and one-shot evaluation on six datasets, without a baseline.
- [LM-Nav](/papers/2207.04429): frozen GPT-3, CLIP and ViNG route a ground robot along a text instruction over a mapped topological graph.
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](/papers/2202.02005): one imitation-learning policy for a real arm over 100 tasks, conditioned on a language or video embedding, tested on held-out tasks.
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](/papers/2306.11029): CLIP continually pretrained on remote-sensing pairs captioned from detection boxes and masks, with zero-shot results on 12 datasets.
- [Expanding Language-Image Pretrained Models for General Video Recognition](/papers/2208.02816): X-CLIP, a CLIP image encoder extended to video with cross-frame message tokens and video-specific text prompts, with zero-shot results on UCF-101 and unseen Kinetics-600 classes.
- [Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models](/papers/2509.19994): a targeted attack that matches several text proxies and benign source-image proxies, evaluated for zero-shot classification and retrieval against anomaly detectors.
- [End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering](/papers/2411.05755): a VLM picks one numbered arrow per step on depth-proposed directions, with a separate stop question, evaluated zero-shot on ObjectNav and GOAT.
- [TANGO: Training-free Embodied AI Agents for Open-world Tasks](/papers/2412.10402): an LLM writes programs over a pretrained point-goal policy, a memory-augmented value map and detectors, evaluated on OVON, GOAT-Bench and OpenEQA.
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](/papers/2311.16863): energy and CO2eq of 1,000 inferences for 88 models, comparing zero-shot multi-purpose models with task-specific ones.
- [Document-Level Event Argument Extraction by Conditional Generation](/papers/2104.05919): BART writes filled event templates from whole documents, with a keyword-seeded trigger model for unseen event types.
- [Large Language Models for Travel Behavior Prediction](/papers/2312.00819): zero-shot prompting of GPT-3.5 and other LLMs on structured trip data for mode choice and trip purpose, and LLM embeddings as features for small-sample supervised models.