Zero-shot learning

Overview

Zero-shot means the model answers for a class or task it has no labelled examples of. The thirty-one cards here share that goal and little else. One route describes every class by a vector of attributes and scores an image against each class description 1812.00420 §5. Another trains a conditional generator on seen classes, then samples feature vectors for unseen classes and fits a classifier on them 1709.00663 §3 1709.00663 alg. 1. Another maps each class name to a word vector, so an unseen category can be matched from a sketch 1904.03451 §4.2 1904.03451 §4.3. Another gives a pretrained language model a plain-language instruction and the input, with no demonstrations 2302.06476 §3. A third labels pixels with class names given at test time, using region features aligned to words 2112.12143 §3.2. Several rely on CLIP's shared image-text space: one generates 3D shapes from text without any text-shape pairs [arxiv:2110.02624 §3], one adds a training-free cache of few-shot examples on top of zero-shot CLIP 2111.03930 §3.2, one starts a trainable head from zero-shot class vectors 2204.08790 §4.2, one turns a reference image into a pseudo word for composed queries 2302.03084 §3.2, and one aligns 3D point clouds with CLIP-space images and captions 2305.08275 §3.3. Another keeps a SLIP image-text model frozen and aligns 3D point clouds to it using category names rather than generated captions 2212.05171 §3.2. The last runs several prompted LLM "experts" over a medical question with no examples 2311.10537 §2. Five more move the idea elsewhere. One fine-tunes a T5 model on many prompted datasets and then tests it on task types held out of training 2110.08207 §3. Another fits power laws to the zero-shot ImageNet accuracy of CLIP-style models trained on open image-text data 2212.07143 §4.1. A third keeps training CLIP on remote-sensing image-text pairs and reports zero-shot classification on 12 datasets 2306.11029 §4.2.3. A fourth chains a frozen LLM and CLIP to route a ground robot from a text instruction, with no robot-language training 2207.04429 §3. A fifth trains one robot policy on 100 manipulation tasks, conditioned on an embedding of a language command, and tests it on tasks held out of training 2202.02005 §3 2202.02005 §6.2. The last prompts GPT-4 on six finance datasets with no examples or one example 2401.11641 §4.

Two more cards use the idea in new settings. One adapts a CLIP-style image-text model to video, so a clip can be matched to action names it was not trained on 2208.02816 §1. The other is a security study: it attacks multimodal embedding models so that an image matches a target concept named only in part, and it tests whether the attack holds on unseen wordings 2509.19994 §1 2509.19994 §5.1.

Two more cards let a pretrained model act as a zero-shot embodied agent. One lets a vision-language model choose a numbered arrow at each step, and the arrows are drawn on the camera image from depth-based free directions 2411.05755 §3.1 2411.05755 §3.4. The other has an LLM write a short program that calls pretrained navigation, detection and question-answering modules 2412.10402 §3.

Three more cards study the zero-shot setting itself rather than a method. One asks whether GPT-3, given no examples, solves analogy puzzles built for students 2212.09196 §1. Another measures the energy of 1,000 inferences for zero-shot generative models and for task-specific models 2311.16863 §3.2. The third tests ChatGPT with one prompt per example, and no shown examples, on seven NLP tasks in 34 to 37 languages 2304.05613 §3 2304.05613 §1.

Two more cards apply the idea to text and transport data. One detects event triggers from a few keywords per type, with no mention-level labels for the new types, and then generates the arguments of each event as a filled template 2104.05919 §2.1 2104.05919 §2.2. The other prompts an LLM with a structured trip description and no examples, to predict travel mode or trip purpose 2312.00819 §3.2.

Four more cards change how a zero-shot answer is chosen or built. One scores each LLM answer option against a short task-specific premise, so that common wordings of one answer do not take probability from another 2104.08315 §3.2. Another ranks hand-written prompts, and their paraphrases and back-translations, by perplexity on task inputs, with no labels involved 2212.04037 §7. A third pretrains a two-tower image-text model on loosely matched Chinese web pairs, with no object detector, and reports zero-shot classification on remote-sensing and news data [arxiv:2110.14378 fig. 4a] 2110.14378 fig. 5a. The fourth trains a small residual network on frozen CLIP text features, using only sentences whose style words have been dropped, and then applies it to image features as well 2311.16445 §4.2.

The first card uses "0-shot" in a narrower sense: LCA_0 is the average accuracy on a new task before any training on it 1812.00420 §3. That measure comes from a continual-learning study, so its zero-shot numbers are secondary to its main contribution.

Variants

Attribute-based joint embedding

A feature extractor maps the input to a D-dimensional vector. Each class is represented by a linear combination of its attribute vector, so an unseen class gets an embedding without any image of it. Training uses cross-entropy over the task's classes, scored by the dot product of the two embeddings 1812.00420 §5 1812.00420 eq. 12 1812.00420 eq. 13. Card: Efficient Lifelong Learning with A-GEM.

The paper uses this model only on the two attribute datasets, Split CUB and Split AWA 1812.00420 §6. Attribute descriptors raise average accuracy A_T and LCA for every method it tests 1812.00420 §6.1. For example, A-GEM with attributes reaches 71% on CUB against 62% without, and 50% on AWA against 44% 1812.00420 tab. 5 1812.00420 tab. 6. Those are accuracies after the whole task sequence, not zero-shot numbers. Zero-shot accuracy of the attribute models rises as more tasks are learned, most for A-GEM 1812.00420 §6.1; the card gives this only in figures 1812.00420 fig. 4 1812.00420 fig. 5.

Generated pseudo-features for unseen classes

This route does not learn a mapping from descriptions to images. A conditional VAE models the distribution of image features given a class embedding, trained on seen classes only 1709.00663 §3. The encoder reads the feature concatenated with the embedding, and the decoder rebuilds the feature from a latent sample and the embedding 1709.00663 §3.1 1709.00663 §3.2. For each unseen class, latent samples from a standard normal prior are decoded with that class's embedding. The results are pseudo-training features for a linear SVM, with N = 300 samples per class and cost C = 100 [arxiv:1709.00663 alg. 1] 1709.00663 §5.3. Card: A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoders.

The paper's disjoint results are per-class accuracy on AwA1, AwA2, CUB and SUN. Its AwA1 score is 71.4 against 65.6 for SJE 1709.00663 tab. 2. The generalised setting, where seen and unseen classes are both tested, is where the gains are largest. There the harmonic mean is 47.2 on AwA1 and 51.2 on AwA2, against 27.5 and 27.8 for the best baselines 1709.00663 tab. 3. On ImageNet with 360 unseen classes the top-5 score is 24.7 against 16.8 1709.00663 tab. 4. The generated features are unimodal, which the authors note as a limit 1709.00663 fig. 3 1709.00663 §5.6. All numbers are author-reported.

Word-vector sketch retrieval

The task is zero-shot sketch-based image retrieval: a sketch query ranks photos from categories that do not appear in training [arxiv:1904.03451 §4.1]. Two VGG-16 encoders with attention map sketches and photos into one space 1904.03451 §4.2. Training combines a triplet ranking loss, a domain loss through a gradient-reversal layer, and a semantic loss that rebuilds each category's word vector from its embedding [arxiv:1904.03451 §4.3] 1904.03451 eq. 4. The word vectors are word2vec, with GloVe and fastText named as alternatives 1904.03451 §4.3. Card: Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval.

The paper adds QuickDraw-Extended, with 330,000 sketches and 204,000 photos 1904.03451 §3.1. Its full-gallery mAP is 0.3691 on Sketchy, 0.1094 on TU-Berlin and 0.0752 on QuickDraw 1904.03451 tab. 2. Against the triplet-only model on Sketchy, each added part raises mAP: attention, the domain loss and the semantic loss, reaching 0.3691 1904.03451 tab. 3. The QuickDraw score is low, and the authors say the problem is still unsolved on that set 1904.03451 §5.2. All numbers are author-reported.

Language-initialised linear head

CLIP is zero-shot on a new task by scoring the image against prompts built from class names. This variant keeps that idea as a starting point for adaptation. Each class column of a linear head is set to the averaged text embedding of its prompts, with bias zero 2204.08790 §4.2. The paper compares it with a random head in linear probing and full fine-tuning 2204.08790 tab. 2. Card: ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models.

The benchmark averages 20 image-classification datasets. CLIP ViT-B/32 scores 56.64% zero-shot, and 5-shot linear probing reaches 65.35% with the text initialisation against 58.09% with a random head 2204.08790 tab. 2. Full fine-tuning from a random head drops to 29.75% at 5 shots, against 63.29% with the text initialisation 2204.08790 tab. 2. With the text initialisation, full fine-tuning passes linear probing from about 20 shots 2204.08790 §5.1 2204.08790 tab. 2. The paper also tries knowledge text from WordNet, Wiktionary and GPT-3 for the class names. For a UniCL checkpoint, zero-shot accuracy rises from 27.15% to 33.95% with Wiktionary and GPT-3 together 2204.08790 fig. 5.

The authors' own IC protocol trains for 10 search epochs and 50 final epochs 2204.08790 §4.1. The released code reports the best per-epoch test accuracy in its final run, so the IC numbers are not fixed-epoch results; see the card's Pitfalls before comparing.

Prompted zero-shot LLM

The model receives one prompt: a task instruction followed by the test input, with no examples 2302.06476 §3. The paper compares ChatGPT (gpt-3.5-turbo) with GPT-3.5 (text-davinci-003) on 20 datasets in seven task categories 2302.06476 §4.1 2302.06476 tab. 1. Card: Is ChatGPT a General-Purpose Natural Language Processing Task Solver?.

Results depend strongly on the task. On arithmetic word problems ChatGPT improves on GSM8K from 59.5% to 78.9% 2302.06476 tab. 11. It also beats GPT-3.5 on several entailment and dialogue sets, and on SST-2 sentiment (93.7% against 88.8%) 2302.06476 tab. 4 2302.06476 tab. 12. It is weaker on summarization, where its mean ROUGE is 31.0 against 32.4 2302.06476 tab. 9, and on named entity recognition, where F1 is about equal at 53.2 against 53.5 but far below fine-tuned models [arxiv:2302.06476 tab. 10]. On the Coin Flip set it drops from 97.8% to 65.8% [arxiv:2302.06476 tab. 11].

For reasoning, the paper adds a two-stage zero-shot chain-of-thought step: a "think step by step" instruction first, then the rationale fed back with an answer trigger 2302.06476 §3. This helps arithmetic but lowers CommonsenseQA from 73.7% to 71.5% and StrategyQA from 61.1% to 55.5% 2302.06476 tab. 3.

The same no-example prompting is applied to analogy puzzles. GPT-3 (text-davinci-003) is given each puzzle as text, and the answer is either written by the model or chosen by its mean token log-probability over the options. The temperature is 0 with one run per item 2212.09196 §4.2. On Digit Matrices the model beats the average undergraduate, with odds ratio 1.88 for generated answers and 6.27 for multiple choice 2212.09196 §2.1. It matches or beats the human average on letter-string analogies, odds ratio 1.76 2212.09196 §2.2. On story analogies it is weaker than students, with odds ratio 0.37 2212.09196 §2.4. Card: Emergent Analogical Reasoning in Large Language Models. The numbers are author-reported, and the paper states no model snapshot date, so the results cannot be checked against current models 2212.09196 §4.2.

A domain case study comes from a finance survey, not a method paper. The authors put a role description, an output format and one example in the system message of GPT-4, then score six finance datasets 2401.11641 §4.2 tab. 2. The reported scores are 0.78 accuracy on Financial PhraseBank sentiment, 0.79 on FiQA-SA, 0.81 entity F1 for financial NER, 0.64 and 0.73 exact match on FinQA and ConvFinQA, and 0.53 accuracy on stock movement 2401.11641 tab. 2. No baseline is reported. The prompt text, model snapshot, decoding settings and per-dataset sample counts are not stated, and which of the paper's prompting strategies produced each score is unclear 2401.11641 §4.2 tab. 2. Card: Revolutionizing Finance with LLMs: An Overview of Applications and Insights.

Premise-normalised scoring of multiple-choice options

Here the model is not asked to choose; each option is scored and the highest score wins. The usual score, the raw probability of an option after the question, is penalised when an answer has many valid wordings, because the probability is split across them 2104.08315 §1 2104.08315 §2. Domain Conditional PMI divides the option's probability after the question by its probability after a short domain premise, such as "the answer is:". The premise is written by hand for each task 2104.08315 §3.2 [arxiv:2104.08315 tab. 7]. No model is trained. Card: Surface Form Competition: Why the Highest Probability Answer Isn't Always Right.

On GPT-3 175B the paper's table gives 89.2 on COPA, against 85.2 for raw probability and 82.8 for length-normalised likelihood 2104.08315 tab. 1. Across 16 splits and every model size, the method is best or tied on 62.5% of splits for GPT-3 175B, against 18.75% for length-normalised likelihood 2104.08315 tab. 2. The paper also reports that it loses on HellaSwag, ARC-Easy and BoolQ in some settings 2104.08315 §6. On COPA Flipped, where the answer string is fixed, the rules that do not correct for surface form give the same score as the corrected one 2104.08315 tab. 5. All numbers are author-reported.

Perplexity-ranked prompt selection

Prompt performance can differ a lot between paraphrases of one instruction. The paper's measure is the perplexity of the whole prompt under the model, averaged over about 1,000 task inputs, without the label 2212.04037 §2 2212.04037 §4.2. Across OPT and Bloom, prompts with lower perplexity usually score better, with negative correlations on most classification tasks 2212.04037 tab. 5. SPELL uses that finding as a selection rule. It paraphrases hand-written seeds with GPT-3, back-translates each through languages, ranks the candidates by mean perplexity, and keeps the three lowest 2212.04037 §3 2212.04037 §7. Card: Demystifying Prompts in Language Models via Perplexity Estimation.

The average gain over the manual prompts is 1.8 accuracy points for OPT and 2.3 for Bloom 2212.04037 §7.1. The per-task spread is wide. AG News drops by 12.5 points with Bloom, and DBpedia drops by 5.7 with OPT 2212.04037 tab. 10. The positive correlations on IMDB at OPT 175B and on Newspop at OPT 1.3B are not discussed in the text 2212.04037 tab. 5. The study uses two model families 2212.04037 §4.1 and English prompts only.

Zero-shot prompting of structured trip data

The prompt for a traveller has six parts, in this order: a task description that names the allowed options, travel time and cost per option as structured data, traveller attributes written as plain sentences, rules and comparison sentences about cost and time, and a JSON answer with a prediction and a reason 2312.00819 §3.3. It contains no training examples, and the model is called at temperature 0 2312.00819 §4.3. The same serialised text can instead be embedded, and a supervised head (multinomial logit, random forest or neural network) fitted on those vectors 2312.00819 §3.4. Card: Large Language Models for Travel Behavior Prediction.

On the Swissmetro mode-choice survey, zero-shot GPT-3.5 reaches 0.586 accuracy. That is above a multinomial logit trained on 10 samples (0.469), and above the same logit with embedding features (0.494). A neural network trained on 1,000 samples reaches 0.635 2312.00819 tab. 3. On the 2017 US travel survey for trip purpose, zero-shot GPT-3.5 reaches 0.402, against 0.313 for the 10-sample logit 2312.00819 tab. 3. GPT-4 and Llama 3.1 70B give 0.570 and 0.540 on mode, and 0.401 and 0.408 on purpose 2312.00819 tab. 3. All numbers are averages over five random samples of balanced 200-row test sets 2312.00819 §4.1. The authors read this as zero-shot LLMs beating small-sample supervised models, and as large-sample supervised models doing best 2312.00819 §4.4.1. Removing the step-by-step guidance from the prompt lowers GPT-3.5's mode accuracy by 5.8%, and GPT-4's by under 1% 2312.00819 §4.4.3. The ablation covers mode choice only.

Multilingual prompting with one prompt per example

The same no-example setup is tested across languages. The paper treats ChatGPT as a black box and builds one prompt per example: a task description, then the input 2304.05613 §3. The description is written in English or in the target language, and the model's reply is parsed with task-specific rules that the paper does not publish 2304.05613 §3. The seven tasks are POS tagging, NER, relation extraction, NLI, QA, commonsense reasoning and summarization, with languages grouped by their share of CommonCrawl 2304.05613 §3 2304.05613 §4-§10. Card: ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning.

Results depend on the task. With English descriptions, average POS accuracy is 84.5% against 79.3% for the supervised XLM-R baseline [arxiv:2304.05613 §4]. The gap is wide elsewhere: average NER macro-F1 is 29.9 against 88.4 for DAMO 2304.05613 §5, average NLI accuracy is 57.0 against 87.1 for mT5-XXL 2304.05613 §7, and average QA F1 is 53.5 against 85.2 2304.05613 §8. Commonsense reasoning is the one case where the target-language description has the higher average, 51.9 against 47.8, while the TRT baseline scores 59.0 2304.05613 §9. Summaries are often too long, and the share of non-empty replies is 42% for Bengali and 43% for Burmese with English descriptions 2304.05613 §10. The model snapshot is not named, and responses were gathered between 1 March and 5 April 2023 2304.05613 §3. Every number is author-reported.

Keyword-seeded event extraction with generated arguments

This route covers two steps for whole documents. Triggers, the words that mark an event, are tagged by a sequence labeller whose class vectors come from a few seed keywords per event type, so a type with no labelled mentions can still be detected 2104.05919 §2.2. Its zero-shot training uses pseudo-labels from similarity between token and class vectors, and tokens with uncertain similarity are left out of the loss 2104.05919 §2.2. For each trigger, a BART encoder-decoder then reads an event template together with the document and writes the template back with its slots filled. Its output is limited to words that appear in the input 2104.05919 §2.1 2104.05919 eq. 2. Card: Document-Level Event Argument Extraction by Conditional Generation.

On the ACE 2005 data, the keyword-based trigger model with no seen event types reaches 55.19 trigger identification F1 2104.05919 tab. 8. Paired with the argument generator trained on the 10 most frequent types, the pipeline reaches 72.31 trigger identification F1 and 48.18 argument identification Head F1. BERT-QA, trained on the same 10 types, reaches 39.37 argument identification Head F1 2104.05919 tab. 8. The supervised reference is 74.36 and 55.22 2104.05919 tab. 8. Argument rows are not given for the zero-shot setting, only for models trained on some types 2104.05919 tab. 8. On WikiEvents, the argument generator scores 64.57 argument classification Head F1 against 56.16 for BERT-QA 2104.05919 tab. 5. The paper's gains depend on which column is compared, so check the metric before quoting one. All numbers are author-reported. The released code contains the argument generator only, so the trigger model and its zero-shot training cannot be reproduced from it 2104.05919 §2.2.

Multitask prompted training

This variant trains the model, rather than only prompting it. The T0 model is a T5-style encoder-decoder trained by maximum likelihood to produce the target text 2110.08207 §5. Each training dataset is written as several prompt templates, and the training mixture draws examples from all of them 2110.08207 §4. Evaluation scores each answer option by its summed log-probability, with no length normalisation 2110.08207 §5. The held-out tests cover four task types, namely natural language inference, coreference, sentence completion and word sense disambiguation. Datasets of those types are excluded from the training mixture 2110.08207 §3. Card: Multitask Prompted Training Enables Zero-Shot Task Generalization.

On the 11 held-out datasets, the 11B model matches or exceeds every GPT-3 model on 9, with Winogrande and HellaSwag as the two exceptions 2110.08207 §6.1. Per-dataset medians are in 2110.08207 tab. 7; for example CB reaches 78.57% and RTE 81.23%. These are medians across all prompts, while GPT-3 is reported with one prompt per dataset, so the two are not like for like 2110.08207 §6.1. The paper selects the checkpoint by validation score on training datasets 2110.08207 §5; the released evaluation uses a fixed step, which the card's Paper vs code section records. Prompt count matters: moving from one prompt to 5.7 raises the median on 8 of 11 datasets 2110.08207 §6.2. The prompts themselves are in a separate collection and tool, so reproducing the training mixture needs both 2110.08207 §4 2110.08207 app. G. One T0 run took about 27 TPU v3-512 hours 2110.08207 app. B.1.

Multi-agent prompting for medical multiple choice

MedAgents needs no training. The model names question experts and option experts for each case, and each writes an analysis 2311.10537 §2.1 2311.10537 §2.2. The analyses are merged into one report 2311.10537 §2.3. Every expert votes YES or NO on the report, and a NO comes with a revision proposal; this repeats for up to five rounds 2311.10537 §2.4 2311.10537 alg. 1. A final prompt picks one option letter from the agreed report 2311.10537 §2.5. Card: MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning.

The paper reports 86.7% average accuracy for GPT-4 over nine datasets, against 80.6% for zero-shot GPT-4 and 85.4% for few-shot chain-of-thought with self-consistency 2311.10537 tab. 2. For GPT-3.5 the average is 72.1% against 67.8% zero-shot 2311.10537 tab. 2. The authors call the few-shot self-consistency baseline comparable, and the gap is small on average 2311.10537 §3.2. In a MedQA ablation with the model not stated, direct prompting scores 49.0% and the full pipeline 67.0% 2311.10537 tab. 3. The paper gives a cost of about $1.41 per 100 question-answer examples and about 40 seconds per example [arxiv:2311.10537 §3.1]. Its temperature is 1.0 for every call 2311.10537 §3.1. The released code uses a different setting; see the card's Paper vs code section.

Author-reported throughout. The authors' own error review of 40 sampled cases attributes most errors to missing or misapplied medical knowledge, with a smaller share to the rationale 2311.10537 §4.6.

Text-queried region matching (open-vocabulary segmentation)

OpenSeg represents an image as N = 128 mask proposals, each with its own feature vector 2112.12143 §3 2112.12143 app. E. The mask branch is trained on class-agnostic COCO masks with a Dice loss, and the alignment branch on image-caption pairs with a grounding loss 2112.12143 §3.1 2112.12143 §3.2. At inference, each class name is embedded by a frozen text encoder, compared with the region features, and combined with the masks to give per-pixel labels 2112.12143 §3.4. Card: Scaling Open-Vocabulary Image Segmentation with Image-Level Labels.

The paper calls its PASCAL-Context and PASCAL-20 results zero-shot evaluation 2112.12143 tab. 2. Adding Localized Narratives caption data raises PASCAL-20 mIoU from 60.0 to 63.8 with ResNet-101 2112.12143 tab. 2. The best row, with an EfficientNet-B7 backbone and prompt engineering, reaches 72.2 2112.12143 tab. 2. The ablation shows that inference with the predicted masks beats pixel-wise labelling without masks on PASCAL-Context (42.1 against 32.1 mIoU on PC-59) 2112.12143 tab. 4. These are author-reported numbers. The 72.2 row is compared with an LSeg score copied from its own paper, not re-run 2112.12143 §4.3 2112.12143 tab. 2.

Text-to-shape through CLIP features

The model is trained only on rendered views, their CLIP image features and shape data without text labels 2110.02624 §3. A voxel autoencoder squeezes each 32^3 grid into a 128-dimensional shape embedding 2110.02624 §3.1 2110.02624 §4. A conditional RealNVP flow then learns to produce those embeddings from the CLIP image features 2110.02624 §3.2. At test time the CLIP text features take the place of the image features. The flow is inverted from Gaussian noise, so one prompt can give several shapes and nothing is optimised per request 2110.02624 §3.3 2110.02624 §1. Card: CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation.

On 224 prompts over ShapeNet categories, the author-reported FID is 2425.25 and category accuracy is 83.33% 2110.02624 tab. 1. The two text2shape baselines score 14881.96 and 16078.05 FID, with 6.84% and 4.27% accuracy 2110.02624 tab. 1. Those baselines were trained on chairs and tables only, while the main model uses 13 classes. Trained on chairs and tables alone, the model scores 14746.90 FID and 30.77% accuracy 2110.02624 app. B. The prompt wording alone moves FID between about 2400 and 3140 2110.02624 tab. 2. The paper does not measure how well the text features match the image features the flow was trained on 2110.02624 §3.3.

Training-free cache on zero-shot CLIP

Zero-shot CLIP scores an image against text prompts built from class names. Tip-Adapter keeps a cache of CLIP features and labels from a few examples per class. For a test image, it finds how similar each cached example is, and adds the labels of similar examples to the zero-shot scores 2111.03930 §3.2 2111.03930 eq. 5 2111.03930 eq. 6. Its weights are set from the cache, so the cache needs no gradient steps 2111.03930 §3.2. An optional step fine-tunes the cache keys for 20 epochs, giving Tip-Adapter-F 2111.03930 §4.1. Card: Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling.

On ImageNet with ResNet-50, zero-shot CLIP scores 60.33% 2111.03930 tab. 2. Training-free Tip-Adapter scores 62.03% with 16 shots per class and 60.70% with one 2111.03930 tab. 2. CLIP-Adapter scores 63.59% at 16 shots, after 200 epochs that take 50 minutes on one GPU; Tip-Adapter-F scores 65.51% after 20 epochs and 5 minutes 2111.03930 tab. 3. The training-free variant is below CLIP-Adapter at every shot count in that table 2111.03930 tab. 2. The abstract calls the two comparable 2111.03930 §1. The paper's eq. 6 has no scale on the zero-shot term, while the card's Paper vs code section records that the released code adds one. Results on other datasets at 16 shots appear only in a figure, with zero-shot CLIP, Tip-Adapter and Tip-Adapter-F all reported 2111.03930 fig. 4.

Text-only adapter for style-invariant CLIP features

CLAP keeps CLIP frozen and learns one small network on top of it. The network is a residual two-layer MLP, and it starts as the identity because its last layer is zero-initialised 2311.16445 §4.1.2. Training uses text only. Each class has many template prompts, and each prompt gets a changed copy in which size, colour or order words are dropped or moved [arxiv:2311.16445 §4.2.1] 2311.16445 tab. 2. A contrastive loss pulls a prompt towards its changed copy, and the changed copy towards the bare class name, so the network learns to keep content and drop style 2311.16445 eq. 4. At inference the same network is applied to CLIP's image features and text features, with a scalar weight α on its residual branch 2311.16445 §4.1.3 2311.16445 app. A.1. Card: CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts.

Averaged over PACS, VLCS, OfficeHome and DomainNet, zero-shot top-1 with a bare class-name prompt is 79.9% for ViT-B/16, against 77.4% for CLIP and 76.1% for the image-augmented baseline 2311.16445 tab. 3. With a noised prompt the gap is larger: 76.1% against 70.4%, and the drop from the clean prompt falls from 7.0 to 3.8 points 2311.16445 tab. 3 [arxiv:2311.16445 tab. 4]. One-shot results under adversarial attacks are also higher, 29.5% against 22.0% 2311.16445 tab. 5. The α values come from the paper's appendix, and the card's Pitfalls section records that the released code picks α on the test environments. All numbers are author-reported.

Zero-shot classification at scale

The model is CLIP-style. Each class embedding is the mean of its prompt embeddings, normalised, and an image takes the class with the highest cosine similarity 2212.07143 §4.1. The paper trains OpenCLIP dual encoders with InfoNCE on LAION subsets, from ViT-B/32 to ViT-g/14, and varies model size, data size and samples seen 2212.07143 §3.2 tab. 18. It fits a line in log-log space to the lowest-error runs in each compute bin 2212.07143 §4.1. The fitted ImageNet zero-shot exponent is -0.11 for OpenCLIP on LAION and -0.16 for OpenAI CLIP 2212.07143 §4.1. Card: Reproducible scaling laws for contrastive language-image learning.

The best measured run, ViT-H/14 on LAION-2B with 34B samples seen (a nominal label), reaches 78.0% zero-shot ImageNet top-1, against 75.5% for the OpenAI ViT-L/14 baseline 2212.07143 tab. 1. Predictions beyond the measured runs, such as 81.92% for ViT-G/14 at 68B samples, are extrapolations and are not measured 2212.07143 tab. 4. The paper also reports linear-probe and fine-tuning gains with scale 2212.07143 §4.3 §4.4. Author-reported throughout.

Two-tower pre-training on loosely matched web pairs

BriVL trains an image encoder and a text encoder with a contrastive loss, over about 650M Chinese image-text pairs from the web [arxiv:2110.14378 §Results: Pre-Training Data Collection]. The pairs are weakly correlated, so a caption may only loosely relate to its picture. The image side is a CNN with multi-scale patch pooling and a small self-attention block, rather than an object detector 2110.14378 §Methods: Image Encoder. To get many negatives without a huge batch, it keeps two momentum-updated queues of recent keys 2110.14378 §Methods: Contrastive Loss. Card: Towards artificial general intelligence via a multimodal foundation model.

On zero-shot remote-sensing scene classification with UCM and 5 unseen classes against 16 seen ones, BriVL reaches 82.41%, against 76.02% for CLIP with ResNet-50x4 2110.14378 fig. 4a. On AID with 30 unseen classes it reaches 58.12% against 50.96% for the same CLIP model 2110.14378 fig. 4b. On Toutiao News, title-only zero-shot accuracy is 62.38% with RoBERTa-large, against 42.55% for RoBERTa-large alone 2110.14378 fig. 5a. The class names for BriVL were Chinese, while CLIP was given English names, so the comparison is confounded 2110.14378 §Pitfalls. The released model and inference code are licensed for non-commercial use without derivatives, as the paper states 2110.14378 §Code availability. Treat the code as unusable for a commercial product. All numbers are author-reported.

Composed retrieval with a pseudo word token

Composed image retrieval takes a reference image plus a text edit, such as "in watercolor", and ranks target images. Pic2Word needs no labelled triples of reference, edit and target 2302.03084 §1. A three-layer MLP maps the frozen CLIP ViT-L/14 image embedding to one pseudo token, which is placed in a prompt such as "a photo of [*], [text]" 2302.03084 §3.2 2302.03084 §3.3 2302.03084 §4.1. The mapper is trained on unlabelled CC3M images with a symmetric contrastive loss, and the CLIP encoders stay frozen 2302.03084 §3.2 2302.03084 §4.1. Card: Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval.

On the CIRR test split, Pic2Word scores 65.3 R@10, above TIRG (64.1) and ARTEMIS (61.3), and below Combiner trained on CIRR (73.2) [arxiv:2302.03084 tab. 3]. On Fashion-IQ validation it averages 24.7 R@10 [arxiv:2302.03084 tab. 4]. Text alone is a strong baseline on both sets: 56.7 R@10 on CIRR test and 17.3 on Fashion-IQ validation 2302.03084 tab. 3 [arxiv:2302.03084 tab. 4]. On COCO object composition Pic2Word reaches 33.4 R@10 against 32.2 for Combiner trained on CIRR 2302.03084 tab. 2. The paper reports Pic2Word ahead of Combiner when fewer than 1,000 labelled triples are available for training Combiner 2302.03084 §4.3 [arxiv:2302.03084 fig. 7]. The mapper's own variants change CIRR R@10 by up to about 7 points 2302.03084 app. B.

Tri-modal 3D pre-training with generated captions

ULIP-2 aligns three things: a 3D point cloud, rendered images of the same object, and text. The image and text encoders are a pretrained OpenCLIP ViT-G/14 model, which the paper treats as frozen 2305.08275 §3.3. Only the 3D encoder learns. Each rendered view gets ten candidate captions from BLIP-2, and the caption that CLIP ranks highest is kept [arxiv:2305.08275 §4.1]. Card: ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding.

For zero-shot 3D classification, the paper reports 50.6% top-1 on Objaverse-LVIS and 84.7% on ModelNet40 with the Point-BERT encoder, both with no manual captions 2305.08275 tab. 1. The table lists OpenShape at 46.8% top-1 on Objaverse-LVIS 2305.08275 tab. 1. On ModelNet40, a holistic BLIP-2 caption gives 69.7% top-1 against 60.4% for manual captions, in an ablation with ShapeNet pre-training 2305.08275 tab. 5. Both figures are author-reported.

Two caveats matter for use. The pre-training recipe is not fully stated: the paper gives no optimiser, learning rate or epoch count 2305.08275 §4.1. The paper does not say whether its "Objaverse + ShapeNet" pre-training includes the Objaverse-LVIS evaluation shapes 2305.08275 tab. 1. The scaling study trains on Objaverse without LVIS and reports 35.7% on Objaverse-LVIS for the chosen encoder size, so the two Objaverse-LVIS numbers come from different training sets 2305.08275 tab. 9. The triplets are costly: the Objaverse set alone has about 10 million images, each needing ten caption generations 2305.08275 tab. 2 2305.08275 §4.1. See the card's Paper vs code section for what the public release contains.

Frozen-encoder 3D alignment with category names

ULIP keeps the image and text encoders of SLIP frozen and trains only a 3D point-cloud encoder 2212.05171 §3.2. Each training object is a triplet of a rendered view, its category name and its point cloud, built from ShapeNet55 CAD models without manual captions 2212.05171 §3.1. The 3D feature is projected to a 512-dimensional space and trained with symmetric contrastive losses against the image and text features 2212.05171 §3.2 2212.05171 eq. 4. The paper's default weights turn the image-text term off, so the 3D encoder is pulled toward the text and image features 2212.05171 eq. 5. For zero-shot classification, each class name is filled into 64 prompts, the embeddings are averaged, and the object takes the nearest class 2212.05171 §4.5. Card: ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding.

On ModelNet40 with all 40 classes, PointBERT with ULIP reaches 60.4% top-1 against 20.2% for PointCLIP 2212.05171 tab. 4. On the Medium and Hard subsets it reaches 40.4% and 37.1%. Those subsets remove categories that also appear in the pre-training list 2212.05171 §4.5. On ScanObjectNN, zero-shot top-1 is 48.5% for PointBERT with ULIP 2212.05171 tab. 5. The two-modality ablation favours aligning both pairs: for PointBERT on ModelNet40, text only gives 44.7%, image only 35.5%, and both 60.4% 2212.05171 tab. 3. All numbers are author-reported. The paper does not say how its reported checkpoint is chosen; the card's Paper vs code section records that the released code keeps the best epoch on the ModelNet40 test split, which makes the headline numbers likely optimistic.

Pre-training starts from rendered views: each CAD model is rendered from 60 viewpoints 2212.05171 §3.1, and the frozen SLIP encoders run in every training step 2212.05171 §3.2. The paper describes averaging 64 prompts during pre-training as well as at test time 2212.05171 §3.1. The released code averages them only at test time, so the paper's pre-training recipe is not reproduced exactly; see the card's Paper vs code section.

Frozen LLM, CLIP and a map graph for navigation

LM-Nav trains nothing for the task. A frozen GPT-3 turns a route instruction into an ordered list of landmark descriptions 2207.04429 §4.2. CLIP scores each landmark against each node of a topological map of images collected on an earlier drive, using the prompt "This is a photo of" followed by the landmark 2207.04429 §4.3 2207.04429 §5.2. A dynamic program then picks the walk that visits the landmarks in order and penalises travel distance 2207.04429 §4.4 2207.04429 alg. 1. The robot follows that walk with a visual navigation model, ViNG, which predicts the temporal distance between two images 2207.04429 §3. Card: LM-Nav.

The paper reports 0.8 net success on 20 outdoor instructions, over more than 6 km, in two environments on a small wheeled robot 2207.04429 tab. 1 2207.04429 §5.2. A GPS-only baseline without the navigation model reaches 0.23 in one environment 2207.04429 tab. 1. The planner beats a max-likelihood baseline on the larger environment, 0.8 against 0.2 success 2207.04429 tab. 4. The main failure is landmark detection: a fire hydrant or cement mixer may go unseen 2207.04429 §5.2 [arxiv:2207.04429 fig. 4]. The language and vision models ran on a remote workstation, and the paper reports no latency during execution 2207.04429 §5.1. All numbers are author-reported.

Visual prompting for VLM navigation

The vision-language model is not fine-tuned. At each step, a depth-based test finds directions the robot can drive in without hitting a wall, and each direction is drawn as a numbered arrow on the RGB image 2411.05755 §3.1 2411.05755 §3.3. Candidate arrows are spaced so that unexplored directions are favoured 2411.05755 §3.2. One model call picks an arrow number, and a separate call on the un-annotated image decides when to stop 2411.05755 §3.4 2411.05755 §3.5. Card: End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering.

On Habitat ObjectNav with Gemini Flash, the paper reports 50.4% success and 0.210 SPL. PIVOT scores 24.6% and 0.106, and the same model without the navigability step scores 33.2% 2411.05755 tab. 1. Turning off wall sliding drops success to 12.9% 2411.05755 tab. 1. The navigability step ignores the robot's size and shape, and the authors say the method should not be assumed to transfer to a real robot without a collision-aware controller 2411.05755 §4.1 2411.05755 §5. On GOAT, success is 16.3% with sliding on, against 8.3% for PIVOT, but below two specialised systems at 29.5% and 24.9% 2411.05755 tab. 2 2411.05755 tab. 3. All numbers are author-reported, and each step costs two model calls.

LLM-written programs over pretrained skills

The paper treats an LLM as a planner. It receives a task instruction and a fixed prompt of 15 worked examples, and writes a short program with one line per module call 2412.10402 §3. The modules are a pretrained point-goal navigation policy, a value map that stores a feature per pixel so that targets seen earlier can be revisited, and object detectors with a CLIP-based filter 2412.10402 §3 2412.10402 §4.1. The planner is GPT-4o 2412.10402 §3. No module is trained for the target tasks 2412.10402 §4.1. Card: TANGO: Training-free Embodied AI Agents for Open-world Tasks.

On HM3D-OVON val unseen, the paper reports 35.5% success and 19.5 SPL, against 35.2% and 19.6 for VLFM 2412.10402 tab. 1. On OpenEQA the score is 37.2, against 38.1 for a Socratic-LLM baseline with frame captions 2412.10402 tab. 2. On GOAT-Bench it reaches 32.1% success against 29.5% for the best prior success row, with SPL 16.5 against 17.2 for Modular GOAT 2412.10402 tab. 3. The text calls the method on par with VLFM on OVON and second on OpenEQA 2412.10402 §4.2. The released evaluator does not call the LLM; it runs fixed programs per task, so its numbers do not test the planning step. See the card's Paper vs code section before using it.

Language-conditioned visuomotor policy

BC-Z is imitation learning for a real arm. One policy takes a camera image and a task embedding, and outputs motions of the end effector [arxiv:2202.02005 §3] 2202.02005 §5.2. The embedding comes from a pretrained sentence encoder, with 512 dimensions, and it conditions a ResNet-18 through FiLM layers in all four blocks 2202.02005 §5.1 2202.02005 §5.3. The training data are demonstrations of 100 tasks, collected with a VR rig at 10 Hz. Some of the data are human corrections made while a learned policy ran, which the paper calls HG-DAgger 2202.02005 §4. Zero-shot means the policy is given a command for a task that has no demonstrations in training 2202.02005 §1. Each action is the offset from the current state to a pose N steps ahead. The policy predicts ten future actions but executes only the first 2202.02005 §5.2. Card: BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning.

On 28 held-out task rows, the language command reaches 32% average success, and the video command 4% 2202.02005 tab. 3. Per-task success in the one-distractor column runs from 0% to 87%, and its overall row is 38%; the four-to-five-distractor overall row is 32% 2202.02005 tab. 2. With 21 training tasks, language conditioning scores 40% and one-hot task IDs 42% 2202.02005 tab. 3. The authors read the near-equal numbers as evidence that the language space is adequate, so the held-out gap sits in the policy 2202.02005 §6.2. A single-task policy trained on more than 300 demonstrations of each of four held-out tasks scores 0% on all four 2202.02005 §6.2. The numbers come from one real robot setup with 12 robots, 7 operators and 125 hours of robot time 2202.02005 §1. All numbers are author-reported. The released code covers only the language-conditioned model, and its training settings differ from the paper's; see the card's Paper vs code section.

Continued CLIP pretraining on remote-sensing pairs

This variant trains the model. It starts from CLIP and keeps training with the symmetric contrastive loss on aerial and satellite image-text pairs 2306.11029 §3.1 2306.11029 eq. 1. The text is generated from existing labels. Box-to-caption writes five captions for each image from its object boxes, and mask-to-box turns segmentation masks into boxes first 2306.11029 §3.2.1 2306.11029 §3.2.2. The training set has 165,745 images, each with five captions, so 828,725 pairs [arxiv:2306.11029 §4.1.2]. Zero-shot classification fills the prompt "a satellite photo of {class name}." 2306.11029 §4.2.3. Card: RemoteCLIP: A Vision Language Foundation Model for Remote Sensing.

Averaged over 12 classification datasets, zero-shot accuracy rises from 56.02% to 62.41% for ViT-B-32, from 65.67% to 71.30% for ViT-L-14, and from 50.61% to 53.46% for ResNet-50 2306.11029 tab. 3. The ViT-L-14 model beats CLIP on 9 of 12 datasets 2306.11029 §4.2.3. The gain is not uniform: the largest drops are on EuroSAT, at −11.15 points for ViT-B-32 and −24.83 for ResNet-50 2306.11029 tab. 3. The authors attribute the drops to input resolution and image distribution 2306.11029 §4.2.3. The largest model took 233.4 hours on four GPUs 2306.11029 §4.1.3. Author-reported throughout.

Cross-frame video encoder with video-specific prompts

X-CLIP reuses a pretrained CLIP image encoder for video, and does not add video-text pretraining 2208.02816 §1. Each frame's [class] token gives a message token, and the message tokens exchange information across frames inside every transformer block 2208.02816 §3.2.1. A small temporal transformer then merges the frame vectors into one video vector 2208.02816 §3.2. On the text side the class name is embedded by the CLIP text encoder, and a prompt generator adds a term built from the video's patch tokens 2208.02816 §3.3.1. A video and a class are scored by cosine similarity 2208.02816 eq. 3. Card: Expanding Language-Image Pretrained Models for General Video Recognition.

The fully supervised result on Kinetics-400 is 87.1% top-1 for ViT-L/14 at 8 frames, against 84.8% for ViViT-H/16×2 2208.02816 tab. 1. In the zero-shot setting, trained on Kinetics-400 with 32 frames and one view, X-CLIP-B/16 scores 72.0% on UCF-101 against 58.3% for ActionCLIP, and 65.2% on 220 unseen Kinetics-600 classes against 42.1% for ER-ZSAR 2208.02816 tab. 3 2208.02816 tab. 4. The unseen-class score averages draws of 160 of those 220 classes 2208.02816 app. C. In the paper's own ablation, video-specific prompts raise zero-shot Kinetics-400 accuracy from 63.9% to 70.0% over a CLIP prompt ensemble [arxiv:2208.02816 tab. 9]. Freezing the text branch costs points in zero-shot, where the paper reports 70.0% against 72.9% with both branches tuned 2208.02816 tab. 7. All numbers are author-reported. The zero-shot and few-shot results cannot be reproduced from the released code alone; see the card's Paper vs code section.

Targeted attack on zero-shot matching

This variant is a security evaluation, not a method for recognising new classes. Image-to-text matching in multimodal embedding models works by zero-shot classification and retrieval against text, so an attacker can push an image toward a chosen text concept 2509.19994 §1. The earlier single-target attack matched one exact wording, and its fooling dropped for close synonyms 2509.19994 §1 2509.19994 fig. 1. This paper instead matches a set of target-side proxies for the concept, and adds a term that keeps the adversarial embedding near embeddings of benign source-class images 2509.19994 §4.1 2509.19994 eq. 4. The weight α trades off generalisation against undetectability: a small α favours generalisation and a large α favours undetectability 2509.19994 §5.5. Card: Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models.

On classification, the average success rate is 98.31% with no defence, against 93.18% for Illusion Attack, and 84.17% under an anomaly-detection defence, against 20.04% for Illusion Attack 2509.19994 tab. 2. On retrieval the average R@1 success rate is 77.72% with no defence and 61.80% with the defence 2509.19994 tab. 3. The main results use proxies and targets from the same dataset. With cross-domain proxies, PTA's retrieval success falls from 55.78% to 43.59%, and the paper says both methods degrade under the shift 2509.19994 tab. 21 2509.19994 §5.7.5. The anomaly detectors' settings and the proxy-generation prompts are not fully stated, so the defence numbers cannot be reproduced from the text 2509.19994 §5.1 2509.19994 §5.7.1. All numbers are author-reported.

When to use which

Implementation sketch

Attribute route, scoring with the paper's joint embedding. The code in the A-GEM repository adds a learned bias to these logits; the paper's equation does not 1812.00420 eq. 13.

# Attribute-based zero-shot scoring  [arxiv:1812.00420 §5]
# Input:  image x, attribute matrix A (C x a) for the candidate classes,
#         feature extractor phi, attribute embedding W (a x D)
# Output: index of the predicted class

def zero_shot_attribute_predict(x, A, phi, W):
    # Map the input to a D-dimensional feature  [arxiv:1812.00420 §5]
    f = phi(x)                  # shape D
    # Each class embedding is a linear combination of its attributes  [§5]
    class_emb = A @ W           # shape C x D
    # Score each class by a dot product, as in eq. 13  [eq. 13]
    scores = class_emb @ f      # shape C
    return argmax(scores)

Prompted route, following the paper's two-stage prompt 2302.06476 §3. The paper does not state how the two parts are joined or what decoding settings are used, so the separators here are an assumption.

# Two-stage zero-shot chain-of-thought  [arxiv:2302.06476 §3]
# Input:  test input x, reasoning instruction p1, answer trigger p2
# Output: answer string

def zero_shot_cot(x, p1, p2):
    # Stage 1: elicit a free-text rationale  [§3]
    rationale = model(p1 + x)
    # Stage 2: feed the rationale back, then ask for the answer  [§3]
    return model(p1 + x + rationale + p2)

Contradictions

No two papers report conflicting numbers for the same task. Within papers and across comparisons:

• The CVAE paper's conclusion says it outperforms all baselines in the generalised setting on all four datasets 1709.00663 §6. Its CUB table gives 34.5 against 34.4 for ALE, a 0.1-point lead [arxiv:1709.00663 tab. 3].
• Tip-Adapter's abstract calls the training-free variant comparable to CLIP-Adapter, but its ImageNet table shows it below CLIP-Adapter at every shot count 2111.03930 §1 2111.03930 tab. 2.
• The shape paper's headline comparison pits a model trained on 13 classes against baselines trained on chairs and tables. Its appendix gives the chairs-and-tables result 2110.02624 tab. 1 2110.02624 app. B.
• MedAgents states a temperature of 1.0 2311.10537 §3.1. The released code uses a different value (see the card's Paper vs code section), and the paper does not say which setting produced its tables.
• Doodle to Search reports Sketchy mAP of 0.3691 on its own split, and 0.3521 for its own model under the random 25-class partition that ZSIH used 1904.03451 tab. 2. ZSIH's TU-Berlin split is not reported by its authors 1904.03451 tab. 2. Compare across papers only under the same split.
• The finance survey's Table 2 gives 0.53 accuracy for stock movement, but the caption of its 1,470-point run gives 51% 2401.11641 tab. 2 2401.11641 fig. 8. The paper does not say whether these are the same run.
• The scaling paper says both exponents come from fits on the Pareto frontier of available models 2212.07143 §4.1. The released notebook fits the OpenAI CLIP points without binning, so the -0.11 and -0.16 values are not from one procedure (see the card's Paper vs code section).
• LM-Nav's text says 85% of experiments followed the instruction without collisions or disengagements 2207.04429 §5.3. Its Table 1 reports 0.8 net success in both environments 2207.04429 tab. 1. The paper does not say how the two figures relate.
• RemoteCLIP's text says it is best on all three retrieval benchmarks 2306.11029 §4.2.1. On UCM, the FBCLM row reaches 61.60 mean recall, against 56.36 for the best RemoteCLIP row 2306.11029 tab. 2. Its ResNet-50 zero-shot average is 53.46 in the backbone table [arxiv:2306.11029 tab. 3] but 55.06 in the pretraining ablation 2306.11029 tab. 5.
• TANGO's abstract calls the method state-of-the-art. Its text says on par with VLFM on OVON and second on OpenEQA, and its tables show higher rows there 2412.10402 §4.2 2412.10402 tab. 1 [arxiv:2412.10402 tab. 2].
• VLMnav's ObjectNav success (50.4%, HM3D Habitat) and TANGO's OVON success (35.5%, HM3D-OVON val unseen) come from different benchmarks and target definitions, so they are not comparable 2411.05755 tab. 1 2412.10402 tab. 1. VLMnav's text gives a 1.2 m success radius, while its config uses 0.3 m, so its own numbers depend on which is used 2411.05755 §4.1.
• BC-Z's text gives 29 held-out tasks 2202.02005 §4 [arxiv:2202.02005 §6.2]. Its table has 28 rows, and the overall row matches a mean over 28 2202.02005 tab. 2. This page uses the table's count.
• ChatGPT's multilingual paper gives 34 languages in its method and 37 in its introduction and conclusion 2304.05613 §3 2304.05613 §1. Its claim that ChatGPT beats XLM-R on 13 of 17 POS languages [arxiv:2304.05613 §4] needs the better of the two prompts per language. Counting the wins in the paper's POS table gives 12 with English descriptions and 9 with target-language ones.
• ULIP's ModelNet40 zero-shot number (60.4% top-1, all classes) and ULIP-2's (84.7%) come from different pre-training triplets. ULIP uses ShapeNet55 category names, and ULIP-2 uses generated captions on Objaverse and ShapeNet 2212.05171 §3.1 2305.08275 §4.1 2305.08275 tab. 1. Compare them only with care.

The cards use "zero-shot" for different setups: attribute descriptors on a new task inside a continual-learning stream, instruction prompting of a pretrained LLM, class names matched to pixel regions, CLIP features for text-to-3D generation, a training-free cache on top of zero-shot CLIP, multi-agent prompting for medical questions, unseen sketch categories matched through word vectors, a language-initialised head for adaptation, composed queries with a pseudo token, 3D classification with generated captions, multitask prompted training, zero-shot ImageNet accuracy under scaling, GPT-4 prompting on finance tasks, landmark matching for robot navigation, VLM action choice for navigation, LLM-written programs over skills, language-conditioned robot policies, and zero-shot classification after remote-sensing pretraining, zero-shot action recognition on video, targeted attacks on text-matched classifiers, analogy puzzles with no examples, per-query energy of zero-shot models, category-name 3D alignment with a frozen image-text encoder, per-language prompting of ChatGPT, keyword-seeded trigger detection with generated event arguments, and zero-shot prompting for travel behaviour, generated pseudo-features for unseen classes, premise-normalised scoring of LLM options, perplexity-ranked prompt selection, a text-only adapter on CLIP features, and two-tower pre-training on web pairs. They report different metrics and tasks, so their numbers are not comparable.

Papers

• A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoders: a conditional VAE on image features and class embeddings that generates pseudo-features for unseen classes before an SVM is fitted.
• Surface Form Competition: Why the Highest Probability Answer Isn't Always Right: Domain Conditional PMI for zero-shot multiple-choice scoring with GPT-2 and GPT-3, using a hand-written domain premise per task.
• Demystifying Prompts in Language Models via Perplexity Estimation: low-perplexity zero-shot prompts for OPT and Bloom, and SPELL, which paraphrases and back-translates seeds and keeps the lowest-perplexity prompts.
• Towards artificial general intelligence via a multimodal foundation model: BriVL, a two-tower image-text model with momentum queues, pretrained on weakly correlated Chinese web pairs and evaluated zero-shot on remote-sensing and news data.
• CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts: a text-only residual MLP on frozen CLIP features, trained so that style-word-dropped prompts keep their content, then applied to images.
• Efficient Lifelong Learning with A-GEM: attribute joint-embedding models and the LCA metric, which measures early accuracy on new tasks; the main contribution is continual learning.
• Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval: sketch-to-photo retrieval for unseen categories, with word-vector and domain losses and the QuickDraw-Extended dataset.
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver?: zero-shot prompting of ChatGPT and GPT-3.5 across 20 datasets, with a two-stage chain-of-thought variant.
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning: zero-shot ChatGPT on seven NLP tasks in 34 to 37 languages, with English and target-language task descriptions.
• CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation: text-to-3D-shape generation trained without text-shape pairs, using CLIP image features and a conditional flow.
• Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling: a training-free key-value cache added to zero-shot CLIP logits, with an optional fine-tuned variant.
• Scaling Open-Vocabulary Image Segmentation with Image-Level Labels: OpenSeg, which segments images for text-named classes using mask proposals trained with COCO masks and caption grounding.
• ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models: a transfer benchmark for language-augmented models, with a text-initialised linear head and knowledge sources for class names.
• Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval: zero-shot composed retrieval, mapping a reference image to one pseudo word token trained on unlabelled images.
• ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding: a 3D point-cloud encoder aligned to frozen SLIP image and text encoders with category names, evaluated for zero-shot 3D classification.
• ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding: tri-modal 3D pre-training with BLIP-2 captions ranked by CLIP, evaluated for zero-shot 3D classification.
• MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning: zero-shot medical multiple-choice answering by role-played experts, a report, and voting rounds.
• Multitask Prompted Training Enables Zero-Shot Task Generalization: T5 fine-tuned on many prompted datasets, evaluated zero-shot on held-out task types.
• Reproducible scaling laws for contrastive language-image learning: OpenCLIP scaling study with fitted power laws for zero-shot ImageNet accuracy and extrapolations.
• Emergent Analogical Reasoning in Large Language Models: GPT-3 prompted with no examples on four analogy task families, compared with student samples.
• Revolutionizing Finance with LLMs: An Overview of Applications and Insights: a finance survey with a small GPT-4 zero- and one-shot evaluation on six datasets, without a baseline.
• LM-Nav: frozen GPT-3, CLIP and ViNG route a ground robot along a text instruction over a mapped topological graph.
• BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning: one imitation-learning policy for a real arm over 100 tasks, conditioned on a language or video embedding, tested on held-out tasks.
• RemoteCLIP: A Vision Language Foundation Model for Remote Sensing: CLIP continually pretrained on remote-sensing pairs captioned from detection boxes and masks, with zero-shot results on 12 datasets.
• Expanding Language-Image Pretrained Models for General Video Recognition: X-CLIP, a CLIP image encoder extended to video with cross-frame message tokens and video-specific text prompts, with zero-shot results on UCF-101 and unseen Kinetics-600 classes.
• Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models: a targeted attack that matches several text proxies and benign source-image proxies, evaluated for zero-shot classification and retrieval against anomaly detectors.
• End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering: a VLM picks one numbered arrow per step on depth-proposed directions, with a separate stop question, evaluated zero-shot on ObjectNav and GOAT.
• TANGO: Training-free Embodied AI Agents for Open-world Tasks: an LLM writes programs over a pretrained point-goal policy, a memory-augmented value map and detectors, evaluated on OVON, GOAT-Bench and OpenEQA.
• Power Hungry Processing: Watts Driving the Cost of AI Deployment?: energy and CO2eq of 1,000 inferences for 88 models, comparing zero-shot multi-purpose models with task-specific ones.
• Document-Level Event Argument Extraction by Conditional Generation: BART writes filled event templates from whole documents, with a keyword-seeded trigger model for unseen event types.
• Large Language Models for Travel Behavior Prediction: zero-shot prompting of GPT-3.5 and other LLMs on structured trip data for mode choice and trip purpose, and LLM embeddings as features for small-sample supervised models.