---
kind: concept
slug: text-classification
title: Text classification
summary: >-
  Assigning one label to a text or sentence pair. Narrow tasks: linear models on TF-IDF, FastText and CNNs stay strong. Few labels: a contrastively tuned sentence encoder with a logistic-regression head nears prompt methods. Cross-domain transfer is weak, and dataset artifacts or altered input distort scores. Zero-shot LLM labels vary widely by task and need checks.
tags: [nlp, classification, few-shot, robustness, multi-task, domain-shift, annotation, label-noise, multilingual]
papers: ["1603.03827", "1610.08914", "1704.06841", "1803.02324", "1806.03713", "1806.05258", "1901.08014", "1904.08067", "1907.11932", "2004.01970", "2005.07503", "2006.03541", "2007.13184", "2102.00225", "2110.01852", "2111.02840", "2202.11176", "2209.11055", "2303.15056", "2310.17389", "2311.16863", "2405.11619", "2410.12985", "2606.25495"]
related: [few-shot-learning, contrastive-learning, adversarial-robustness, adversarial-text-generation, benchmark-validity, challenge-set-evaluation, language-model-pretraining, masked-language-modeling, moderation-analysis, multi-task-learning, sentiment-analysis, text-embedding, support-vector-machine, convolutional-neural-network, recurrent-neural-network, attention-mechanism, domain-adaptation, dataset-curation, explainable-ai, random-forest, llm-as-a-judge, noisy-labels, prompt-engineering, toxicity-detection, sequence-labeling, word-embedding, llm-serving, multilingual-evaluation, llm-evaluation, neural-machine-translation, data-augmentation, subword-tokenization]
status: draft
updated_at: 2026-11-01
---

## Overview

Text classification gives a text one label, or a sentence pair one label. The
twenty-four cards on this page cover a wide range of inputs. They include personal
attacks in English Wikipedia talk comments [arxiv:1610.08914 §3], medical
sentences in 26 categories [arxiv:1704.06841 §3], mental health conditions
inferred from a user's Reddit posts [arxiv:1806.05258 §3.1], rumour veracity on
tweet reply branches [arxiv:1806.03713 §3], sentiment and sarcasm in short
sentences [arxiv:1901.08014 §2], positive and negative tweets and reviews
[arxiv:2006.03541 §4.1], toxicity in chatbot prompts [arxiv:2310.17389 §2],
phishing and spam emails [arxiv:2405.11619 §2], natural language inference
hypotheses [arxiv:1803.02324 §2], five GLUE sentence tasks
[arxiv:2111.02840 §3.1], few-shot tasks such as sentiment, emotion and spam
[arxiv:2209.11055 tab. 2], industrial labels from a private dataset
[arxiv:2102.00225 §5], and text-annotation tasks on political and
content-moderation samples [arxiv:2303.15056 §4.1], and dialog acts for each
utterance in a sequence of dialogs [arxiv:1603.03827 §3.1]. Two newer cards add
COVID-19 tweets in five classification sets [arxiv:2005.07503 tab. 1] and
Arabic tweets for five-class sentiment and spam [arxiv:2606.25495 §5.1]. Two
more cover offensive tweets in Arabic, Greek and Turkish [arxiv:2007.13184 tab. 2],
and depression severity in English and Greek Reddit posts [arxiv:2410.12985 tab. 2].
A survey of text data augmentation [arxiv:2110.01852 §2] and a byte-level
multilingual toxicity encoder [arxiv:2202.11176 tab. 1] complete the set.

Three questions come up again and again. The first is how many labelled
examples a method needs. The second is whether a simple model is good enough:
two cards report that a linear model on TF-IDF or a FastText model is among the
strongest options on its own data. The third is whether a model that does well
on its training domain still does well on another one. The chatbot-prompt card
reports that detectors trained on other toxicity data score low on chatbot
traffic [arxiv:2310.17389 tab. 1] [arxiv:2310.17389 tab. 2]. The adversarial benchmark asks a related
question about altered input: a changed example is only a fair test if its
label stays the same, so it keeps an example only when annotators agree on the
original label [arxiv:2111.02840 §3.3]. A fourth question is where the labels
come from. Two cards on this page study label quality directly: one shows that
an input-only classifier can beat chance on a benchmark because of how the
data was written [arxiv:1803.02324 tab. 2], and the other reports a gain from
relabelling disagreeing examples by hand [arxiv:2102.00225 tab. 2].

## Variants

### Convolutional network over word vectors

A sentence is turned into a fixed-length matrix of 100-dimensional word2vec
vectors, padded or cut to 50 tokens. Two groups of convolutions with 256
filters of width 5 are followed by max pooling, then a 128-unit dense layer and
a 26-way softmax [arxiv:1704.06841 §3] [arxiv:1704.06841 fig. 1]. The authors
report 0.68 accuracy, against 0.28 to 0.51 for a doc2vec classifier, mean
embeddings and bag-of-words with k-means features [arxiv:1704.06841 fig. 2].
The training recipe (optimiser, learning rate, batch size, epochs) is not stated,
and the dataset is not released. The training sentences come from encyclopedia
articles, and the paper does not test patient notes [arxiv:1704.06841 §5].
Card: [Medical Text Classification using Convolutional Neural Networks](/papers/1704.06841).

### Sequential classification with earlier texts as context

For a sequence of short texts, such as dialog utterances, each text is first
turned into a fixed-size vector. The encoder is an LSTM with max pooling or a
CNN with filters of height 3 [arxiv:1603.03827 §2.1] [arxiv:1603.03827 tab. 2].
A small two-layer network then labels the current text from its own vector
and from the vectors of a few earlier texts [arxiv:1603.03827 §2.2]. The
author-reported best accuracies are 66.2 on DSTC 4 (LSTM), 84.6 on MRDA (CNN)
and 73.1 on SwDA (CNN), against majority-class scores of 25.8, 59.1 and 33.7
[arxiv:1603.03827 tab. 4]. In the authors' grid, settings with history beat the
setting without it, and raising the history at the vector level helped more
than raising it at the label level [arxiv:1603.03827 §4] [arxiv:1603.03827
tab. 3]. The paper says the CNN beats the LSTM on every dataset, but on DSTC 4
its own table gives the LSTM the higher headline score (66.2 against 65.5)
[arxiv:1603.03827 tab. 4]. The grid averages are over five runs, while the
headline numbers are single runs picked on validation data. The learning rate,
batch size and epochs are not stated, and the paper does not say how a
validation split was made for two datasets [arxiv:1603.03827 §3.1]
[arxiv:1603.03827 §3.2]. Card: [Sequential Short-Text Classification with
Recurrent and Convolutional Neural Networks](/papers/1603.03827).

### Recurrent and convolutional models on word embeddings or TF-IDF

For binary positive and negative sentiment, a DNN, a CNN and an LSTM are each
trained on eight tweet and review sets. Each model gets either learned word
embeddings or TF-IDF vectors. The embedding input uses a 15,000-word vocabulary,
width 300 and sequences of at most 40 tokens [arxiv:2006.03541 §4.4]. Scores
come from 10-fold cross-validation, and the paper runs no classical baseline
[arxiv:2006.03541 §4.2]. Word embeddings give higher accuracy than TF-IDF for the
CNN and the LSTM on all eight sets. On Sentiment140 the CNN reaches 0.8006 with
embeddings and 0.7669 with TF-IDF, and the LSTM reaches 0.8282 against 0.5696
[arxiv:2006.03541 tab. 2]. The LSTM with embeddings is the most accurate on that
largest tweet set, but it takes 1 h 4 min to train there, against 7 min 3 s for
the CNN with embeddings [arxiv:2006.03541 tab. 7] [arxiv:2006.03541 tab. 8]. The
authors judge the CNN to give the best balance of accuracy and training time
[arxiv:2006.03541 §6]. The optimiser and learning rate are not stated, and the
"echoes" setting is not defined [arxiv:2006.03541 §5]. Card: [Sentiment
Analysis Based on Deep Learning: A Comparative Study](/papers/2006.03541).

### Classical pipeline as a map (survey)

This is a survey, not a method, and it runs no experiments of its own. It
splits the task into four phases: text cleaning and features, optional
dimensionality reduction, a classifier, and evaluation [arxiv:1904.08067 §1]
[arxiv:1904.08067 fig. 1]. The feature options it lists are bag-of-words, TF-IDF,
Word2Vec, GloVe and FastText [arxiv:1904.08067 §2.3] [arxiv:1904.08067 §2.4]. Its
classifier chapter covers naive Bayes, logistic regression, support vector
machines, decision trees and random forests, next to CNN, RNN and attention
models [arxiv:1904.08067 §4]. Its comparisons are qualitative, with pros and
cons per option [arxiv:1904.08067 §6]. It warns that accuracy misleads on
imbalanced data [arxiv:1904.08067 §1] [arxiv:1904.08067 §6]. Card:
[Text Classification Algorithms: A Survey](/papers/1904.08067). Use it to pick
options to test, and take the numbers from the experimental variants below.

### Data augmentation for small labelled sets (survey)

This is a survey, not a method, and it reports no experiments. It sorts text
augmentation into three families by how far the new text departs from the
original: paraphrasing with small rewordings, noising with swaps, deletions,
insertions or substitutions, and sampling of new text from rules, generators,
self-training or mixup [arxiv:2110.01852 §2]. Paraphrasing and noising mostly
ignore the label, while sampling methods use the labels and the task format
[arxiv:2110.01852 §2.4] [arxiv:2110.01852 §2.3]. Random insertion and
substitution can flip a label, so the survey suggests label-independent
resources at word level, and same-label examples at sentence level
[arxiv:2110.01852 §2.2.3] [arxiv:2110.01852 §2.2.4]. Mixup is the only online
method, and it mixes the labels as well, so its targets can differ from the
original labels [arxiv:2110.01852 §2.3.5]. The survey states no default values
for its probabilities and counts, so those choices are yours to make
[arxiv:2110.01852 fig. 12]. Card:
[Data Augmentation Approaches in Natural Language Processing: A Survey](/papers/2110.01852).

### Linear model on TF-IDF features

Words are weighted by term frequency times inverse document frequency, and a
linear classifier is trained on the result. On a merged set of 82,486 emails
with an 80/20 split, a linear-kernel SVC on TF-IDF reaches accuracy 0.991 and
F1 0.99 on the 16,498-email test split [arxiv:2405.11619 tab. 2]. The same
classifiers on Word2Vec inputs reach F1 0.71 to 0.83 in that table
[arxiv:2405.11619 tab. 2]. The paper uses one split and one random state, and
the best row is picked from eleven rows scored on that test split, so the
headline number is optimistic [arxiv:2405.11619 tab. 2]. The paper also adds
LIME explanations for single predictions [arxiv:2405.11619 §2]. Card:
[Novel Interpretable and Robust Web-based AI Platform for Phishing Email Detection](/papers/2405.11619).

The same family does not win everywhere. On Reddit users, a linear SVM on
TF-IDF features is best on one of nine binary conditions, and FastText is best
on seven [arxiv:1806.05258 tab. 4]. See Contradictions.

### Shallow neural models on user-level text

On the SMHD benchmark, each user is labelled with one or more of nine mental
health conditions, and the classifier reads all of that user's posts. Models
compared are logistic regression, linear SVM and XGBoost on bag-of-words
features, a FastText model with hidden size 100 and character n-grams, and a
CNN with one model per condition [arxiv:1806.05258 §5]. FastText has the best
F1 on seven of nine binary tasks, the linear SVM on bipolar, and the CNN on
eating [arxiv:1806.05258 tab. 4]. The authors note that bag-of-words models
favour precision over recall, and they explain the CNN's lower scores by the
fact that it reads only part of each user's text [arxiv:1806.05258 §5].
Scores are lowest for the three smallest groups [arxiv:1806.05258 §5]. The
dataset is shared under a data-use agreement, not openly. Card:
[SMHD](/papers/1806.05258).

### Multi-task learning with a shared encoder

One encoder is trained for several labels at once, and each label has its own
output layers [arxiv:1901.08014 §2]. In the sentiment and sarcasm model, a
shared GRU feeds two attention-pooled branches. A neural tensor network fuses
them before the sarcasm output [arxiv:1901.08014 §2]. On 994 sentences with
10-fold cross-validation, the fused model with shared attention reaches an
average F-score of 86.66. Standalone classifiers reach 83.75, and the cited
prior system, which also uses gaze features, reaches 82.13 [arxiv:1901.08014
tab. 1]. The paper reports no variance, and a gap of about one point between
the multi-task variants is within what small-data noise can produce. Card: [Sentiment and Sarcasm Classification with
Multitask Learning](/papers/1901.08014).

In rumour verification, one LSTM stack is shared by the stance, detection and
veracity tasks [arxiv:1806.03713 §4.2]. On PHEME with five events, the three-task
model reaches macro-F 0.396. The single-task branchLSTM reaches 0.336 and the
strongest reported baseline, NileTMRG*, reaches 0.339 [arxiv:1806.03713 tab. 3].
On RumourEval, the veracity-and-stance model reaches 0.558 against 0.539 for
NileTMRG* [arxiv:1806.03713 tab. 3]. The paper describes the loss as a plain sum
of task losses [arxiv:1806.03713 §4.2]. The released code uses fixed weights
instead, and it differs from the paper in other respects, so matching the
reported numbers requires choosing between the paper's text and the code. Card:
[All-in-one: Multi-task Learning for Rumour Verification](/papers/1806.03713).

### Pretrained encoder fine-tuned on in-domain labels

A RoBERTa-base model trained on the training half of ToxicChat reaches F1 72.1
on the test half, with a standard deviation of 1.2 over five seeds
[arxiv:2310.17389 tab. 2]. The best model trained on another dataset reaches
51.6, and a model trained on the combination of six other sets reaches 42.7
[arxiv:2310.17389 tab. 2]. Off-the-shelf tools score lower still. OpenAI
Moderation reaches F1 20.6, the Perspective API at threshold 0.7 reaches 5.3,
and ToxDectRoberta reaches 34.6 [arxiv:2310.17389 tab. 1]. The labelled data
are real first-turn prompts to one chatbot demo, in English only. The dataset
training hyperparameters are not stated [arxiv:2310.17389 §4.1], and the paper
does not state the dataset licence. Card:
[ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World
User-AI Conversation](/papers/2310.17389).

### Byte-level multilingual encoder with a multi-label head

The model reads UTF-8 bytes. A learned step scores blocks of one to four bytes
at each position and sums them into latent subwords, so no per-language
vocabulary is built [arxiv:2202.11176 §3.1]. The encoder is pretrained to fill
in masked spans. For fine-tuning the decoder is dropped, the first position is
pooled, and a sigmoid output per label lets several labels be on at once
[arxiv:2202.11176 §3.3]. On the authors' production multilingual set the
average AUC-ROC is 0.910 for the 102M model and 0.919 for the 268M model. The
comparison rows are mT5-base at 0.906, mT5-small at 0.878 and an in-house mBERT
at 0.840 [arxiv:2202.11176 tab. 1]. On the two code-switching sets the 102M
model is below mT5-base on JMTCC-CS, at 0.919 against 0.939, while the 268M
model is highest on both sets, at 0.945 and 0.902 [arxiv:2202.11176 tab. 3].
On obfuscated English Civil Comments, the authors report that the model beats
every baseline at every obfuscation rate above 0%, and that all models lose
accuracy as the rate rises [arxiv:2202.11176 §6.2]. The pretraining corpus
is partly proprietary, with about 4.6 billion texts in one half of the mix,
and the paper names no released weights or training code, so the design can
be reproduced in outline but not the pretrained model [arxiv:2202.11176 §3.4].
The fine-tuning stopping rule is "until convergence" and the number of labels
per task is not stated [arxiv:2202.11176 §5] [arxiv:2202.11176 §3.3]. Card:
[A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](/papers/2202.11176).

### Continued pretraining on in-domain text, then fine-tuning

BERT-Large is trained further, with its original masked-token and
next-sentence objectives, on about 22.5M cleaned COVID-19 tweets. Usernames and
links are replaced by shared placeholder tokens, and emojis become text aliases
[arxiv:2005.07503 §2]. The continued model is then fine-tuned on each labelled
set. On five classification sets the author-reported mean F1 rises from 0.802
for BERT-Large to 0.833 [arxiv:2005.07503 tab. 2]. The authors report their
largest relative gain on the COVID-19 category set (ΔMP 25.88%). That measure
divides the gain by the room the baseline leaves, and the paper's printed
formula has its numerator reversed [arxiv:2005.07503 §3.2]. On raw F1 the
largest gain is on the vaccine-sentiment set, from 0.824 to 0.869
[arxiv:2005.07503 tab. 2]. Downstream gains rise quickly up to about 200k
pretraining steps and then flatten [arxiv:2005.07503 §3.3]. Pretraining took
120 hours on one TPU v3-8 [arxiv:2005.07503 §2]. The fine-tuning recipe was
tuned on BERT-Large and reused for the new model [arxiv:2005.07503 §2.1], and
the authors note that out-of-domain text may need longer pretraining
[arxiv:2005.07503 §4]. Card:
[COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](/papers/2005.07503).

### Language-specific encoder on imbalanced sentiment and spam

MARBERT, a BERT-style encoder pretrained on Arabic tweets, is fine-tuned in two
separate runs: one for five-class sentiment and one for spam. Both use private
customer tweets from a Saudi telecom company [arxiv:2606.25495 §4.2]
[arxiv:2606.25495 §5.1]. The model drops next-sentence prediction, because it
was pretrained on short texts [arxiv:2606.25495 §4.2]. The best of six
settings reaches sentiment F1 0.75, with learning rate 2e-5, training batch 96,
maximum length 256 and 5 epochs [arxiv:2606.25495 tab. 4]. At batch size 96,
per-class F1 is 0.85 for negative, 0.92 for positive, 0.89 for neutral, 0.54
for sarcasm and 0.56 for indeterminate [arxiv:2606.25495 tab. 5]. The sarcasm
class was weak at batch size 16, at 0.33 [arxiv:2606.25495 tab. 5]. The
authors rejected over- and undersampling, and raised the sarcasm examples from
about 200 to about 1,400 by collecting more tweets [arxiv:2606.25495 §5.3]. The
spam F1 of 0.98 [arxiv:2606.25495 tab. 3] is on labels from a keyword list, not
on hand-labelled data [arxiv:2606.25495 §5.1]. The paper does not run any
baseline on its own data, and its comparison rows come from other papers and
datasets [arxiv:2606.25495 tab. 3]. Card:
[Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model](/papers/2606.25495).

### Language-specific BERT with a CNN head on the last layers

A language-specific BERT-base is fine-tuned with a small CNN on top. The CNN
reads the outputs of the last four BERT layers as four channels. It has 160
filters in five heights, from one to five tokens, then global max pooling and a
sigmoid output for offensive or not [arxiv:2007.13184 §4.4] [arxiv:2007.13184
fig. 1]. On OffensEval 2020 subtask A the author-reported macro-F1 is 0.897 for
Arabic, 0.843 for Greek and 0.814 for Turkish, which the authors report as 4th,
4th and 3rd place [arxiv:2007.13184 tab. 2] [arxiv:2007.13184 §1]. Against BERT
alone the average gain is 0.010, and Turkish is slightly lower, at 0.814
against 0.816 [arxiv:2007.13184 tab. 2]. The paper reports single numbers with
no repeated runs [arxiv:2007.13184 §5], so the size of the gain is uncertain.
The training recipe is only partly stated, and the card's Paper vs code section
lists where the released code differs on epochs and checkpoint selection
[arxiv:2007.13184 §4.4]. Card: [KUISAIL at SemEval-2020 Task 12: BERT-CNN for
Offensive Speech Identification in Social Media](/papers/2007.13184).

### Few-shot contrastive fine-tuning with a linear head

The method tunes a pretrained sentence encoder on pairs built from a few labelled
examples. Pairs with the same label are pulled together and pairs with different
labels are pushed apart. A logistic-regression head is then trained on the
resulting embeddings [arxiv:2209.11055 §3.1]. No prompt or verbalizer is needed.
Card: [Efficient Few-Shot Learning Without Prompts](/papers/2209.11055).

With 8 examples per class, the 110M-parameter MPNet body averages 62.3 ± 4.9 over
five English datasets, with AG News left out. T-Few 3B, which uses prompts,
averages 63.4 ± 1.9 and ADAPET 58.3 ± 3.6. Fine-tuning RoBERTa-large averages
43.0 ± 5.2 [arxiv:2209.11055 tab. 2]. With 64 examples per class the same body
averages 75.3 ± 1.3, against 73.8 ± 2.2 for ADAPET and 70.3 ± 1.5 for T-Few 3B
[arxiv:2209.11055 tab. 2]. These are the paper's own runs. The T-Few figure is a
median over 5 seeds and the SetFit figure a mean over 10 splits [arxiv:2209.11055 §4.3]
[arxiv:2209.11055 §4.4].

Cost is reported as formula estimates, not measured values. The 110M body needs
about 19 times fewer inference FLOPs than T-Few 3B, and a 15M distilled student
needs about 123 times fewer [arxiv:2209.11055 tab. 5]. The paper also reports
roughly 30 seconds of training per split on one AWS GPU instance
[arxiv:2209.11055 §7.2].

Two more settings are reported. With a multilingual sentence encoder, the method
beats fine-tuning and ADAPET at 8 examples per class in every setting tested,
on a lower-is-better regression metric [arxiv:2209.11055 §6] [arxiv:2209.11055 tab. 4].
A 15M student distilled from the 110M teacher reaches 60.3 ± 1.6 at 8 examples
per class [arxiv:2209.11055 tab. 5].

### Curated adversarial evaluation (robustness)

This is an evaluation variant rather than a training method. The benchmark takes
the GLUE dev sets for five tasks and runs 14 attack entries on them, grouped into
11 perturbation columns. The groups are synonym swaps, typos, knowledge-based
substitutions, syntactic rewrites, distracting sentences and human-written sets
[arxiv:2111.02840 tab. 1] [arxiv:2111.02840 §3.2]. Examples are kept only if
they transfer across three surrogate models, change fewer than 15% of the
words, and receive a 4-of-5 annotator vote for the original label
[arxiv:2111.02840 §3.3]. Card:
[Adversarial GLUE](/papers/2111.02840).

The author-reported result is that ten models average 30.3 to 60.9 on this
benchmark, against 85.7 to 93.2 on GLUE [arxiv:2111.02840 tab. 4]. BERT (Large)
goes from 85.76 on GLUE to 33.68 on the adversarial set [arxiv:2111.02840 tab. 4].
The paper finds that the human-written sets (C8 to C11) hurt most, and that
distracting sentences (C7) hurt more than syntactic rewrites (C6)
[arxiv:2111.02840 §4]. Only the dev split is public. Test scores need a hidden
test set [arxiv:2111.02840 §3.5].

### Word-substitution attack on a classifier

The attack is black-box. It ranks words by how much removing each one changes
the model's output, then replaces the most important ones with embedding
neighbours that pass a sentence-similarity check and a part-of-speech check,
until the label flips [arxiv:1907.11932 alg. 1] [arxiv:1907.11932 eq. 2]. The
paper attacks WordCNN, WordLSTM and BERT on five classification sets. For BERT, accuracy
falls to 11.5 on MR, 13.6 on IMDB, 6.6 on Yelp and 12.5 on AG, with 6.1% to
22.0% of words perturbed [arxiv:1907.11932 tab. 3]. On entailment, BERT falls
to 4.0 on SNLI and 9.6 on MultiNLI matched, with 15.2% to 18.5% of words
perturbed [arxiv:1907.11932 tab. 4]. Human checks on 100 adversarial examples
each for MR and SNLI give label agreement with the original of 92% and 85%
[arxiv:1907.11932 §Results, Human Evaluation]. Adversarial training raises BERT's MR after-attack accuracy
from 11.5 to 18.7, from one run [arxiv:1907.11932 tab. 12]. The official code
differs from the paper's stated thresholds and similarity measure in several
places [arxiv:1907.11932 §Paper vs code]. Card:
[Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](/papers/1907.11932).

### Masked-language-model substitution attack

BAE is also black-box. It scores each token by how much deleting it lowers the
probability of the true label, then masks tokens in that order. A BERT masked
language model proposes replacements or insertions for each mask, and the
candidates are filtered by sentence similarity and, for replacements, by part of
speech [arxiv:2004.01970 §2] [arxiv:2004.01970 app. A]. The paper defines four
modes: replace only (BAE-R), insert only (BAE-I), either per token (BAE-R/I),
and replace then insert (BAE-R+I) [arxiv:2004.01970 §2]. On BERT, the author-reported
accuracy after attack is 19.2 on MR for BAE-R+I, against 30.7 for TextFooler,
and 14.0 on Amazon, against 30.0 [arxiv:2004.01970 tab. 1]. BAE-R alone is the
exception: on BERT it leaves more accuracy than TextFooler on six of the seven
datasets, and less only on TREC [arxiv:2004.01970 tab. 1] [arxiv:2004.01970 tab. 2]. In a small human check on BERT, rated naturalness is
higher for BAE-R than for TextFooler on all three datasets tested
[arxiv:2004.01970 tab. 3]. The paper's results use 100-example test sets for
Amazon, Yelp and IMDB [arxiv:2004.01970 tab. 5], so the variance of those
numbers is large. Card: [BAE: BERT-based Adversarial Examples for Text Classification](/papers/2004.01970).

### Zero-shot LLM annotation

The model is given the same codebook that human coders use and labels each
text with no training examples. The paper uses `gpt-3.5-turbo` and compares
it with MTurk crowd-workers and trained research assistants on five tasks
over four samples [arxiv:2303.15056 §4.1] [arxiv:2303.15056 §4.2]. On average
across the four datasets, ChatGPT's accuracy against trained annotators is
about 25 points above MTurk [arxiv:2303.15056 §2]. Intercoder agreement is
91% at the default temperature and 97% at temperature 0.2, against 56% for
MTurk and 79% for trained annotators [arxiv:2303.15056 §2]. The stated cost
is below $0.003 per annotation, which the authors put at about thirty times
cheaper than MTurk [arxiv:2303.15056 §1]. The samples are US political and
content-moderation texts. The language mix of three samples is not stated, and
the card records no published code. Card:
[ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](/papers/2303.15056).

### Zero-shot LLM classification through machine translation

The same zero-shot idea, applied to a clinical label. GPT-3.5-turbo at
temperature 0 assigns each Reddit post one of four depression-severity levels.
It does this first on the English posts, then on Greek posts that the same
model has translated from the English [arxiv:2410.12985 §3] [arxiv:2410.12985
fig. 1]. The author-reported macro F1 is 0.17 in English and 0.16 in Greek
[arxiv:2410.12985 tab. 2]. The Minimum class holds most posts, and the English
recall for that class is 0.14 [arxiv:2410.12985 tab. 1] [arxiv:2410.12985
tab. 2]. Greek F1 is lower than English for three of the four levels
[arxiv:2410.12985 tab. 2]. The Greek prompt is not stated, and the paper has no
translation quality metric [arxiv:2410.12985 tab. 1] [arxiv:2410.12985 §4].
Each language has one run, and the English run is the only baseline
[arxiv:2410.12985 §4]. The authors limit the approach to helping train
professionals, not to diagnosis [arxiv:2410.12985 §5]. Card:
[Leveraging LLMs for Translating and Classifying Mental Health Data](/papers/2410.12985).

### Crowd-labelled training data and a bag-of-n-grams labeller

Crowd workers label a sample of comments, a bag-of-n-grams classifier is trained
on their labels, and the classifier then labels a corpus too large to read by
hand. Each sampled comment gets at least 10 crowd labels, and annotators must
pass a test before they contribute [arxiv:1610.08914 §3] [arxiv:1610.08914 tab. 1].
The classifier is checked against small annotator ensembles on 8,000 comments.
On AUC it beats an ensemble of three raters (95.49) and reaches 97.19, against
97.13 for five raters. On Spearman correlation it sits between three raters
(64.75) and five (68.27) [arxiv:1610.08914 tab. 3]. The paper reads this as
matching about three raters [arxiv:1610.08914 §4.3]. The model is then run on 63
million English talk comments with a fixed score threshold of 0.425. At that
threshold the reported precision and recall are both 0.63 [arxiv:1610.08914 §5.1].
The paper does not name, in its text, the one classifier it uses downstream, and
its own dev-split table gives several close candidates [arxiv:1610.08914 §4.3]
[arxiv:1610.08914 tab. 2]. The labels are for personal attacks, and the
measured quality holds for that property on that corpus, so validate the
threshold on your own data. Card:
[Ex Machina: Personal Attacks Seen at Scale](/papers/1610.08914).

### Relabel disagreeing examples, then train with the first model's output

A first model is trained on the original labels. Examples where its prediction
disagrees with the label are relabelled by hand. A second model then gets the
first model's one-hot prediction as an extra input and is trained on the
corrected labels [arxiv:2102.00225 §5] [arxiv:2102.00225 fig. 1]. On a private
industrial dataset with BERT-Base, the first model reaches 83.3% test accuracy
and the second 92.5%. The second model without the extra input reaches 91.7%
[arxiv:2102.00225 tab. 2]. On 5,000 human-checked items, the three models score
88.0%, 97.2% and 97.7% [arxiv:2102.00225 tab. 2]. The paper reports one run per
model and no variance, and the gain from the extra input is 0.8 points
[arxiv:2102.00225 tab. 2]. Card:
[Learning From How Humans Correct](/papers/2102.00225).

### Input-only baseline as a dataset-artifact check

A classifier that sees only the hypothesis of an NLI pair, with no premise, is
a fastText model over words and bigrams. On SNLI it reaches 67.0% against 34.3%
for the majority class. On MultiNLI matched and mismatched it reaches 53.9% and
52.3%, against 35.4% and 35.2% [arxiv:1803.02324 tab. 2]. Words such as
"nobody" and "no" are tied to the contradiction class, and words such as
"outdoors" to entailment [arxiv:1803.02324 tab. 4]. The paper then splits the
test sets into hard examples, which this classifier gets wrong, and easy ones.
Three NLI models score 84.7, 85.8 and 86.5 on the full SNLI test set, and 69.4,
71.3 and 72.7 on its hard subset [arxiv:1803.02324 tab. 5]. The authors read
this as most of the models' success coming from easy examples
[arxiv:1803.02324 §4]. The method is a classifier plus a split rule, and the
card records no official code. Card:
[Annotation Artifacts in Natural Language Inference Data](/papers/1803.02324).

### Energy cost of running a classifier

This is a measurement study, not a method. It runs 88 models on ten tasks,
with 1,000 inferences per dataset, one input at a time, and records energy with
CodeCarbon on eight A100 GPUs in one AWS region [arxiv:2311.16863 §3.2]
[arxiv:2311.16863 tab. 1]. Mean energy per 1,000 inferences is 0.002 kWh for
text classification and 2.907 kWh for image generation [arxiv:2311.16863 tab. 2]
[arxiv:2311.16863 §4.1]. Zero-shot multi-purpose models (Flan-T5 and BLOOMz)
cost more than task-specific models on the same task, most clearly for
classification and question answering [arxiv:2311.16863 §4.2]. The paper
states a grid carbon intensity that its own data do not fully match, so treat
the carbon figures with care and read the energy figures first
[arxiv:2311.16863 §3.2]. Card: [Power Hungry Processing: Watts Driving the Cost
of AI Deployment?](/papers/2311.16863).

## When to use which

| Situation | Use | Why |
|---|---|---|
| About 8 to 64 labelled examples per class, a sentence encoder available, no budget for prompts or very large models | Few-shot contrastive fine-tuning with a linear head | Matches the prompt-based T-Few 3B (63.4) to within about 1 point at 8 per class, and beats ADAPET at 8 and 64 per class [arxiv:2209.11055 tab. 2] |
| Tight inference cost or latency with few labels | 110M body, or a distilled 15M student | FLOP estimates favour both, with a score drop for the student [arxiv:2209.11055 tab. 5] |
| Labels in several languages with few examples | Multilingual sentence encoder body | Reported to beat fine-tuning and ADAPET at 8 examples per class in every setting [arxiv:2209.11055 §6] |
| Large labelled set of emails or similar text, a fast classical baseline, and a need for explanations | TF-IDF with a linear SVM, plus LIME | Reported as the best of the compared models on one split [arxiv:2405.11619 tab. 2]. Check on your own data, since the same family lags FastText on SMHD [arxiv:1806.05258 tab. 4] |
| Many posts per user, several labels, shallow model wanted | FastText-style model | Best F1 on seven of nine binary conditions in the SMHD benchmark [arxiv:1806.05258 tab. 4] |
| Several labels for the same input, where the labels may help each other | Multi-task model with shared encoder | Gains over single-task models are reported on one small benchmark and on rumour data [arxiv:1901.08014 tab. 1] [arxiv:1806.03713 tab. 3]. Without variance, treat small gaps as unproven |
| Binary sentiment with a small neural model trained from scratch, and word embeddings available | Embeddings as input; a CNN when training time counts, an LSTM when accuracy matters more | Embeddings beat TF-IDF for the CNN and LSTM on all eight sets [arxiv:2006.03541 tab. 2]. On Sentiment140 the LSTM trains about nine times longer than the CNN [arxiv:2006.03541 tab. 7]. The paper runs no variance check, so treat small gaps as unproven |
| Sentence-level categories with word embeddings and no budget for a transformer, in a narrow domain | CNN over word vectors | Reported to beat bag-of-words and doc2vec baselines on the authors' data [arxiv:1704.06841 fig. 2]. Patient notes were not tested [arxiv:1704.06841 §5] |
| Deploying to a new domain such as chatbot prompts | Label in-domain data, then fine-tune a pretrained encoder | Models trained on other toxicity data score F1 at most 51.6 on the target domain, against 72.1 in-domain [arxiv:2310.17389 tab. 2] |
| Large unlabelled corpus from the target domain, and a classification task on that domain | Continue pretraining on the in-domain text, then fine-tune | Mean F1 rose from 0.802 to 0.833 over five sets, author-reported. The table has no variance or significance test [arxiv:2005.07503 tab. 2] [arxiv:2005.07503 §3.2]. Gains flatten after about 200k steps [arxiv:2005.07503 §3.3] |
| Non-English short social-media text with a pretrained encoder for that language, and a rare class you must get right | Fine-tune the language-specific encoder, and collect more examples of the rare class | Sarcasm F1 reached 0.54 and indeterminate 0.56 at batch size 96, against 0.85 to 0.92 for the common classes [arxiv:2606.25495 tab. 5]. The authors chose collection over resampling [arxiv:2606.25495 §5.3]. One private dataset only |
| Multilingual or code-switched short text, where per-language vocabularies are a burden, and the budget covers pretraining | Byte-level encoder with a learned tokenizer and a multi-label head | Author-reported average AUC-ROC of 0.910 (102M) and 0.919 (268M) on production data, against 0.906 for mT5-base [arxiv:2202.11176 tab. 1]. The 102M model trails mT5-base on one code-switched set [arxiv:2202.11176 tab. 3]. The pretrained model is not released, so you must pretrain your own |
| Small labelled set, and a label-preserving rewrite of each example is available | Paraphrase or noise first, and sample new text only with labels and a task format | Paraphrasing and noising mostly ignore labels, while sampling uses them [arxiv:2110.01852 §2.4]. Word-level noise can flip a label [arxiv:2110.01852 §2.2.3] |
| Deploying a model that must hold up to altered input | Test on an adversarial set and expect a large gap to clean accuracy | Fine-tuned encoders lose up to 55 points on the adversarial average [arxiv:2111.02840 tab. 4] |
| Comparing robustness numbers across papers | Use the same attack set and split | Each attack set is built from different attack choices and filters [arxiv:2111.02840 §3.2] |
| Measuring how robust a deployed classifier is to word-level edits, black-box | Word-substitution attack as a stress test | Reported to cut BERT's accuracy to single digits on several sets, with few words changed [arxiv:1907.11932 tab. 3] [arxiv:1907.11932 tab. 4]. Check the official code's thresholds before reusing it |
| Large volume of items to label, a validated codebook, and budget for a sample of human checks | Zero-shot LLM annotation, checked against human coders on a sample | Reported about 25 points above MTurk on accuracy against trained annotators, on four samples [arxiv:2303.15056 §2]. The samples are not a general test, so validate on your own data |
| Labels from a noisy process, a budget for hand relabelling of disagreements | Relabel, then train a second model with the first model's prediction as input | Reported 9.2 points above the first model on one industrial dataset, one run [arxiv:2102.00225 tab. 2] |
| Judging whether a benchmark measures inference | Run an input-only baseline first | If it beats the majority class by a wide margin, the labels can be guessed from the hypothesis alone [arxiv:1803.02324 tab. 2] |
| Black-box stress test of a BERT classifier, where replacements should fit the sentence | Masked-LM substitution attack (BAE) | Reported as BAE-R+I lowering BERT's accuracy further than TextFooler on all seven datasets (replace-only does not), with author-reported naturalness gains on three small human checks [arxiv:2004.01970 tab. 1] [arxiv:2004.01970 tab. 3] |
| Huge corpus, a property that crowd workers can label reliably, and a need to label all of it | Train a bag-of-n-grams classifier on crowd labels, and check it against small annotator ensembles | Reported to beat an ensemble of three raters on AUC and Spearman on one corpus [arxiv:1610.08914 tab. 3]. Re-run the ensemble check on your own labels before trusting the machine labels |
| Ordered short texts, such as dialog turns, where earlier texts give context | Sequential model that reads the vectors of earlier texts | History helped in the authors' grid [arxiv:1603.03827 tab. 3]. Headline scores are single runs, so check the gain on your own validation data [arxiv:1603.03827 tab. 4] |
| High-volume classification where inference energy or cost counts | Task-specific classifier rather than a multi-purpose generative model | Zero-shot multi-purpose models cost more per inference than task-specific models on the same task [arxiv:2311.16863 §4.2]. Text classification averages 0.002 kWh per 1,000 inferences [arxiv:2311.16863 tab. 2] |
| Choosing among classical features and classifiers before running your own comparison | Read the survey as a map, then test the shortlist | The survey gives qualitative pros and cons and no experiments [arxiv:1904.08067 §6] |
| Offensive or toxic short text in Arabic, Greek or Turkish, a language-specific BERT available, and GPU budget for fine-tuning | Language-specific BERT-base with a CNN head on the last four layers | Highest average of the compared models on OffensEval 2020 subtask A, but on Turkish BERT alone is slightly higher. Single runs, so check the gain on your own validation data [arxiv:2007.13184 tab. 2] |
| Clinical or mental-health labels in a non-English language, zero-shot, with an LLM translating the input | Do not use this pipeline as the sole method | Macro F1 0.17 in English and 0.16 in Greek, one run, and the English run is the only baseline [arxiv:2410.12985 tab. 2]. The authors limit the approach to training professionals [arxiv:2410.12985 §5] |

## Implementation sketch

Few-shot path, after [arxiv:2209.11055 §3.1]. The paper sets the number of pairs
per class and the training epochs. The official repository's defaults for the
pair count and the body learning rate differ from the paper, so set them
explicitly when you reproduce a result
([Efficient Few-Shot Learning Without Prompts](/papers/2209.11055)).

```pseudocode
# Build labelled pairs from the K examples  [arxiv:2209.11055 §3.1]
# Same label -> target 1.0; different label -> target 0.0
pairs = []
for i, j in sampled_index_pairs(K):      # R pairs per class in the paper
    target = 1.0 if label[i] == label[j] else 0.0
    pairs.append((text[i], text[j], target))

# Tune the encoder so the cosine similarity of each pair matches its target
# [arxiv:2209.11055 §3.1]; 1 epoch, batch 16 [arxiv:2209.11055 §4.4]
encoder = fine_tune_cosine(encoder, pairs)

# Fit a logistic-regression head on the embeddings of the K examples
# [arxiv:2209.11055 §3.1]
head = fit_logistic_regression([encoder.encode(t) for t in text], label)

# Classify an unseen text  [arxiv:2209.11055 §3.1]
predicted = head.predict(encoder.encode(new_text))
```

TF-IDF baseline, after [arxiv:2405.11619 §2]. The paper does not state
whether the vocabulary and IDF weights were fitted on training rows only. Fit
them on training rows, so that test information does not leak into the model.

```pseudocode
# Weight words by TF-IDF; fit on training texts only  [arxiv:2405.11619 §2]
vectorizer = TfidfVectorizer()
train_x = vectorizer.fit_transform(train_texts)
test_x = vectorizer.transform(test_texts)

# Linear-kernel SVC with a fixed random state  [arxiv:2405.11619 §2]
# The regularisation strength is not stated in the paper; set and report it
model = SVC(kernel="linear", random_state=42)
model.fit(train_x, train_labels)
predicted = model.predict(test_x)
```

Multi-task path, after [arxiv:1901.08014 §2]. The paper gives equal priority to
its two losses and sums them. The released code of [arxiv:1806.03713] weights its
losses differently, so state which choice you use.

```pseudocode
# Shared encoder, one output head per task  [arxiv:1901.08014 §2]
hidden = shared_encoder(text)
loss_a = cross_entropy(head_a(hidden), labels_a)
loss_b = cross_entropy(head_b(hidden), labels_b)

# Equal priority: a plain sum of the task losses  [arxiv:1901.08014 §2]
loss = loss_a + loss_b
```

Robustness check, after [arxiv:2111.02840 §3.3] and [tab. 4]:

```pseudocode
# Score a model on adversarial dev examples for one task  [arxiv:2111.02840 §3.5]
correct = 0
for example in adversarial_dev_set:
    if model.predict(example.text) == example.label:
        correct += 1
accuracy = correct / len(adversarial_dev_set)
# Compare with clean GLUE accuracy on the same task; large gaps are expected
# [arxiv:2111.02840 tab. 4]
```

Relabel-then-stack path, after [arxiv:2102.00225 §5]. The paper's second model
takes the first model's one-hot output as an extra input. Relabelling is by
hand, so check the set you relabel on a held-out sample first.

```pseudocode
# Train the first model on the original labels  [arxiv:2102.00225 §5]
model_a = fine_tune(encoder, texts, labels)

# Find the items where model A disagrees with the label, then relabel them
disagree = [i for i in range(len(texts))
            if model_a.predict(texts[i]) != labels[i]]
corrected = relabel_by_hand(texts, labels, disagree)

# Train model C on corrected labels, with model A's one-hot output as input
# [arxiv:2102.00225 tab. 2]
model_c = fine_tune(encoder, texts, corrected,
                    extra=one_hot(model_a.predict(texts)))
```

## Contradictions

Two cards compare word-vector and TF-IDF features with different results, and
the comparisons cannot be ranked against each other. On medical sentences, a
CNN over word2vec reaches 0.68 against 0.51 for bag-of-words with logistic
regression [arxiv:1704.06841 fig. 2]. On a merged email set, a linear SVC on
TF-IDF reaches F1 0.99 while the Word2Vec-based classifiers reach F1 0.71 to
0.83 [arxiv:2405.11619 tab. 2]. The two papers use different data, tasks,
baselines and Word2Vec settings, and the email paper does not state how word
vectors are turned into document vectors [arxiv:2405.11619 §2].

On user-level text, a linear SVM on TF-IDF is best on one of nine conditions
and FastText on seven [arxiv:1806.05258 tab. 4]. The phishing paper reports
TF-IDF with a linear SVM as best among its three classifiers, and it tests no
neural model [arxiv:2405.11619 tab. 2]. The two results do not conflict, because
the tasks and inputs differ, but they show that a linear winner on one dataset
does not carry over.

On binary sentiment, the pattern flips for the DNN. TF-IDF beats word embeddings
for the DNN on five of eight sets, and the two tie on one [arxiv:2006.03541
tab. 2]. The embeddings win for the CNN and the LSTM on all eight. The TF-IDF
rows feed neural models, not a linear classifier, so they do not test the
linear-model result above. The paper gives one number per cell and no spread, so
the small gaps in that table may be noise [arxiv:2006.03541 tab. 2].

The few-shot results and the adversarial results measure different things. The
few-shot paper does not test altered inputs [arxiv:2209.11055 §4.4]. The
adversarial benchmark trains on full GLUE training data rather than few-shot
samples [arxiv:2111.02840 §3.1]. The two sets of numbers cannot be compared
directly, because the few-shot results are on five other English datasets
[arxiv:2209.11055 tab. 2].

The annotation numbers use different references. The LLM-annotation paper
measures accuracy against trained annotators and intercoder agreement on
four samples [arxiv:2303.15056 §2]. The relabelling paper measures accuracy
against hand-checked labels on 5,000 items from one private dataset
[arxiv:2102.00225 tab. 2]. Neither result can be read as a rate for the other,
and the first paper does not give per-task accuracy in its text
[arxiv:2303.15056 fig. 1].

The two newest results are not comparable with each other either. CT-BERT
reports F1 on five English sets against BERT-Large, under one recipe for both
models [arxiv:2005.07503 tab. 2]. MARBERT reports F1 on one private Arabic
dataset, and its comparison rows come from other papers and datasets
[arxiv:2606.25495 tab. 3]. The MARBERT abstract's claim of accuracy gains over
existing techniques is therefore not supported by a baseline on the same data
[arxiv:2606.25495 tab. 2].

The two zero-shot LLM results also differ. The annotation paper reports
ChatGPT about 25 points above crowd-workers on accuracy against trained
annotators, with intercoder agreement of 91% to 97% on four samples
[arxiv:2303.15056 §2]. The severity paper reports GPT-3.5-turbo at macro F1
0.17 on four depression-severity levels, against gold labels, with the English
run as its only baseline [arxiv:2410.12985 tab. 2]. The tasks, label scales and
references differ, and macro F1 weights the rare classes equally, so the two
numbers cannot be ranked against each other. Together they show that zero-shot
LLM labels depend strongly on the task.

## Papers

- [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](/papers/1603.03827): an LSTM or CNN encoder with a history-aware two-layer classifier for dialog acts, with author-reported best accuracies on three dialog datasets.
- [Medical Text Classification using Convolutional Neural Networks](/papers/1704.06841): a CNN over word2vec vectors for 26 medical sentence categories, with author-reported gains over simple baselines on an unreleased dataset.
- [Annotation Artifacts in Natural Language Inference Data](/papers/1803.02324): a hypothesis-only fastText classifier that beats the majority class on SNLI and MultiNLI, and hard-subset scores well below full-test scores for three NLI models.
- [All-in-one: Multi-task Learning for Rumour Verification](/papers/1806.03713): one shared LSTM for stance, detection and veracity on Twitter reply branches, with small author-reported gains over single-task models.
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](/papers/1806.05258): a Reddit dataset of self-reported diagnoses with matched controls, and baseline comparisons of bag-of-words, FastText and CNN models.
- [Sentiment and Sarcasm Classification with Multitask Learning](/papers/1901.08014): a shared GRU with attention and a tensor fusion for joint sentiment and sarcasm labels, reported on 994 sentences.
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](/papers/1907.11932): a black-box word-substitution attack that cuts accuracy of several classifiers, with author-reported human checks.
- [Learning From How Humans Correct](/papers/2102.00225): a case study of relabelling disagreeing examples in an industrial text classifier, then training with the first model's output as input.
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](/papers/2303.15056): zero-shot ChatGPT annotation compared with crowd-workers and trained annotators on four samples, with author-reported agreement and cost figures.
- [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](/papers/2310.17389): a benchmark of real chatbot prompts, showing that detectors trained on other domains transfer poorly.
- [Novel Interpretable and Robust Web-based AI Platform for Phishing Email Detection](/papers/2405.11619): TF-IDF with linear and tree classifiers on merged spam emails, with LIME explanations and one reported split.
- [Adversarial GLUE](/papers/2111.02840): an adversarial test set for five GLUE classification tasks, with large author-reported drops from clean GLUE scores.
- [Efficient Few-Shot Learning Without Prompts](/papers/2209.11055): contrastive fine-tuning of a sentence encoder plus logistic regression, for few-shot text classification without prompts.
- [Ex Machina: Personal Attacks Seen at Scale](/papers/1610.08914): crowd-labelled Wikipedia talk comments and a bag-of-n-grams classifier, checked against small annotator ensembles and run on 63 million comments.
- [Text Classification Algorithms: A Survey](/papers/1904.08067): a survey of features, dimensionality reduction, classifiers and metrics, with qualitative comparisons and no experiments.
- [BAE: BERT-based Adversarial Examples for Text Classification](/papers/2004.01970): a black-box attack that uses BERT's masked-LM guesses as word replacements and insertions, with author-reported gains over TextFooler.
- [Sentiment Analysis Based on Deep Learning: A Comparative Study](/papers/2006.03541): a DNN, CNN and LSTM compared on word embeddings and TF-IDF for binary sentiment on eight tweet and review sets, with author-reported accuracy and training times.
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](/papers/2311.16863): energy and carbon per 1,000 inferences for 88 models on ten tasks, with text classification among the cheapest tasks measured.
- [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](/papers/2005.07503): BERT-Large continued on 22.5M COVID-19 tweets, with author-reported mean F1 gains on five classification sets.
- [Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model](/papers/2606.25495): MARBERT fine-tuned for five-class sentiment and for spam on private telecom tweets, with per-class F1 and no baseline run on its own data.
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](/papers/2007.13184): a language-specific BERT-base with a CNN over its last four layers, with author-reported macro-F1 on three OffensEval 2020 sets.
- [Data Augmentation Approaches in Natural Language Processing: A Survey](/papers/2110.01852): a survey that groups text augmentation into paraphrasing, noising and sampling, with the label risks of each, and no experiments.
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](/papers/2202.11176): a byte-level multilingual toxicity encoder with a learned tokenizer and a multi-label head, with author-reported AUC-ROC on production, code-switched and public sets.
- [Leveraging LLMs for Translating and Classifying Mental Health Data](/papers/2410.12985): GPT-3.5-turbo zero-shot depression-severity classification in English and in Greek translated by the same model, with low author-reported macro F1.