---
kind: concept
slug: sentiment-analysis
title: Sentiment analysis
summary: >-
  Sentiment analysis labels the opinion in a text, as polarity, a score or emotion. Lexicon tools, CNN and LSTM classifiers, fine-tuned pretrained encoders, multimodal fusion and prompted LLMs are the options. Reported numbers use different datasets, labels and metrics and cannot be compared; sarcasm, emoticons and aspect-level opinions are hard cases.
tags: [nlp, sentiment, text-classification, sarcasm, multi-task-learning, multimodal, llm, lexicon, arabic]
papers: ["1406.0032", "1610.08815", "1806.06228", "1810.02508", "1901.08014", "1905.02947", "1911.05544", "2005.00357", "2005.03545", "2006.03541", "2109.00412", "2301.08826", "2310.09219", "2401.11641", "2606.25495"]
related: [topic-modelling, text-summarization, feature-selection, multi-task-learning, attention-mechanism, emotion-recognition-in-conversation, multimodal-fusion, mutual-information-maximization, deep-canonical-correlation-analysis, prompt-engineering, text-classification, text-embedding, convolutional-neural-network, language-model-pretraining, masked-language-modeling, transfer-learning, word-embedding, algorithmic-fairness, hallucination-detection, word-embedding-association-test]
status: draft
updated_at: 2026-11-01
---

## Overview

Sentiment analysis assigns an opinion label to a piece of text. The label set
differs between the cited studies: positive and negative, a three-way split into
positive, negative and neutral, a five-way split that adds sarcasm and an
indeterminate class, a continuous intensity score, or emotion
[arxiv:2301.08826 §2.1] [arxiv:1810.02508 tab. 13] [arxiv:2005.03545 §3.1]
[arxiv:2606.25495 §5.1]. A sarcasm paper trains one network to label sentiment
and sarcasm together, with sarcasm as a yes/no label [arxiv:1901.08014 §2]. The
review applies sentiment analysis to student feedback, where it sits next to
feature extraction, topic modelling and summarisation [arxiv:2301.08826 §2]. The
review is a literature survey with no experiments of its own, so every number
from it is a result its authors quote from another study [arxiv:2301.08826 §5].
The sarcasm paper reports its own experiments, on one small benchmark
[arxiv:1901.08014 §3] [arxiv:1901.08014 §4]. The multimodal and conversational
studies also report their own experiments, on public benchmarks
[arxiv:2005.03545 §4] [arxiv:1810.02508 §4]. A third multimodal study adds
mutual-information terms to a regressor over the same three channels
[arxiv:2109.00412 §3.2]. A hierarchical fusion model over text, audio and video
reports its own results on CMU-MOSI and IEMOCAP, with a context variant for
clips of utterances [arxiv:1806.06228 §4.1] [arxiv:1806.06228 §4.4.3]. A
sarcasm study feeds sentiment, emotion and personality networks into a sarcasm
classifier [arxiv:1610.08815 §4] [arxiv:1610.08815 §5.2]. A study of
recommendation letters uses a sentiment classifier only to score tone, not to
study sentiment models [arxiv:2310.09219 §4.3.4]. A canonical-correlation model over the same channels
reports its own benchmark numbers [arxiv:1911.05544 §4]. A survey of emotion
recognition in conversation labels each utterance with an emotion class rather
than a polarity, and collects results from earlier papers
[arxiv:1905.02947 §1] [arxiv:1905.02947 §4]. A comparison of a DNN, a CNN and an
LSTM on eight tweet and review sets reports word embeddings ahead of TF-IDF for
the CNN and LSTM on every set [arxiv:2006.03541 tab. 2]. A comparison of eight
polarity tools measures how many messages each one labels at all, and how often
the tools agree with one another [arxiv:1406.0032 §1] [arxiv:1406.0032 §3.2].
The finance study tests one prompted LLM on sentiment and other finance tasks
[arxiv:2401.11641 §4]. A fine-tuned Arabic encoder reports five-class sentiment
F1 0.75 on private telecom tweets [arxiv:2606.25495 tab. 4]. A position paper
argues that the field is less solved than its polarity benchmarks suggest, and
lists open problems without running any experiments [arxiv:2005.00357 §1].

## Variants

### Dictionary tools

Dictionary methods score text against word lists. VADER and SentiWordNet are
named as dictionary baselines for Twitter polarity, with their values not stated
[arxiv:2301.08826 §2.1]. TextBlob appears in a feedback analysis tool with recall
96.17%, precision 67.47% and F 79.30% [arxiv:2301.08826 §2.3.2]. Card:
[Review of NLP methods for education feedback](/papers/2301.08826).

Eight tools, from emoticon lookups and LIWC to SentiWordNet and SenticNet, were
compared on Twitter events and six labelled sets [arxiv:1406.0032 §2]
[arxiv:1406.0032 §3.1]. Coverage, the share of messages a tool labels at all,
ranged from about 4% to 95% on real events [arxiv:1406.0032 §1]. SentiWordNet
and SenticNet had the highest coverage, at 90% to 95% on the AirFrance and H1N1
events [arxiv:1406.0032 §4.1] [arxiv:1406.0032 fig. 1]. Emoticons had the best
average F, 0.846 over the labelled sets, but the lowest coverage
[arxiv:1406.0032 §4.3] [arxiv:1406.0032 tab. 5]. A combination of seven tools
reached 95% coverage with F 0.730 [arxiv:1406.0032 §5.1] [arxiv:1406.0032 tab. 5].
The rule that maps F to each tool's weight, and the rule for conflicting labels,
are not stated, so this combination cannot be rebuilt from the paper alone
[arxiv:1406.0032 §5.1]. Card:
[Comparing and Combining Sentiment Analysis Methods](/papers/1406.0032).

### Trained classifiers

A CNN on student feedback reports accuracy 87.95%, precision 87%, recall 78% and
F1 81% [arxiv:2301.08826 §2.4.5]. An LSTM on the same kind of feedback reports 92%
accuracy on positive text and 79% on negative text [arxiv:2301.08826 §2.4.5].
Neither entry gives a baseline value [arxiv:2301.08826 §2.4.5].

### Neural classifiers on tweets and reviews

A DNN, a CNN and an LSTM ("RNN" in the paper) are compared for binary polarity on
eight tweet and review sets. Each model is fed either word embeddings (WE) or
TF-IDF [arxiv:2006.03541 §4.3] [arxiv:2006.03541 tab. 2]. On Sentiment140, WE gives 0.8006 for the
CNN and 0.8282 for the LSTM, against 0.7669 and 0.5696 with TF-IDF
[arxiv:2006.03541 tab. 2]. The LSTM with WE beats the CNN with WE on six of the
eight sets. Book reviews (0.7663 for the CNN against 0.7335 for the LSTM) and
music reviews (0.7445 against 0.7310) are the exceptions [arxiv:2006.03541 tab. 2].
The paper reports one number per cell, with no seeds or spread [arxiv:2006.03541
tab. 2]. Card: [Sentiment Analysis Based on Deep Learning: A Comparative Study](/papers/2006.03541).

### Fine-tuned pretrained encoder on Arabic tweets

A MARBERT encoder, pretrained on dialectal and standard Arabic tweets, is
fine-tuned on more than 22,000 manually labelled customer tweets from one
telecom company, with five sentiment classes [arxiv:2606.25495 §5.1]
[arxiv:2606.25495 §4.2]. The best of six settings, with training batch size 96,
learning rate 2e-5, maximum length 256 and five epochs, reaches F1 0.75
[arxiv:2606.25495 tab. 4]. At batch size 96 the per-class values are 0.85
(negative), 0.92 (positive), 0.89 (neutral), 0.54 (sarcasm) and 0.56
(indeterminate). At batch size 16 the sarcasm value is 0.33
[arxiv:2606.25495 tab. 5]. The authors rejected over- and undersampling, and
raised the sarcasm examples from about 200 to about 1,400 by collecting more
tweets [arxiv:2606.25495 §5.3]. The data is held privately by the client, and
the paper's comparison table uses figures from other datasets
[arxiv:2606.25495 §5.1] [arxiv:2606.25495 tab. 2]. Card:
[Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model](/papers/2606.25495).

### Attention and embeddings for emotion

For emotion in student feedback, a plain LSTM reaches 86.27%, an LSTM with
attention 87.49%, and multi-head attention 90.03%. Multi-head attention with GloVe
and CoVe embeddings reaches 94.13% [arxiv:2301.08826 §2.1]. The review also
reports GloVe about 5–7% better than Word2Vec in a cited study, without the task
detail [arxiv:2301.08826 §2.1].

### Aspect-level opinions

A MOOC study reports F1 86.13% for the aspect category and 82.10% for the aspect's
sentiment [arxiv:2301.08826 §2.3.2]. Another reports 91% accuracy for aspect
extraction and 93% for sentiment detection [arxiv:2301.08826 §2.4.2]. The review
lists aspect-based sentiment analysis as a challenge of its own [arxiv:2301.08826 §3.5].
Card: [Review of NLP methods for education feedback](/papers/2301.08826).

### Joint sentiment and sarcasm

One GRU reads the sentence with attention pooling for each task. A neural tensor
network fuses the two task vectors before the sarcasm softmax, and each task has
its own two-class softmax [arxiv:1901.08014 §2] [arxiv:1901.08014 fig. 1]. On 994 sentences with
10-fold cross-validation, the best variant reaches average F 86.66. A simple
multi-task model reaches 84.37, and the cited state of the art 82.13
[arxiv:1901.08014 tab. 1]. For sentiment alone, the best variant reaches F 83.03
against 77.30 for the state of the art [arxiv:1901.08014 tab. 1]. The paper
reports that the sarcasm signal helps mainly when there is a clear sentiment shift
[arxiv:1901.08014 §4]. The baseline uses eye-movement features that this model
does not use [arxiv:1901.08014 §4], so the gap is not a like-for-like test. The
paper has no official code and does not state the learning rate, batch size or
epochs [arxiv:1901.08014 §2]. Card: [Sentiment and Sarcasm Classification with Multitask Learning](/papers/1901.08014).

### Sarcasm with features from sentiment, emotion and personality networks

Three CNNs are first trained on sentiment, emotion and personality corpora. Their
outputs are concatenated with the features of a baseline sarcasm CNN, and an SVM
makes the final decision [arxiv:1610.08815 §4] [arxiv:1610.08815 §5.2]. Word
vectors are word2vec and are trained along with the network [arxiv:1610.08815 §4.1].
On Dataset 1 the all-feature SVM reaches F1 97.71, against 97.60 for the baseline
CNN-SVM and 95.04 for the baseline CNN [arxiv:1610.08815 tab. 2]. On Dataset 2 the
figures are 94.80 against 92.32, and on Dataset 3, 93.30 against 92.20
[arxiv:1610.08815 tab. 2]. Adding only the sentiment features to the baseline features
moves Dataset 1 from 97.60 to 97.67 in the CNN-SVM setting [arxiv:1610.08815 tab. 2]. Trained on Dataset 3 and tested on
Dataset 1, the model reaches 76.78, against 53.02 for Ptacek et al. (2014) in the
authors' re-implementation [arxiv:1610.08815 tab. 3]. Training on Dataset 1 and
testing on Dataset 3 drops F1 to 33.05 [arxiv:1610.08815 §5.6]. All numbers are
F1 from 5-fold cross-validation, and the prior methods are the authors' own
re-implementations [arxiv:1610.08815 §5.2]. Card:
[A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks](/papers/1610.08815).

### Multimodal sentiment (text, audio and video)

MISA splits each modality's representation into a part shared across modalities
and a part private to one modality, then fuses both for the prediction
[arxiv:2005.03545 §3] [arxiv:2005.03545 tab. 4]. Inputs are word-aligned features from short
video utterances [arxiv:2005.03545 §3.1]. On CMU-MOSI with BERT text features, the
authors report MAE 0.783, correlation 0.761, binary accuracy 81.8% and 83.4% for
the two conventions, and 7-class accuracy 42.3% [arxiv:2005.03545 tab. 1]. On
CMU-MOSEI they report MAE 0.555, correlation 0.756 and binary accuracy 83.6% and
85.5% [arxiv:2005.03545 tab. 2]. Some baseline rows were copied from the cited
works rather than re-run [arxiv:2005.03545 §4.2]. Removing the language input
drops MOSI correlation to 0.041, while removing visual or audio input raises MAE
to 0.798 or 0.849 [arxiv:2005.03545 tab. 4]. Card: [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](/papers/2005.03545).

### Hierarchical fusion of text, audio and video

HFusion encodes each modality per utterance and fuses each pair of modalities
per dimension with learned weights. A GRU adds context across the utterances of
a clip, and the three pair vectors are fused again [arxiv:1806.06228 §3.1]
[arxiv:1806.06228 alg. 1]. The authors contrast this with early fusion, which
concatenates the unimodal vectors [arxiv:1806.06228 §3.1]. On CMU-MOSI without
context, the reported accuracy with the Poria et al. features is 74.6 for all
three modalities, against baselines of 73.5 and 71.2 [arxiv:1806.06228 tab. 2].
With this paper's own features the figure is 77.9, against 77.0 and 77.3 for the
baselines [arxiv:1806.06228 tab. 2]. With context (CHFusion) the reported MOSI
accuracy for all three modalities is 80.0, against 78.7 and 77.2 for the two
comparison methods [arxiv:1806.06228 tab. 3]. On IEMOCAP with four emotion classes,
the three-modality accuracy is 76.5 and F-score 76.8 [arxiv:1806.06228 tab. 3].
The gains are 1 to 2.4 points, and the number of runs, seeds and any spread are
not stated [arxiv:1806.06228 §4.4.2]. The authors' repository is MIT-licensed,
but it differs from the paper on several hyperparameters, and some unimodal
encoders are not in it, so reproduction needs care (see the card's Pitfalls).
Card: [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228).

### Canonical correlation across modalities

ICCN forms outer products between a BERT text vector and the LSTM-encoded audio
and video states, and passes each product through a 2D CNN. The two CNNs are
trained with a deep canonical correlation (CCA) loss so that their outputs
correlate, and the text vector is joined to both outputs for a small downstream
model [arxiv:1911.05544 §3] [arxiv:1911.05544 eq. 5]. On CMU-MOSI it reports
binary accuracy 83.07, F 83.02, MAE 0.862, 7-class accuracy 39.01 and correlation
0.714, the best of the compared methods on all five metrics [arxiv:1911.05544
tab. 1]. On CMU-MOSEI it is best on MAE (0.565) and 7-class accuracy (51.58), while
MFM is higher on binary accuracy, F and correlation [arxiv:1911.05544 tab. 1]. On
IEMOCAP it is best for three of its four emotions [arxiv:1911.05544 tab. 2]. Each
of its three ablations scores below the full model on every column of its
ablation table [arxiv:1911.05544 tab. 3]. Card:
[ICCN (Interaction Canonical Correlation Network)](/papers/1911.05544).

### Mutual-information terms (text, visual and acoustic)

MMIM trains a regressor on text, visual and acoustic inputs. Its loss is the
task loss plus two mutual-information (MI) terms, weighted by α and β
[arxiv:2109.00412 §3.6] [arxiv:2109.00412 eq. 13]. The input-level term is a
Barber-Agakov lower bound on the MI between the text and each other channel,
with an entropy estimate from a two-class Gaussian mixture
[arxiv:2109.00412 §3.4] [arxiv:2109.00412 eq. 2] [arxiv:2109.00412 eq. 7]. The
fusion-level term is a contrastive loss that asks the fused vector to predict
each channel's vector [arxiv:2109.00412 §3.5] [arxiv:2109.00412 eq. 11]. On
CMU-MOSEI it reports MAE 0.526, correlation 0.772 and 7-class accuracy 54.24,
against 0.530 and 0.765 for Self-MM in the same table [arxiv:2109.00412 tab. 2].
On CMU-MOSI it reports MAE 0.700 and correlation 0.800 [arxiv:2109.00412 tab. 2].
The paper says the gains are small on several metrics [arxiv:2109.00412 §4.3],
and on one MOSEI binary-accuracy column its 82.24 is below Self-MM's 82.81
[arxiv:2109.00412 tab. 2]. Removing all input-level MI terms raises MOSEI MAE to
0.541 and lowers correlation to 0.752 [arxiv:2109.00412 tab. 4]. Dropping the
memory of earlier batches from the entropy estimate gives NaN in every metric
[arxiv:2109.00412 tab. 4]. Some baseline rows were copied from other papers and
others were reproduced by the authors [arxiv:2109.00412 tab. 2]. The paper card's
"Paper vs code" section records that the released code enables the MI terms
only with a flag, and that it picks the reported epoch on test MAE. Card:
[Multimodal InfoMax](/papers/2109.00412).

### Conversation context

On MELD, a multi-party dataset from TV dialogue, DialogueRNN models the speakers
and the turns before the target utterance. For three-way sentiment, its text-only
version reaches weighted F 66.10, against 64.25 for a text-CNN baseline. Adding
audio reaches 67.56, the best value in the table [arxiv:1810.02508 tab. 13]. The
authors report about 1.4 points from adding audio to the text-only DialogueRNN
[arxiv:1810.02508 §4.3]. The test split has 2,610 utterances [arxiv:1810.02508 tab. 8].
Card: [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](/papers/1810.02508).

The survey of emotion recognition in conversation reports DialogueRNN, CMN and
ICON on IEMOCAP with six emotion classes. DialogueRNN reaches weighted F1 62.75,
against 58.54 for ICON and 56.13 for CMN [arxiv:1905.02947 tab. 3]. The ICON row
was re-run by the survey authors [arxiv:1905.02947 tab. 3]. For DialogueRNN, the
survey gives 47.5% accuracy on utterances whose emotion changes from the previous
one and 69.2% on utterances without a change. It does not name the dataset for
these two figures [arxiv:1905.02947 §4]. The survey says CMN, ICON, IANN and
DialogueRNN all perform poorly on shifted utterances [arxiv:1905.02947 §4]. Card:
[Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](/papers/1905.02947).

### Sentiment as a measurement tool (tone in generated text)

A study of ChatGPT and Alpaca reference letters scores each sentence for
formality, positivity and agency, using a sentiment classifier tuned on SST-2
for positivity [arxiv:2310.09219 §3.2.2] [arxiv:2310.09219 §4.3.4]. Here the
classifier is a measuring instrument for bias, not the object of study. In the
biography setting, positivity differs between men's and women's letters with t =
5.93 (p = 1.58e-09) for ChatGPT, and with t = 1.47 (p = 0.07) for Alpaca
[arxiv:2310.09219 tab. 4]. Scores from a classifier of this kind depend on its
checkpoint, so any reuse of such a measure should pin the model and report its
version. The card records that the default checkpoint is not pinned in the
released code. Card:
[Gender-bias study of ChatGPT and Alpaca reference letters](/papers/2310.09219).

### Prompted large language models

The finance study prompts GPT-4 with zero-shot or one-shot instructions and
reports accuracy 0.78 on the Financial PhraseBank three-class task and 0.79 on
FiQA sentiment [arxiv:2401.11641 tab. 2]. No baseline model is in the table
[arxiv:2401.11641 tab. 2]. The prompts are described in words only, and the model
snapshot and decoding settings are not stated [arxiv:2401.11641 §4.2]. Card:
[Revolutionizing Finance with LLMs: An Overview of Applications and Insights](/papers/2401.11641).

## Open problems (position paper)

This is a position paper, not a method, so it has no results table. Its
argument is that near-perfect scores on polarity benchmarks such as IMDB and
SST-2 hide unsolved problems [arxiv:2005.00357 §1] [arxiv:2005.00357 fig. 1].
The open areas it lists are aspect-level sentiment, multimodal sentiment,
conversational context, sentiment reasoning (who holds the opinion and what
causes it), domain adaptation, multilingual and code-mixed text, sarcasm,
sentiment-aware generation and bias [arxiv:2005.00357 §3.1]–[arxiv:2005.00357 §3.9].
It states that simple concatenation performs about as well as most multimodal
fusion methods, but gives no numbers for this [arxiv:2005.00357 §3.2.1]. It
also argues that a bag-of-words input may explain the gains of an adversarial
domain-adaptation method [arxiv:2005.00357 §3.5]. Both claims are the authors'
experience, not measured results in the paper. The paper defines no measurable
target for "true sentiment understanding" [arxiv:2005.00357 §1]. Its dataset
sizes and the Wikipedia contributor share it quotes are not checked against
their sources [arxiv:2005.00357 §3.7.1] [arxiv:2005.00357 §3.9.1]. Card:
[Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research](/papers/2005.00357).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Quick baseline, no labelled feedback | Dictionary tool such as TextBlob, VADER or SentiWordNet | The review reports these as baselines or tools; their values are partly not stated [arxiv:2301.08826 §2.1] [arxiv:2301.08826 §2.3.2] |
| Need to label as many messages as possible, no training | Combination of several lexicon tools | The combination reaches 95% coverage with F 0.730; single tools range from about 4% to 95% coverage, and Emoticons has the best average F (0.846) but the lowest coverage. The combination's weighting rule is not stated [arxiv:1406.0032 §1] [arxiv:1406.0032 §4.3] [arxiv:1406.0032 §5.1] [arxiv:1406.0032 tab. 5] |
| Labelled feedback is available | CNN or LSTM classifier | Reported accuracies of 87.95% (CNN) and 92% / 79% per class (LSTM), with no baselines given [arxiv:2301.08826 §2.4.5] |
| Labelled tweets or reviews, binary polarity, and training time matters | CNN with word embeddings | Embeddings beat TF-IDF for the CNN on all eight sets; the LSTM with embeddings is higher on six, but the paper's Sentiment140 GPU run takes 7 min 3 s for the CNN against 1 h 4 min 16 s for the LSTM. The authors call the CNN the best balance of accuracy and time [arxiv:2006.03541 §6] [arxiv:2006.03541 tab. 2] [arxiv:2006.03541 tab. 7] [arxiv:2006.03541 tab. 8] |
| Labelled tweets in a low-resource language, rare classes present | Fine-tune a pretrained encoder for that language (MARBERT for Arabic tweets) | F1 0.75 on private tweets with batch size 96; sarcasm 0.54 and indeterminate 0.56 are the weak classes [arxiv:2606.25495 tab. 4] [arxiv:2606.25495 tab. 5]. No baseline run on the same data is reported |
| Sentiment and sarcasm are both needed | Joint GRU with attention and tensor fusion | Average F 86.66 against 84.37 for a simple multi-task model and 83.75 for standalone classifiers, on one 994-sentence set with no variance reported [arxiv:1901.08014 tab. 1]. Training settings are missing, so expect to tune them yourself [arxiv:1901.08014 §2] |
| Opinions tied to one aspect of a course | Aspect-level model | Reported F1 of 86.13% (category) and 82.10% (sentiment) in one MOOC study [arxiv:2301.08826 §2.3.2]; the review flags this as a challenge [arxiv:2301.08826 §3.5] |
| Emotion labels instead of polarity | Attention over an LSTM, with GloVe or CoVe embeddings | In the cited study, multi-head attention beats a plain LSTM (90.03% vs 86.27%) and the best setting reaches 94.13% [arxiv:2301.08826 §2.1] |
| Speech or video comes with the text | Multimodal fusion such as MISA, MMIM, ICCN or HFusion | Reported on two benchmarks, with language carrying most of the signal: without it, MISA's MOSI correlation falls to 0.041 [arxiv:2005.03545 tab. 4]. MMIM's margin over Self-MM is small (MOSEI MAE 0.526 against 0.530) [arxiv:2109.00412 tab. 2]. ICCN is best on MOSI in all five metrics, but MFM is ahead on several MOSEI and IEMOCAP metrics [arxiv:1911.05544 tab. 1] [arxiv:1911.05544 tab. 2]. HFusion's gains are 1 to 2 points over its baselines, and its context variant reaches 80.0 on MOSI [arxiv:1806.06228 tab. 2] [arxiv:1806.06228 tab. 3]. Results are author-reported |
| Labelled turns inside a conversation, speakers known | Context model such as DialogueRNN | Weighted F 66.10 with text only, 67.56 with audio, against 64.25 for text-CNN [arxiv:1810.02508 tab. 13]. On IEMOCAP emotion labels, the survey reports weighted F1 62.75 for DialogueRNN against 58.54 for ICON [arxiv:1905.02947 tab. 3]. Expect weak results on emotion shifts [arxiv:1905.02947 §4] |
| No labelled data and no training budget, but a hosted LLM | Zero- or one-shot prompting, checked on your own labels first | GPT-4 reports 0.78 and 0.79 accuracy on two finance sets, with no baseline and no published prompts [arxiv:2401.11641 tab. 2] [arxiv:2401.11641 §4.2] |
| Sarcasm in tweets, labelled data available | Joint sentiment-and-sarcasm model, or a sarcasm CNN with transferred sentiment features | The joint model reports F 86.66 on one 994-sentence benchmark [arxiv:1901.08014 tab. 1]. The transferred-feature model reports F1 97.71 on Dataset 1, but only 76.78 when trained on Dataset 3 and tested on Dataset 1 [arxiv:1610.08815 tab. 2] [arxiv:1610.08815 tab. 3]. Test on data that differs from your training set before relying on either |
| Sarcasm or emoticons in the text, with no labelled data | Test on your own data | The review names sarcasm and emoticons as open challenges [arxiv:2301.08826 §3.2] [arxiv:2301.08826 §3.4]; no emoticon numbers are given |
| Measuring tone in generated text, for example bias audits | Sentiment classifier as one of several tone measures, with the checkpoint pinned | The study reports positivity gaps for ChatGPT letters (t = 5.93) and Alpaca letters (t = 1.47, p = 0.07) [arxiv:2310.09219 tab. 4]. The paper names SST-2 for the sentiment classifier [arxiv:2310.09219 §4.3.4], but the released code does not pin the checkpoint (see the card's Pitfalls) |

## Implementation sketch

The review gives no algorithm for sentiment analysis [arxiv:2301.08826 §2].
The sketch below follows the pipeline of one comparative study, which states its
cleaning steps, features and folds [arxiv:2006.03541 §4.2] [§4.3] [§4.4]. It is
ours, and the paper gives no code.

```pseudocode
# Binary sentiment with cross-validation: our sketch of the study pipeline
# Follows [arxiv:2006.03541 §4.2] [§4.3] [§4.4]; not the paper's code
# Input:  texts with polarity labels; Output: one accuracy per fold

def sentiment_scores(texts, labels, model_name):
    # Keep positive and negative samples only; map them to 1 and 0  [§4.1] [§4.3]
    texts, labels = binarise(texts, labels)
    # Clean each text: remove mentions, links and symbols; lower-case  [§4.3]
    cleaned = [preprocess(text) for text in texts]
    # Word ids with a vocabulary of 15,000 and length 40  [§4.4]
    token_ids = to_token_ids(cleaned, vocab=15000, max_len=40)
    scores = []
    # 10-fold cross-validation, one score per held-out fold  [§4.2]
    for fold in kfold(token_ids, labels, k=10):
        # Train on the training part; optimiser and learning rate not stated
        model = build_model(model_name)               # e.g. CNN or LSTM  [§4.4]
        train(model, fold.train, batch_size=4096)     # batch size  [§5]
        # Score the held-out part  [§4.2]
        scores.append(accuracy(model, fold.test))
    return scores
```

## Contradictions

No direct disagreement between the cited results appears on this page. The
accuracies, F1 values, weighted F scores, MAE and correlations come from different
datasets, languages, labels, tasks and baselines, and some are Twitter, e-commerce,
TV dialogue or finance data rather than student feedback
[arxiv:2301.08826 §2.1] [arxiv:2301.08826 §2.3.2] [arxiv:1810.02508 §3]
[arxiv:2005.03545 §4.1] [arxiv:2606.25495 §5.1]. Regression scores (MAE, correlation) and classification
scores (accuracy, F) cannot be ranked against each other. The review does not
claim that its numbers are comparable [arxiv:2301.08826 §2].

The conversation-emotion survey uses emotion labels, not polarity. Its MELD
weighted F1 of 57.03 for DialogueRNN covers seven emotion classes
[arxiv:1905.02947 tab. 5]. The MELD weighted F of 66.10 above covers three
sentiment classes [arxiv:1810.02508 tab. 13]. The two numbers measure different
tasks, so do not read one against the other. Its tables also mix results from
several earlier papers. Most of its comparisons are against DialogueRNN, whose
paper three of the four survey authors co-wrote [arxiv:1905.02947 §4].

MISA's numbers differ between its own paper and a later comparison table. MISA
reports MOSI MAE 0.783 and MOSEI MAE 0.555 [arxiv:2005.03545 tab. 1]
[arxiv:2005.03545 tab. 2]. The MMIM table lists MISA at MOSI MAE 0.804 and MOSEI
MAE 0.568, with different binary-accuracy values, and marks that row as taken
from the Self-MM paper rather than MISA's own [arxiv:2109.00412 tab. 2]. Both
sets are reported here as their papers give them. Use one table at a time, and
do not compare a MISA row from one paper with an MMIM row from the other. The
ICCN table uses yet another MOSI and MOSEI set of baseline rows
[arxiv:1911.05544 tab. 1], so the same caution applies.

ICCN's text calls it state of the art, but its own Table 1 shows MFM ahead on
several MOSEI and IEMOCAP metrics [arxiv:1911.05544 §6] [arxiv:1911.05544 tab. 1]
[arxiv:1911.05544 tab. 2]. The paper says hyperparameters are chosen on validation
performance [arxiv:1911.05544 §4]. The released MOSEI export instead keeps the run
with the lowest test MAE, and its best printed run (MAE 0.5693, correlation 0.7122)
does not match the MOSEI numbers in Table 1 (0.565 and 0.713)
[code: mosei_dcca_exp (2).html#L13616-L13619] [code: mosei_dcca_exp (2).html#L13661-L13667].
Use the paper's tables with that gap in mind.

Inside the lexicon comparison, the text and the table disagree. Section 5.1 says
the combined F of 0.730 is higher than every tool except Emoticons, but Table 5
lists SentiStrength at 0.765 [arxiv:1406.0032 §5.1] [arxiv:1406.0032 tab. 5]. The
SentiStrength row uses figures from its own paper rather than a run on these
datasets [arxiv:1406.0032 §4.3], so it is not directly comparable with the other
rows. The F values in Table 5 also do not equal the harmonic mean of the precision
and recall in the same table. For Emoticons that mean is about 0.86, against the
listed 0.846, and the paper does not say how F was averaged [arxiv:1406.0032 §3.2]
[arxiv:1406.0032 tab. 5]. Coverage numbers depend on the order in which tools are
combined: one passage reports above 92.75% after two tools, another 70% after two
[arxiv:1406.0032 §4.1] [arxiv:1406.0032 §5.1].

Inside the MARBERT paper, the abstract claims accuracy gains over existing
techniques. The paper reports F1, and its comparison table uses accuracies from
other datasets, so no baseline is run on the telecom data [arxiv:2606.25495 abstract]
[arxiv:2606.25495 tab. 2]. The abstract's total of 24,513 tweets also differs from
the sum of its per-class counts, 24,477 [arxiv:2606.25495 abstract]. The main text
gives "more than 22K" sentiment tweets [arxiv:2606.25495 §5.1]. Which set is the
training data is not stated.

Inside the sarcasm paper, the abstract describes a 3 to 4 percent gain over the
state of the art. The Table 1 values give differences of 3.32 (sarcasm F), 4.53
(average F) and 5.73 (sentiment F) points, by our arithmetic from the table
[arxiv:1901.08014 tab. 1]. The two do not match cleanly, so use the table numbers.

Inside the finance study, FiQA sentiment is described as a continuous score from
-1 to 1, yet Table 2 reports accuracy, and the paper does not say how scores were
turned into classes [arxiv:2401.11641 §4.1] [arxiv:2401.11641 tab. 2].

Inside the comparative study, the text gives two accounts of the embedding layer.
One says the vectors are first trained with Word2Vec, the other that they start
from random weights and are learned, and Figure 9 lists the embedding parameters
as non-trainable [arxiv:2006.03541 §4.4] [arxiv:2006.03541 fig. 9]. The paper does not say which
setup produced its results. Its conclusion also says the CNN "outperforms other
models", while its own table shows the LSTM with embeddings ahead on six of eight
sets [arxiv:2006.03541 §6] [arxiv:2006.03541 tab. 2]. The claim is best read as
the authors' judgement of accuracy against training time.

## Papers

- [Review of NLP methods for education feedback](/papers/2301.08826): a literature review of text methods for student feedback, covering sentiment annotation, aspect-level analysis and the main challenges.
- [Comparing and Combining Sentiment Analysis Methods](/papers/1406.0032): a comparison of eight polarity tools on coverage and agreement over Twitter events and six labelled sets, with a combined method built from seven of them.
- [A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks](/papers/1610.08815): a sarcasm CNN with features from pre-trained sentiment, emotion and personality networks, with author-reported F1 on three Twitter sets.
- [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228): a hierarchical fusion model over text, audio and video, with author-reported results on CMU-MOSI and IEMOCAP and a context variant.
- [Sentiment and Sarcasm Classification with Multitask Learning](/papers/1901.08014): one GRU with attention and tensor fusion that labels sentiment and sarcasm together, with author-reported results on a 994-sentence benchmark.
- [Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research](/papers/2005.00357): a position paper that traces the field's milestones and lists nine open directions, with no experiments.
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](/papers/2005.03545): a fusion model that splits each modality into shared and private parts, with author-reported results on CMU-MOSI and CMU-MOSEI.
- [ICCN (Interaction Canonical Correlation Network)](/papers/1911.05544): a model that correlates text-conditioned audio and video features with a deep CCA loss, with author-reported results on CMU-MOSI, CMU-MOSEI and IEMOCAP.
- [Multimodal InfoMax (MMIM)](/papers/2109.00412): a regressor trained with mutual-information bounds on text, visual and audio inputs, with author-reported results on CMU-MOSI and CMU-MOSEI.
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](/papers/1810.02508): a dialogue dataset with sentiment and emotion labels and baselines that use context and audio.
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](/papers/1905.02947): a survey of emotion recognition in conversation, with dataset statistics and author-collected results for CMN, ICON, IANN, DialogueRNN and HRLCE.
- [Gender-bias study of ChatGPT and Alpaca reference letters](/papers/2310.09219): a bias study of ChatGPT and Alpaca reference letters that uses sentiment as one tone measure, with author-reported gaps by gender.
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](/papers/2401.11641): a finance survey with a small author-run GPT-4 evaluation that includes sentiment labelling.
- [Sentiment Analysis Based on Deep Learning: A Comparative Study](/papers/2006.03541): a DNN, CNN and LSTM compared with word embeddings or TF-IDF on eight tweet and review sets, with author-reported accuracies from 10-fold cross-validation.
- [Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model](/papers/2606.25495): a MARBERT fine-tune for five-class sentiment on private telecom tweets, with per-class F1 and a note on imbalance handling.