Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research

In plain words

Sentiment analysis means working out how someone feels from what they wrote or said: positive, negative or neutral, and about what. Many researchers now assume the field is finished, because the usual benchmarks for the simple version of the task score close to perfect. This paper says that assumption is wrong. Those benchmarks test shallow classification. They say little about why a person feels something, which part of a sentence the feeling is aimed at, or how sarcasm, mixed languages or a speaker's earlier words change the meaning.

The paper tells the story of the field in broad strokes. It starts with early systems that counted words from lists, moves through statistical machine learning, and ends with today's large pretrained language models. It then points to harder problems that are still open: tying opinions to specific parts of a product, reading tone from voice and face as well as text, understanding a conversation, finding the reason behind a feeling, working in new domains and languages, catching sarcasm, generating emotional text, and avoiding unfair bias.

The authors run no experiments of their own. Their case is an argument supported by examples, plus a sketch of how a speaker's state, intent and emotion might shape what they say. Read it as a map of unsolved problems, not as a method to copy.

TL;DR

• The paper says it is not intended as a survey of the field; it covers a selection of milestones and open directions §1.
• Its thesis is that sentiment analysis is treated as saturated because benchmarks for polarity classification have near-perfect scores §1 fig. 1.
• It traces the field from rule-based and lexicon methods, through machine learning and deep learning, to contextual transformer encoders §2.3.
• It proposes nine open areas, each with example sentences, dataset gaps and suggested directions §3.1–§3.9.
• It contains no models, hyperparameters or new results, and no official code §1.

Problem

The authors argue that the field looks mature only because of heavy work on polarity classification (positive, negative or neutral) and on a few benchmarks. On IMDB and SST-2, many modern methods reach near-perfect scores §1 fig. 1. They argue this hides under-explored problems: implicit sentiment, aspect-level and multimodal analysis, the role of context, sentiment reasoning, and bias §1 §3.

What fails without these pieces, according to the paper: a classifier that labels a sentence in isolation misses sentiment that depends on the preceding turn, on sarcasm, on a second aspect in the same sentence, or on a modality other than text §3.1.2 §3.2 §3.3.2 §3.7.1.

Assumptions and gaps:

• The goal "true sentiment understanding" is used throughout but never defined as a measurable target §1 §4.
• The paper does not say which metric or dataset would show that the goal has been reached.

Method

The paper has three parts. Section 2 summarises the history, by granularity §2.2 and by method family §2.3. Section 3 gives nine open-direction subsections, each with claims, examples and gaps. Section 3.8.1 offers a conceptual model of one speaker's turn fig. 13. No step defines a function or an algorithm that could be run.

Historical milestones §2:

1. Early work treated subjectivity in sentences as the object of study. Research became active around 2000, when opinionated online text became available §2.1.
2. Granularity levels: document-level, sentence-level, aspect-level and phrase-level §2.2.
3. Rule-based methods used valence shifters and rules for how sentiment composes. Lexicons were built by hand, automatically or semi-supervised. Lexicons named: SentiWordNet, SO-CAL, SCL-OPP, SCL-NMA and the NRC lexicon §2.3.
4. Machine learning used SVMs, Naive Bayes, nearest-neighbour methods, bag-of-words, lexicon and part-of-speech features §2.3.
5. Deep learning used a recursive tensor network trained on a fully labelled treebank, CNNs, RNNs and attention. Transformer encoders (BERT, XLNet, RoBERTa) reached state-of-the-art results on several datasets §2.3.
6. Sentiment-aware word embeddings put sentiment information into the training loss. The paper says promising progress has been limited §2.3.
7. Micro-blogs such as Twitter are a separate setting: short text, slang, emoticons and sarcasm §2.4.

Open directions §3:

• Aspect-based sentiment (ABSA). The two subtasks are aspect extraction and aspect-level polarity §3.1. Open items: (a) automatic categorisation of aspect terms with entity linking over a semantic graph such as Probase, which the authors suggest §3.1.1 fig. 5; (b) implicit aspect sentiment, using the "bill" example and the term "opinion-oriented implicatures" §3.1.2; (c) joint extraction of aspects and polarity, framed as sequence labelling §3.1.3; (d) relations between aspects in one sentence §3.1.4; (e) small datasets, with Amazon product reviews and SemEval-2016 named as examples §3.1.5.
• Multimodal sentiment. Fusion methods named: multiple kernel learning, tensor fusion and memory networks §3.2. The authors report that, in their experience, simple concatenation performs on par with most of these methods §3.2.1. Datasets MOSI, MOSEI and MELD are called too small, with weak inter-annotator agreement §3.2.2. Per-modality labels are proposed §3.2.3 fig. 6.
• Contextual sentiment. Topic changes word polarity, as with "big house" §3.3.1. Word-sense disambiguation matters, with the "carried" example §3.3.1. In conversation, the paper names two dependencies: self-dependency (emotional inertia) and inter-personal dependency. Distant context is said to matter less than local context §3.3.2 fig. 7 fig. 8. The authors say LSTM-based sequence models have been ineffective at modelling sentiment flow, and suggest logic rules, finite-state transducers, belief propagation and a gate that decides when to use context §3.3.2. User, cultural and situational context is treated in §3.3.3. Commonsense knowledge graphs are suggested as a source §3.3.4 fig. 9.
• Sentiment reasoning. "Who" is the opinion holder, which need not be the speaker §3.4.1. "Why" is the stimulus or cause of the sentiment §3.4.2 fig. 10. The paper says no dataset has the annotations this needs and that a taxonomy for reasoning is hard to define §3.4.2.
• Domain adaptation. Existing approaches use domain-dependent embeddings, lexicons, pivot words, shared deep representations and adversarial domain-invariant learning §3.5. The paper argues that bag-of-words input can account for an adversarial method's gains, and that pretrained embeddings may carry the gains instead §3.5. Suggested: external knowledge graphs §3.5.1 fig. 11; multi-source and multi-target setups §3.5.2.
• Multilingual sentiment. Few curated lexicons exist for Hindi, French and Arabic §3.6.1. Code-mixed text (Hindi in Roman script within English) is hard for rule and deep methods §3.6.2. Machine translation may lose sentiment for contronyms, negation, diacritics and idioms, per a cited study of Arabic-to-English translation. Suggested fix: integrate sentiment into the translation encoder §3.6.3.
• Sarcasm. Context helps: authorial context (the author's past posts) and conversational context §3.7.1. MUStARD is a multimodal sarcasm set described as having about 500 instances §3.7.1 fig. 12. Annotation of perceived sarcasm has low agreement. Writers' own labels (iSarcasm) are proposed as a fix, and best-worst scaling as another §3.7.2. Target identification (who or what is mocked) is a separate task §3.7.3. Figurative-to-literal conversion and generation of sarcasm are open §3.7.4. Irony and humour are linked to sentiment §3.7.5.
• Sentiment-aware generation. Three areas. (a) Conditional generative models: disentangled latent codes with VAEs, GANs or seq2seq models, and attribute-to-text translation §3.8.1. The paper sketches a conceptual model with variables for topic, personality, background knowledge, speaker state, intent, emotion, utterances and sensory inputs §3.8.1 fig. 13. Aspect-level summarisation is an application fig. 14. (b) Dialogue: determine the response's sentiment, then generate it §3.8.2. The authors report that emotional response models give generic, emotionally incoherent replies, and that automatic evaluation remains hard §3.8.2. (c) Style transfer: flip sentiment by replacing sentiment-bearing words. Rule-based and adversarial methods exist, and non-parallel data is a problem §3.8.3.
• Bias. Three sources: word embeddings (a cited survey finds under 15% of Wikipedia contributors are women), model architecture that conditions on demographic metadata, and skewed training data §3.9.1. The Equity Evaluation Corpus is described as 8400 templated sentences, 7 templates by 60 names by 4 emotions by 5 words §3.9.2. De-biasing methods named: counterfactual word substitution used for training, regularisation, and masking of bias-bearing terms. The authors note few de-biasing works exist, and that BERT's context dependence makes bias harder to measure than in static embeddings §3.9.3.

Pseudocode for the conceptual turn model in §3.8.1 (the paper gives no functional forms, so the function names are placeholders):

# Conceptual turn generator; no functional forms given  [§3.8.1] [fig. 13]
# Input:  history (utterances so far), background (B), personality,
#         prev_intent (intent at turn t-2), sensory_inputs (Q)
# Output: utterance text
def generate_turn(history, background, personality, prev_intent,
                  sensory_inputs):
    # 1. Speaker state S: latent memory built by cognition from the
    #    history, background B and inter-personal ties  [§3.8.1]
    state = cognition(history, background)
    # 2. Intent I: from the current state, personality and the intent
    #    two turns earlier  [§3.8.1]
    intent = form_intent(state, personality, prev_intent)
    # 3. Emotion E: from P, S, I and Q. Q alone can drive emotion before
    #    cognition ("affective primacy")  [§3.8.1]
    emotion = affect(personality, state, intent, sensory_inputs)
    # 4. Utterance: intent, state and emotion together give the text
    #    [§3.8.1]
    return realise(intent, state, emotion)

Key equations: none. The source has no numbered equations.

Hyperparameters

Not applicable. The paper trains no model and gives no learning rates, batch sizes, architectures or training schedules §1 §3.

Results

No experiments. The paper reports no model results of its own. Author-reported figures and dataset facts that appear in the text:

Paper vs code

No official code.

The paper links a curated list of sentiment papers at github.com/declare-lab/awesome-sentiment-analysis §1. That is a reading list, not code, so it is not cloned and gives no method details. There is no project page with code.

Pitfalls

• Not an experiment. Every claim about where the field stands is an argument. Claims such as "simple concatenation performs at par" §3.2.1, "LSTMs have been ineffectual" for sentiment flow §3.3.2, and "entity linking should perform reasonably" §3.1.1 rest on the authors' experience and have no numbers in the text. Treat them as hypotheses.
• Saturation claim rests on a figure. The main thesis relies on fig. 1, whose caption gives no values, no query date and no exact method. The figure is not reproduced here.
• Dated. The text is written around 2020 and calls BERT, XLNet and RoBERTa state of the art §2.3. The source says the paper will be updated, and the card records the LaTeX source fetched with the arXiv v5 metadata (updated 2020-11-16). Check the current arXiv version before relying on any "recent" claim.
• No defined goal. "True sentiment understanding" §1 §4 is the central aim but is not defined or measured. An implementer cannot test against it.
• Conceptual model is unimplemented. §3.8.1 and fig. 13 name variables and their dependencies but give no functions, parameters or training signal. The pseudocode above only restates the text's dependencies.
• Cited works are described only as this paper describes them. Results attributed to other authors (e.g. the contronym and negation translation problem §3.6.3, the Wikipedia share §3.9.1, the MUStARD size §3.7.1) are not checked against those sources. No cards exist for them.
• Definitions vary. The paper uses "aspect" for topic, entity or attribute, also known as opinion target §3.1, footnote. Sentiment analysis and opinion mining are treated as synonyms §2.
• Figure-based and text-based claims differ. Figure 2's bar lengths are said to show "present popularity" fig. 2. Treat them as the authors' impression, not as a measured quantity.

Lineage

• Builds on: none stated. The paper points readers to prior detailed reviews of the machine-learning era by Liu §2.3.
• Compared against: none. The paper compares no methods numerically.
• Superseded by: not known from content in this wiki.