--- kind: concept slug: emotion-recognition-in-conversation title: Emotion recognition in conversation summary: >- Label each utterance's emotion in a dialogue. Three text designs: DialogueRNN (per-speaker GRU states, attention over history), DialogueGCN (windowed relational graph over a BiGRU) and COSMIC (speaker states plus COMET features), all taking speaker labels. HFusion adds audio and video. CEM's emotion head sits in a reply generator. tags: [nlp, dialogue, emotion-recognition, graph-neural-network, commonsense, multimodal] papers: ["1811.00405", "1908.11540", "2010.02795", "2109.05739", "1810.02508", "1905.02947", "1806.06228", "2005.00357"] related: [relational-graph-convolution, graph-neural-network, recurrent-neural-network, commonsense-knowledge-injection, language-model-pretraining, attention-mechanism, dialogue-generation, sentiment-analysis, multimodal-fusion] status: draft updated_at: 2026-11-01 --- ## Overview Emotion recognition in conversation (ERC) assigns an emotion label to each utterance in a dialogue. Most of the page covers the text-only setting. One variant, HFusion, also uses audio and video. A single line such as "okay" cannot be labelled reliably alone; the label depends on the lines around it and on who said them [arxiv:1908.11540 §1]. A 2019 survey gives the same framing: the label depends on the utterance, its speaker and the preceding conversation [arxiv:1905.02947 §1]. The survey lists the hard cases as emotion shift, sarcasm and talk with more than two people. It also says no public dataset labels why an emotion occurs, which it calls emotion reasoning [arxiv:1905.02947 §2]. Earlier models ran a recurrent network over the utterances. The DialogueGCN authors argue that recurrent networks carry long-range context poorly, and that attention pooling over a recurrent state ignores which speaker wrote each context utterance and where it sits relative to the target [arxiv:1908.11540 §1]. Their working assumptions: the emotion of a speaker depends on the other party's utterances (inter-speaker influence) and on that speaker's own earlier utterances (self-dependency), and future utterances also carry signal [arxiv:1908.11540 §1, §3]. The DialogueRNN authors make the same point about speaker roles in dyadic talk, and model each party's state separately [arxiv:1811.00405 §1]. The COSMIC authors make a different case. Context-independent embeddings, and recurrent or graph models over them, miss contextual cues and the difference between related emotion classes. Their proposed missing signal is commonsense knowledge about mental states and cause and effect [arxiv:2010.02795 §1, §2, §3.3]. A 2020 position paper on sentiment analysis lists conversational context as an open problem. It names two dependencies in a conversation: a speaker's dependence on their own earlier emotion, and dependence on the other participants [arxiv:2005.00357 §3.3.2]. It also says LSTM-based sequence models have been ineffective at modelling sentiment flow [arxiv:2005.00357 §3.3.2]. The paper runs no experiments, so these are hypotheses here, not results. Four papers on wikillm do per-utterance ERC. The first three put a sequential model over the utterances, then a per-utterance classifier, and all take the speaker of each utterance as input. They differ in how context moves between utterances. HFusion also runs a sequential model over the utterances, but its input is text, audio and video, and its card does not state whether it uses speakers [arxiv:1806.06228 §3.3]. DialogueRNN keeps one recurrent state per speaker and a global state, and attends over earlier global states [arxiv:1811.00405 §3.3]. DialogueGCN adds a graph over a window of neighbouring utterances with typed edges [arxiv:1908.11540 §3.3]. COSMIC keeps no graph and updates per-speaker and per-listener recurrent states, with commonsense vectors as extra input [arxiv:2010.02795 §3.4]. The three text models report IEMOCAP results on different input features, so their numbers are not directly comparable. HFusion's IEMOCAP scores also use audio and video (see Contradictions). The benchmark used most here is MELD: about 13,000 utterances from the TV series Friends, with text, audio and video, and seven emotion labels [arxiv:1810.02508 §1] [arxiv:1810.02508 tab. 8]. DialogueGCN and COSMIC both report results on it [arxiv:1908.11540 tab. 4] [arxiv:2010.02795 tab. 4]. The MELD paper says the video clips come from a TV series and cites a fair-use check for short clips. Do not assume the clips can be redistributed [arxiv:1810.02508 §3]. Separately, CEM is an empathetic reply generator. Its emotion classifier outputs one label for the dialogue context, from a 32-label set, not a label for each utterance [arxiv:2109.05739 §3.3] [arxiv:2109.05739 tab. 2]. It is covered below as a variant with that limit stated, and its numbers are not comparable to the three above. ## Variants ### Per-speaker recurrent states with attention over history (DialogueRNN) The model keeps a state for each participant and one global state for the whole conversation [arxiv:1811.00405 §3.3]. When an utterance arrives, attention over the earlier global states gives a context vector [arxiv:1811.00405 eq. 2–4]. Only the speaker's own state is updated from that context [arxiv:1811.00405 eq. 5]. An emotion state then follows the speaker's updated state, and a ReLU layer and a softmax give the label [arxiv:1811.00405 eq. 8–11]. The listener states in the paper are left unchanged by default. The authors found an active listener update slightly worse [arxiv:1811.00405 §3.3.4] [arxiv:1811.00405 §5.2.1]. A bidirectional version runs two copies forwards and backwards and concatenates their emotion states. Its attention over all emotion states of a dialogue reads later turns [arxiv:1811.00405 §3.4.2–§3.4.4]. The plain model reads only earlier turns, by its design in §3.3. The paper does not test that streaming setting itself. Reported results, IEMOCAP text, weighted F1 [arxiv:1811.00405 tab. 2]: - DialogueRNN scores 59.89, against 56.13 for CMN, the prior model in its table. The authors put the gain at 3.76 points [arxiv:1811.00405 §5.1.1]. - The bidirectional model with emotion attention, BiDialogueRNN+Att, scores 62.75 [arxiv:1811.00405 tab. 2]. It reads the whole dialogue. - Without the party state the score is 55.56, and without the emotion GRU 57.38 [arxiv:1811.00405 tab. 4]. - On the four AVEC attributes, BiDialogueRNN+Att has the lowest mean absolute error of the methods in the table [arxiv:1811.00405 tab. 2]. The paper's only multi-party claim is that the method should scale. That is future work, and only dyadic datasets are tested [arxiv:1811.00405 §6]. Card: [DialogueRNN](/papers/1811.00405). ### Windowed relational graph over a sequential encoder (DialogueGCN) Each utterance is embedded by a CNN over pretrained word vectors, trained separately at the utterance level [arxiv:1908.11540 §3.2]. A bidirectional GRU then runs over the utterance sequence [arxiv:1908.11540 §3.3.1]. Each utterance becomes a node, linked to the 10 utterances before it and the 10 after, plus itself [arxiv:1908.11540 §3.3.2]. Edge weights come from a softmax over a bilinear score between the two nodes, normalised over each node's incoming edges [arxiv:1908.11540 eq. 1]. Each edge also has a relation type: the speaker of the source and the speaker of the target, combined with whether the source comes before the target. For two speakers that is 8 types, and in general 2M² for M speakers [arxiv:1908.11540 §3.3.2 / tab. 1]. Two graph layers follow. The first is relation-specific, in the style of R-GCN [arxiv:1908.11540 eq. 2]. The second drops the relation types and uses an unweighted sum over the window [arxiv:1908.11540 eq. 3]. A classifier concatenates the sequential and graph features, applies attention over the whole dialogue, and outputs a label per utterance [arxiv:1908.11540 eq. 4–9]. Reported results, text only, mean of 5 runs [arxiv:1908.11540 §5.1]: - IEMOCAP, weighted F1 64.18, against 62.75 for DialogueRNN, the prior best on that table [arxiv:1908.11540 tab. 3]. The DialogueRNN value is discussed in Contradictions. - MELD, F1 58.10, against 57.03 for DialogueRNN [arxiv:1908.11540 tab. 4]. - AVEC, mean absolute error on valence, arousal, expectancy and power is the lowest of the compared methods on all four [arxiv:1908.11540 tab. 4]. The authors also report that the window size, the speaker-pair relations and the sequential encoder each contribute. Measured on IEMOCAP: | Setting | F1 | Source | |---|---|---| | Full model, window (10, 10) | 64.18 | [arxiv:1908.11540 §5.2] | | Window (8, 8) | 62.48 | [arxiv:1908.11540 §5.2] | | Window (4, 4) | 59.41 | [arxiv:1908.11540 §5.2] | | Window (0, 0), self-edges only | 55.80 | [arxiv:1908.11540 §5.2] | | No sequential encoder | 56.71 | [arxiv:1908.11540 tab. 5] | | No graph (speaker-level) encoder | 55.30 | [arxiv:1908.11540 tab. 5] | | No speaker-pair relations, before/after kept | 61.03 | [arxiv:1908.11540 tab. 6] | | No before/after relations, speaker pairs kept | 62.52 | [arxiv:1908.11540 tab. 6] | Card: [DialogueGCN](/papers/1908.11540). ### Speaker-state recurrent model with commonsense features (COSMIC) Each utterance is first turned into a fixed vector by RoBERTa-Large, fine-tuned on the utterance emotion labels. The vector is the mean of the [CLS] states from the last four layers, 1024-d [arxiv:2010.02795 §3.2]. The encoder has 355M parameters [arxiv:2010.02795 §3.2], so this stage needs a fine-tuning run. Each utterance also gets five commonsense vectors from COMET, a model trained on the ATOMIC knowledge base. The COMET encoder is run on the utterance followed by one relation phrase, and each vector is 768-d. The five relations are the speaker's intent, the effect on the speaker, the reaction of the speaker, the effect on the others and the reaction of the others [arxiv:2010.02795 §3.3] [arxiv:2010.02795 tab. 1]. A set of GRU cells then runs over the conversation. One shared context state and one attention vector cover the whole dialogue. Each participant has an internal state, an external state and an intent state. The emotion state of the current utterance is built from the utterance, the speaker's states and the previous emotion state, and a softmax layer gives the label [arxiv:2010.02795 §3.4] [arxiv:2010.02795 tab. 2]. The paper's diagram uses bidirectional cells [arxiv:2010.02795 fig. 2]. Reported results, text only, weighted F1 on IEMOCAP [arxiv:2010.02795 tab. 4]: - 65.28, against 64.76 for RoBERTa DialogueRNN. The authors say the gain is smaller on IEMOCAP than on DailyDialog [arxiv:2010.02795 §5.2]. - Removing the COMET features from both speaker and listener sides gives 63.05. Removing only the speaker side gives 63.27, and only the listener side 65.05 [arxiv:2010.02795 tab. 4] [arxiv:2010.02795 §5.3]. - The authors call the commonsense gain "not very substantial", and report no significant gain from four other COMET relations [arxiv:2010.02795 §5.5]. - A case study on one IEMOCAP conversation shows the features helping with sudden shifts and with angry versus frustrated [arxiv:2010.02795 fig. 3] [arxiv:2010.02795 §5.4]. This is a single conversation. Card: [COSMIC](/papers/2010.02795). ### Hierarchical fusion of text, audio and video with context (HFusion) This variant adds audio and video. The authors argue that concatenating the unimodal vectors cannot filter conflicting or redundant information between modalities [arxiv:1806.06228 §3.1]. Each utterance is encoded per modality: text by a CNN over word2vec vectors, audio by openSMILE features and video by a 3D CNN [arxiv:1806.06228 §3.2]. A GRU per modality then runs over the utterances of a clip [arxiv:1806.06228 §3.3]. Each modality is mapped to a common width. Each pair of modalities is fused per dimension with learned weights and tanh, and each pair sequence gets its own GRU [arxiv:1806.06228 §3.4] [arxiv:1806.06228 eq. 1–3]. The three pair vectors are fused the same way, a last GRU runs over the result, and a softmax gives the label [arxiv:1806.06228 §3.4] [arxiv:1806.06228 §3.5]. Reported results on IEMOCAP with context (CHFusion), four classes [arxiv:1806.06228 tab. 3]: - Text, audio and video together give 76.5 accuracy and 76.8 F-score. The two baseline columns read 74.1 and 73.6. The card does not say which metric those baselines report. - Per class, on the same setup, neutral has the lowest F-score, 71.2, against 77.0 to 81.4 for the other three classes [arxiv:1806.06228 tab. 4]. - The context gain is shown on CMU-MOSI, a binary sentiment set, not on IEMOCAP. There the context model is 1 to 2 points above the same model without context, with no table [arxiv:1806.06228 §4.4.3]. The card gives no IEMOCAP result without context. The public code (MIT) trains the pair-level models first and then the trimodal model on their saved activations, although the paper describes one end-to-end method [arxiv:1806.06228 §2] [code: hfusion.py#L302-L348]. It does not include the unimodal encoders [code: hfusion.py#L144-L150]. Card: [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228). ### Dialogue-level emotion head inside a reply generator (CEM) CEM generates the next reply in an empathetic chat. Its emotion head is a side output, not a labeller of each utterance. Five commonsense relations are generated by COMET for the last user turn: xReact, xWant, xNeed, xIntent and xEffect [arxiv:2109.05739 §3.2]. The xReact vector goes through an affective encoder, and the other four through a cognitive encoder [arxiv:2109.05739 eq. 2-3]. The affective stream, with the xReact vector added to each context token, gives the single emotion label through a softmax [arxiv:2109.05739 eq. 8-12]. The emotion label set has 32 classes [arxiv:2109.05739 §2.3]. The fused context is read by the decoder through cross-attention [arxiv:2109.05739 eq. 15-16]. Training sums three terms: the reply NLL, the emotion cross-entropy, and a frequency-weighted diversity loss that gives frequent tokens less weight, with weights 1, 1 and 1.5 [arxiv:2109.05739 §3.6] [arxiv:2109.05739 eq. 21]. The diversity loss targets generic replies. The authors found some trigrams in at least half of the training replies [arxiv:2109.05739 tab. 1]. Reported results on EmpatheticDialogues, 8:1:1 split, author-reported [arxiv:2109.05739 tab. 2]: - Emotion accuracy 39.11, the highest in the table. The four baselines that report it range from 30.41 to 33.57, and the ablations from 33.76 to 38.82. - Perplexity 36.11, against 36.93 for the best baseline. The ablation without the diversity loss is lower, at 35.60, although the text calls the full model the lowest [arxiv:2109.05739 §4.3]. - Dist-1 0.66 and Dist-2 2.99, against best baselines of 0.47 and 2.10. The paper prints these without a unit. - In crowd pairwise tests on 100 response pairs per comparison, three raters per pair, empathy preference was 52.0% for CEM and 38.0% for MoEL. Preference for CEM over MIME and EmpDG was 50.3% and 54.3% [arxiv:2109.05739 tab. 3]. The sign test marks the MoEL and MIME results at p < 0.05 [arxiv:2109.05739 tab. 3]. These numbers measure reply quality and one emotion head on a different label set. They say nothing about per-utterance ERC. The COMET step runs on each new last turn, so a new dialogue needs a COMET model at inference. The released code produces these inferences during data preparation [code: src/utils/data/loader.py#L66-L93]. Card: [CEM](/papers/2109.05739). ## When to use which | Situation | Use | Why | |---|---|---| | Text only, two-party dialogue, only static word vectors available | DialogueGCN with window (10, 10) | Highest weighted F1 in its own comparison table, 64.18 against 62.75 for DialogueRNN, with GloVe-based utterance features [arxiv:1908.11540 tab. 3] [arxiv:1908.11540 §3.2]. The gain is about 1.4 points, from one benchmark and a mean of 5 runs. | | Text only, a 355M-parameter encoder can be fine-tuned, and the COMET resources can be obtained | COSMIC | 65.28 weighted F1 on IEMOCAP [arxiv:2010.02795 tab. 4]. Removing both commonsense sets costs 2.23 points on the same benchmark [arxiv:2010.02795 tab. 4]. | | Text only, no pretrained encoder or COMET step, and speaker labels available | DialogueRNN, bidirectional with emotion attention if future turns are allowed | 62.75 weighted F1 on IEMOCAP for the bidirectional attention model, against 59.89 for the plain model [arxiv:1811.00405 tab. 2]. The bidirectional model needs the whole dialogue [arxiv:1811.00405 §3.4.2]. | | Text plus audio and video, with context across utterances, and three unimodal encoders to build | HFusion with context (CHFusion) | Its IEMOCAP accuracy is 76.5 with all three modalities, against 74.1 and 73.6 for two baselines [arxiv:1806.06228 tab. 3]. Its context gain is shown on MOSI, not IEMOCAP [arxiv:1806.06228 §4.4.3]. The public code does not include the unimodal encoders [code: hfusion.py#L144-L150]. | | Multi-party dialogue with many speakers | Either graph or COSMIC, with the score of the card you choose | DialogueGCN's relation count grows as 2M² [arxiv:1908.11540 §3.3.2]. On MELD its gain over DialogueRNN is 1.07 F1, which the authors attribute to short utterances and many speakers who each say little [arxiv:1908.11540 tab. 4, §5.1]. COSMIC reports 65.21 on the 7-class MELD setting [arxiv:2010.02795 tab. 4]. Those numbers come from different input features and are not a like-for-like comparison. DialogueRNN has no multi-party result; its authors list scaling as future work [arxiv:1811.00405 §6]. The survey says DialogueRNN can handle multi-party talk at scale, which the DialogueRNN card does not support; see Contradictions [arxiv:1905.02947 §4]. | | Speaker labels unavailable | Not established by any card | DialogueRNN updates one state per speaker [arxiv:1811.00405 §3.3]. DialogueGCN takes known speakers as input [arxiv:1908.11540 §3.1], and dropping the speaker-pair relations costs 3.15 F1 against the full model [arxiv:1908.11540 tab. 6]. COSMIC also assumes each utterance comes with its speaker [arxiv:2010.02795 §3.1]. CEM tags each context token with a speaker-state embedding [arxiv:2109.05739 §3.1]. None of the ERC papers tests a speaker-free variant. The HFusion card does not state whether it uses speakers. | | Empathetic reply generation, where the reply should reflect the user's situation as well as their emotion | CEM | Its reported gains are on EmpatheticDialogues, with empathy preference of 52.0% against 38.0% for MoEL [arxiv:2109.05739 tab. 3]. It needs a COMET run on each new last turn [code: src/utils/data/loader.py#L66-L93]. Its emotion head is not a substitute for per-utterance ERC. | | Turn-by-turn prediction, later turns not yet available | Plain DialogueRNN (unidirectional, no emotion attention) | Its context attention covers earlier global states only [arxiv:1811.00405 §3.3]. Its IEMOCAP F1 is 59.89, against 62.75 for the bidirectional attention model, which reads the whole dialogue [arxiv:1811.00405 tab. 2]. The paper does not test a streaming setting, so the causal reading is an inference from the design. DialogueGCN's graph reads ten utterances after the target by design [arxiv:1908.11540 §3.3.2], and COSMIC's backward pass reverses the dialogue, so each prediction uses future utterances [code: COSMIC/erc-training/commonsense_model.py#L244-L256]. | | Dialogues longer than 110 utterances | Not DialogueGCN with the public code as released | The public code sizes its attention layer by a maximum length of 110 [code: DialogueGCN/train_IEMOCAP.py#L253]. The paper states no such limit. | | Tight compute budget | DialogueGCN with a smaller window, e.g. (4, 4) | F1 falls to 59.41 at (4, 4) against 64.18 at (10, 10) [arxiv:1908.11540 §5.2]. Edges per dialogue grow with the window [arxiv:1908.11540 §3.3.2]. COSMIC adds two offline feature stages, RoBERTa fine-tuning and COMET encoding [arxiv:2010.02795 §3.2] [arxiv:2010.02795 §3.3]. DialogueRNN's cost grows with dialogue length, since attention runs over the whole history [arxiv:1811.00405 §3.3.3]. The papers give no timing or memory figures, so their cost is not compared here. | ## Implementation sketch Common core of the windowed graph variant, language-agnostic. The sequential encoder, the window and the relation ids are the parts a variant would change. The COSMIC variant keeps the sequential step but replaces the graph with per-speaker recurrent states and adds COMET vectors as input; its per-utterance update is in the card [arxiv:2010.02795 §3.4]. DialogueRNN also uses per-speaker states, without a graph; its recurrence is in the card [arxiv:1811.00405 §3.3]. ```pseudocode # Input: utterance_vecs (N, D), speaker_of (N,), window p = f = 10 # 1. Speaker-agnostic sequential context [arxiv:1908.11540 §3.3.1] g = bigru(utterance_vecs) # (N, 2 * D_e) # 2. Windowed directed edges, including the self-loop [§3.3.2] edges = [(i, j) for i in range(N) for j in range(max(0, i - p), min(N, i + f + 1))] # 3. Relation id: speaker pair x (source before target?) [tab. 1] def relation(i, j): return rel_index(speaker_of[i], speaker_of[j], i < j) # 4. Edge weights: softmax of bilinear scores over each window [eq. 1] alpha = softmax_over_window(g, edges, W_e) # 5. First layer: relation-specific, weighted [eq. 2] h1[i] = relu(alpha[i, i] * W0 @ g[i] + sum(alpha[i, j] / c[i, r] * W_r[r] @ g[j] for (i2, j) in edges if i2 == i for r in [relation(i, j)])) # 6. Second layer: unweighted, relation-free [eq. 3] h2[i] = relu(W0_2 @ h1[i] + sum(W2 @ h1[j] for (i2, j) in edges if i2 == i and j != i)) # 7. Classifier on [g_i, h2_i] with attention over the dialogue [eq. 4-8] labels = classify(attend_over_dialogue([g, h2])) ``` Where the public code differs from the paper (card's *Paper vs code* section), as stated in [code: DialogueGCN/model.py#L540-L568] and [code: DialogueGCN/model.py#L796-L807]: - The edge score is a learned vector per absolute position, not the bilinear form of eq. 1. - The graph layers have no activation, although the paper writes σ in both. - The sequential encoder is a 2-layer LSTM, not a GRU, in the default script and the README command. - The windowed edges are built in Python loops per batch, which is slow for large batches [code: DialogueGCN/model.py#L676-L705]. ## Contradictions - **Reported versus reproduced IEMOCAP F1 (DialogueGCN).** The paper reports 64.18 [arxiv:1908.11540 tab. 3]. The README reports 64.67 and 64.44 on two machines, with a different configuration: LSTM base, learning rate 3e-4, dropout 0.4, L2 set to zero, nodal attention and class weights on (the README of the official repository). The README attributes the spread to nondeterministic CUDA atomics. Neither side is resolved here. - **Model selection (DialogueGCN).** The paper describes a validation split of 10% and a mean of 5 runs [arxiv:1908.11540 tab. 2, §5.1]. The public script uses no validation split and prints the maximum test F1 over all epochs [code: DialogueGCN/train_IEMOCAP.py#L322-L359]. The reported IEMOCAP numbers may therefore be optimistic. - **Model selection (DialogueRNN).** The paper does not say which epoch it reports. The IEMOCAP script evaluates on the test split every epoch and keeps the epoch with the lowest test loss [code: DialogueRNN/train_IEMOCAP.py#L194-L196]. The script sets the validation split to zero, so the reported numbers may be optimistic, as with DialogueGCN. - **Loss weighting (DialogueRNN).** The paper's loss is unweighted cross-entropy with an explicit L2 term [arxiv:1811.00405 §3.3.7]. The IEMOCAP script applies hard-coded per-class weights [code: DialogueRNN/train_IEMOCAP.py#L163-L172], and the paper does not explain them. Accuracy and F1 will differ between the two. - **Model selection (COSMIC).** The table caption says the scores are taken at the best validation score [arxiv:2010.02795 tab. 4]. The IEMOCAP script reports two test scores, one at the best validation loss and one at the best validation weighted F1 [code: COSMIC/erc-training/train_iemocap.py#L230-L237]. The paper does not say which one it reports. - **Run count (COSMIC).** The same table caption says "five runs" and then "10 runs" for two rows [arxiv:2010.02795 tab. 4]. The paper does not resolve this. - **DialogueRNN numbers across papers.** The DialogueGCN table gives DialogueRNN 62.75 on IEMOCAP [arxiv:1908.11540 tab. 3]. The DialogueRNN card lists 62.75 for its bidirectional attention model and 59.89 for the plain model [arxiv:1811.00405 tab. 2]. The COSMIC table gives DialogueRNN 62.57, a GloVe-based row [arxiv:2010.02795 tab. 4]. The cards cannot say which DialogueRNN variant each table reports. The two papers also use different utterance features (a GloVe CNN in DialogueGCN, RoBERTa-Large in COSMIC), so the top scores, 64.18 and 65.28, are not directly comparable. The MELD DialogueRNN text-only value, 57.03, is the same in the DialogueGCN and COSMIC tables [arxiv:1908.11540 tab. 4] [arxiv:2010.02795 tab. 4] and matches the MELD paper's text-only row [arxiv:1810.02508 tab. 11]. The MELD paper's best 7-way score, 60.25, uses text and audio, so it is not comparable with the text-only numbers here [arxiv:1810.02508 tab. 11]. - **Hyperparameter values (DialogueGCN).** The paper does not state the learning rate, dropout or L2 weight. The script defaults (1e-4, 0.5, 1e-5) and the README values (3e-4, 0.4, 0) disagree [code: DialogueGCN/train_IEMOCAP.py#L202-L208]. - **Which COSMIC setting gives Table 4.** The paper describes listener updates in its equations [arxiv:2010.02795 §3.4], but the code turns them off by default [code: COSMIC/erc-training/train_iemocap.py#L120]. The paper does not say which setting produced Table 4. - **Label count (DialogueGCN).** The problem definition lists 8 emotion labels [arxiv:1908.11540 §3.1]. IEMOCAP uses 6 and MELD 7 [arxiv:1908.11540 §4.1] [arxiv:1810.02508 §3.2]. HFusion uses four IEMOCAP classes, happy, sad, neutral and anger [arxiv:1806.06228 §4.4.2] [arxiv:1806.06228 tab. 4]. Set the class count per dataset. - **Multi-party claim (DialogueRNN).** The survey says DialogueRNN "can handle multi-party conversations at scale", unlike CMN and ICON [arxiv:1905.02947 §4]. The DialogueRNN paper says scaling to more than two parties is future work and that only dyadic datasets were tested [arxiv:1811.00405 §6]. The survey gives no test of its own, so the paper's statement is the one this page relies on. - **Survey numbers are not independent.** The survey's DialogueRNN rows repeat the original paper's values: 62.75 weighted F1 on IEMOCAP [arxiv:1905.02947 tab. 3] and 57.03 on MELD [arxiv:1905.02947 tab. 5]. Three of the survey's four authors also wrote DialogueRNN, according to the two papers' author lists. Cite the original card for these numbers, not the survey. - **Metric and label set (CEM against the ERC papers).** CEM's 39.11 is accuracy over 32 EmpatheticDialogues labels, one per dialogue [arxiv:2109.05739 tab. 2] [arxiv:2109.05739 §2.3]. The ERC papers report weighted F1 over IEMOCAP and MELD labels [arxiv:1908.11540 tab. 3] [arxiv:2010.02795 tab. 4] [arxiv:1811.00405 tab. 2]. The three figures do not measure the same task. - **Metric and input (HFusion against the text models).** HFusion's IEMOCAP scores are accuracy and F-score, from text, audio and video [arxiv:1806.06228 tab. 3]. The text models report weighted F1 from text alone. HFusion's 76.5 accuracy is therefore not comparable with their 62.75 to 65.28. - **IEMOCAP counts (HFusion).** §4.1.2 gives 1083 angry, 1630 happy, 1083 sad and 1683 neutral "videos". The IEMOCAP rows of tab. 1 sum to different counts [arxiv:1806.06228 §4.1.2] [arxiv:1806.06228 tab. 1]. The paper does not resolve this. - **Sequence models for sentiment flow.** The position paper says LSTM-based sequence models have been ineffective at modelling sentiment flow [arxiv:2005.00357 §3.3.2]. The three text models and HFusion all run gated recurrent encoders, and each reports gains over earlier work on its own benchmark [arxiv:1811.00405 tab. 2] [arxiv:1908.11540 tab. 3] [arxiv:2010.02795 tab. 4]. The position paper gives no experiment, so the cards cannot settle the disagreement. - **Concatenation in multimodal fusion.** The position paper says simple concatenation performs on par with most fusion methods, from the authors' experience and without numbers [arxiv:2005.00357 §3.2.1]. HFusion argues that concatenating unimodal vectors cannot filter conflicting or redundant information between modalities [arxiv:1806.06228 §3.1]. Neither side gives a controlled test here. - **MELD size and annotation.** The position paper calls MELD too small, with weak inter-annotator agreement [arxiv:2005.00357 §3.2.2]. The MELD card is the source for the dataset's own size and labels [arxiv:1810.02508 §1]. The position paper gives no agreement figure, so this claim is unquantified. - **Ablation rounding (DialogueGCN).** The text gives 36.7 for the no-encoder ablation, and the table gives 36.75 [arxiv:1908.11540 §5.3 / tab. 5]. This is a rounding difference inside the same paper. ## Papers - [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](/papers/1905.02947): a 2019 survey of ERC challenges, six public datasets and recent models. It proposes no method, and its DialogueRNN numbers repeat the original paper. - [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](/papers/1811.00405): per-speaker GRU states, a global state and an emotion GRU, with attention over earlier global states; bidirectional and attention variants for the whole dialogue. - [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](/papers/1908.11540): a windowed relational graph over a BiGRU encoder, with speaker-pair and before/after edge types, for text-only ERC. - [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](/papers/2010.02795): RoBERTa-Large utterance features plus five COMET commonsense vectors, tracked by speaker, listener, intent and emotion GRU states. - [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228): text, audio and video fused in two stages per utterance clip, with a GRU for context at each stage; the one multimodal variant here. - [CEM: Commonsense-aware Empathetic Response Generation](/papers/2109.05739): COMET commonsense on the last user turn, fused into a Transformer reply generator, with a dialogue-level emotion head. - [Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research](/papers/2005.00357): a 2020 position paper with no experiments; its conversational-context and multimodal sections are cited above as open problems. - [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](/papers/1810.02508): about 13,000 utterances from a TV series with text, audio and video, labelled with seven emotions and three sentiments.