CEM: Commonsense-aware Empathetic Response Generation

In plain words

A chatbot that only sees the words of a message tends to answer kindly but vaguely: "that is great" or "sorry to hear that". The authors argue that real empathy has two parts. One part is feeling what the other person feels. The other part is understanding the person's situation, such as what they want, what they need and what might happen next. A message like "I'm ready for a puppy" implies a goal the words do not state.

The method, called CEM, fills that gap with general common sense. For the user's last message, a separate language model writes down likely feelings, wants, needs, intentions and effects. The reply generator reads these notes alongside the conversation. It can then ask about the puppy rather than congratulate the user on passing an exam. A second part of the training nudges the model away from stock phrases that appear again and again in the training data.

The authors report more informative and more empathetic replies than earlier models on a standard empathetic chat dataset. Automatic scores cover five earlier models, and human raters compare against three of them.

TL;DR

• Adds commonsense to empathetic chat: COMET infers five relations about the last user turn, and the model fuses them into the reply encoder §3.2 §3.4.
• Affective (xReact) and cognitive (xWant, xNeed, xIntent, xEffect) streams are encoded separately, then fused with a sigmoid gate and an MLP §3.1-§3.4.
• Trained with three losses: NLL, emotion cross-entropy and a frequency-weighted diversity loss, with weights 1, 1 and 1.5 §3.6.
• On EmpatheticDialogues the author-reported PPL is 36.11 (best baseline 36.93) and Dist-1/Dist-2 are 0.66/2.99 (best baselines 0.47/2.10) tab. 2.
• Human pairwise tests sample 100 response pairs, each annotated by three crowd workers, against MoEL, MIME and EmpDG §4.4 tab. 3.

Problem

Empathetic dialogue systems mostly detect the user's emotion and reply to it. The authors argue that empathy also has a cognitive side, which is understanding the user's situation and the feelings implied by it, and that emotion detection alone misses this §1 §2.1. Users rarely state everything about their situation, so a system has to infer it from common sense §2.2.

Models trained on this dataset also tend to produce generic replies. The authors counted frequent trigrams in the training responses. Some trigrams, such as "That is a", "Oh no that" and "To hear that", appear in at least 67% of responses, and "I am so", "Sorry to hear" and "Wow that is" in at least 50% tab. 1. They argue that a model trained on this data learns to prefer these safe phrases §3.6.

Assumptions: the model is trained and tested on EmpatheticDialogues, a dataset of about 25k conversations with 32 emotion labels, split 8:1:1 §2.3 §4.2. The commonsense source is ATOMIC-2020 through a COMET model, which the authors say handles events that are not in ATOMIC itself §2.2.

Method

CEM is built on a standard Transformer encoder-decoder §3. Processing runs in an order of five blocks: first the dialogue is encoded, then commonsense is fetched and encoded, then the dialogue is enriched with it, then the streams are weighted and mixed, and finally the reply is decoded §3.

Context encoding §3.1. The dialogue history is concatenated, a [CLS] token is prepended, and each token gets the sum of a word embedding, a position embedding and a speaker-state embedding that tells the two parties apart. A context encoder produces H_CTX eq. 1.

Knowledge acquisition §3.2. Five relation tokens (xReact, xWant, xNeed, xIntent, xEffect) are appended to the last utterance. COMET generates five inferences per relation, and the five are concatenated into one sequence per relation. xAttr is dropped on purpose, because predicting it judges the other person. The xReact sequence goes through an affective encoder and the four cognitive sequences through a cognitive encoder eq. 2 eq. 3. xReact outputs are reduced by averaging over positions, and cognitive outputs by taking the first ([CLS]) state eq. 4 eq. 5. xReact inferences are usually emotion words, which is why averaging is used there.

Context refinement §3.3. Each knowledge vector is concatenated to every token of H_CTX, and each resulting stream is encoded. One affection-refined stream uses the xReact vector, and one cognition-refined stream per cognitive relation uses that relation's vector eq. 6-eq. 9. The [CLS] state of the affection-refined stream is classified into one of the emotion labels with a linear layer and a softmax eq. 10-eq. 12. An overview of the full model is in fig. 2.

Knowledge selection §3.4. The five refined streams are concatenated per token, which gives a width of 5d eq. 13 eq. 14. A sigmoid of this concatenation is multiplied element-wise with the concatenation itself, and an MLP with ReLU mixes the result back to width d eq. 15. The paper does not give the MLP's hidden width.

Response generation §3.5. The decoder generates the reply token by token, and its cross-attention reads the fused representation from eq. 15 instead of plain H_CTX eq. 16.

Training objective §3.6. The loss is a weighted sum of three terms: NLL of the gold reply eq. 17, emotion cross-entropy eq. 12 and a diversity loss eq. 20. The diversity loss is an NLL in which each vocabulary token gets a weight from its relative frequency, so frequent tokens count less eq. 18 eq. 19. The paper cites Frequency-Aware Cross-Entropy (FACE) for this scheme. The weights are γ1 = 1, γ2 = 1 and γ3 = 1.5 eq. 21.

Stated equations (LaTeX as printed in the paper; numbering follows the compiled PDF):

H_{CTX} = \mathbf{Enc}_{CTX}(\bm{E}_C) eq. 1

\bm{H}_{xReact} = \mathbf{Enc}_{Aff}(\bm{E}_{CS_{xReact}}) eq. 2

\bm{H}_{r} = \mathbf{Enc}_{Cog}(\bm{E}_{CS_{r}}) eq. 3

\bm{h}_{xReact} = \mathrm{Average}(\bm{H}_{xReact}) eq. 4

\bm{h}_{r} = \bm{H}_{r}[0] eq. 5

\bm{U}_{xReact}[i] = \bm{H}_{CTX}[i] \oplus \bm{h}_{xReact} eq. 6

\bm{U}_{r}[i] = \bm{H}_{CTX}[i] \oplus \bm{h}_{r} eq. 7

\bm{H}_{Aff} = \mathbf{Enc}_{CTX\text{-}Aff}(\bm{U}_{xReact}) eq. 8

\bm{H}_{Cog, r} = \mathbf{Enc}_{CTX\text{-}Cog}(\bm{U}_{r}) eq. 9

\bm{h}_{Aff} = \bm{H}_{Aff}[0] eq. 10

P_{emo} = \mathrm{Softmax}(\bm{W}_e \bm{h}_{Aff}) eq. 11

\mathcal{L}_{emo} = - \log(P_{emo}(e^*)) eq. 12

\bm{H}_{Cog}[i] = \mathop{\textstyle \bigoplus}_{r} \bm{H}_{Cog, r}[i] eq. 13

\bm{H}_{Refine}[i] = \bm{H}_{Aff}[i] \oplus \bm{H}_{Cog}[i] eq. 14

\widetilde{\bm{H}}_{CTX} = \mathbf{MLP}(\sigma(\bm{H}_{Refine}) \odot \bm{H}_{Refine}) eq. 15

P(y_t \mid y_{<t}, C) = \mathbf{Dec}(\bm{E}_{y_{<t}}, \widetilde{\bm{H}}_{CTX}) eq. 16

\mathcal{L}_{nll} = - \sum_{t=1}^{T} \log P(y_t \mid C, y_{<t}) eq. 17

RF_i = \frac{\mathrm{freq}(c_i)}{\sum_{j=1}^{V} \mathrm{freq}(c_i)} eq. 18 (denominator as printed; see Pitfalls)

w_i = a \times RF_i + 1 eq. 19 (with a = -(\max_j RF_j)^{-1} in the text §3.6)

\mathcal{L}_{div} = - \sum_{t=1}^{T} \sum_{i=1}^{V} w_i \, \delta_t(c_i) \log P(c_i \mid y_{<t}, C) eq. 20

\mathcal{L} = \gamma_1 \mathcal{L}_{nll} + \gamma_2 \mathcal{L}_{emo} + \gamma_3 \mathcal{L}_{div} eq. 21

The following sketch follows the paper's equations. The first block is the commonsense branch, the second the refinement and gating, and the third the loss. The implementation in the repository is a single model file that does the same steps model.py.

# Commonsense branch: encode the five COMET inference sets  [§3.2] [eq. 2-5]
# Input:  relation_sets: dict, relation name -> 5 inferences
# Output: react_vector (d,), cognition_vectors: 4 vectors of shape (d,)

def encode_commonsense(relation_sets):
    # Affective relation xReact: encode, then average over positions  [eq. 4]
    react_states = affect_encoder(embed(relation_sets["xReact"]))
    react_vector = mean(react_states, axis=position)
    # Cognitive relations: encode each, keep the first ([CLS]) state  [eq. 5]
    cognition_vectors = []
    for relation in ["xWant", "xNeed", "xIntent", "xEffect"]:
        cognition_states = cognition_encoder(embed(relation_sets[relation]))
        cognition_vectors.append(cognition_states[0])
    return react_vector, cognition_vectors

# Refinement, gating and mixing  [§3.3] [§3.4] [eq. 6-15]
# Input:  context (L, d); react_vector (d,); cognition_vectors (4 x (d,))
# Output: fused context (L, d) for decoder cross-attention

def refine_and_mix(context, react_vector, cognition_vectors):
    # Concatenate the xReact vector to every token, then encode  [eq. 6] [eq. 8]
    affect_input = concat_each_token(context, react_vector)
    affect_refined = affect_context_encoder(affect_input)
    # One cognition-refined stream per cognitive relation  [eq. 7] [eq. 9]
    cognition_refined = []
    for vector in cognition_vectors:
        cognition_input = concat_each_token(context, vector)
        cognition_refined.append(cognition_context_encoder(cognition_input))
    # Concatenate the affective and cognitive streams per token  [eq. 13-14]
    refined = concat_last_dim([affect_refined] + cognition_refined)  # (L, 5d)
    # Sigmoid gate, multiplied element-wise with the input  [eq. 15]
    gated = sigmoid(refined) * refined
    # MLP with ReLU maps the 5d features back to width d  [eq. 15]
    return mlp(gated)  # (L, d)

# Diversity weights from token frequencies  [§3.6] [eq. 18] [eq. 19]
# Input:  counts (V,): token frequencies
# Output: weights (V,), averaging to 1 over the vocabulary

def diversity_weights(counts):
    # Relative frequency of each token  [eq. 18] (see Pitfalls)
    relative = counts / sum(counts)
    # Negative slope, so frequent tokens get small weights  [§3.6]
    slope = -1 / max(relative)
    weights = slope * relative + 1
    # Rescale so the weights average to one  [§3.6]
    return weights / mean(weights)

# Training objective for one batch  [§3.6] [eq. 12] [eq. 17] [eq. 20] [eq. 21]
# Input:  logprobs (B, T, V); target (B, T); emotion_logits (B, q)
# Output: scalar loss

def total_loss(logprobs, target, emotion_logits, emotion_label, weights):
    # Negative log-likelihood of the gold reply  [eq. 17]
    nll_loss = nll(logprobs, target)
    # Cross-entropy of the affective emotion classifier  [eq. 12]
    emotion_loss = cross_entropy(emotion_logits, emotion_label)
    # Weighted NLL that penalises frequent tokens  [eq. 20]
    diversity_loss = weighted_nll(logprobs, target, weights)
    # Combine with gamma1 = 1, gamma2 = 1, gamma3 = 1.5  [§3.6] [eq. 21]
    return 1.0 * nll_loss + 1.0 * emotion_loss + 1.5 * diversity_loss

Hyperparameters

Values stated by the paper come first, then what the code uses where they differ. "not stated" means neither the paper nor the code gives a value that the paper uses.

Results

Author-reported. Setup: EmpatheticDialogues, 8:1:1 split, test batch size 1, decoding up to 30 steps, five baselines (Transformer, Multi-TRS, MoEL, MIME, EmpDG) and three ablations §4.1 §4.2. Perplexity, Dist-1, Dist-2 and emotion accuracy as the paper reports them tab. 2. The paper prints Dist values without a unit. The repository's evaluator prints them multiplied by 100 evaluate.py. Bold marks as in the paper.

Automatic evaluation tab. 2

Human evaluation tab. 3

Pairwise preference over 100 sampled response pairs per comparison, three crowd workers per pair, ties not shown. Aspects: coherence (Coh.), empathy (Emp.), informativeness (Inf.). κ is Fleiss' kappa. † marks p < 0.1 and ‡ marks p < 0.05 (sign test) tab. 3.

Case studies in tab. 4 show example replies. The paper describes them as qualitative and gives no metrics for them §4.6.

Paper vs code

Repository: https://github.com/Sahandfer/CEM at commit f45d00a3cb17f48d8ff8bf39923b694c61d4695e. The repository has no LICENSE file, and the README does not state a license, so the code is not quoted in this card. It is cited by path and line only. Other checks: the code runs the same five-stage pipeline as the paper, and the loss weights and the choice of affective versus cognitive encoders match model.py.

• Decoding length. Paper: 30 steps at test and inference §4.2. Code: the test and validation loops call decoding with 50 steps main.py main.py. The decoder default is 30 model.py.
• Learning rate. Paper: initial 1e-4, varied according to Vaswani et al. §4.2. Code: the optimiser is Noam-scheduled by default, with factor 1, model size 300 and warmup 8000 config.py model.py. The Noam rate is set on every step common.py. At step 8000 the peak is 300^(-1/2) × 8000^(-1/2) ≈ 6.5e-4 (inferred from the formula common.py). With the Noam schedule on (the default), the 1e-4 value is not used.
• Negative log-likelihood normalisation. Paper: sum over the reply tokens eq. 17. Code: the reply NLL is a mean over non-padding tokens model.py. Scale differs by the reply length, which the paper does not state.
• Diversity loss normalisation. Paper: no division eq. 20. Code: the sum is divided by the number of non-padding target tokens model.py.
• Frequency source. Paper: relative frequency from the training corpus §3.6. Code: counts from the model's own argmax predictions over training batches, accumulated across batches and with EOS excluded. The weights therefore change during training model.py model.py.
• Frequency formula. The paper's printed denominator for RF_i sums over freq(c_i), not over j eq. 18. Code divides by the total count model.py.
• xReact average. Paper: average of the xReact hidden states eq. 4. Code: a mean over every position with no padding mask model.py. Padding positions can enter the average. Whether that matters depends on the batch.
• MLP in the "w/o Cog" ablation. Paper: the MLP is replaced with a linear layer §4.5. Code: no linear layer at all. The affective refined context goes straight to the decoder model.py model.py.
• MLP width. Paper: width not stated §3.4. Code: 5d → 3d → d, no biases model.py.
• Gradient clipping. Paper: not stated. Code: max_grad_norm = 2.0 is parsed and stored config.py config.py, but no clipping call exists in the repository.
• Early stopping. Paper: "early stopping", no details §4.2. Code: validation every 2000 steps, checkpoints saved from step 12000 on, training stops after three validations without a lower dev PPL main.py main.py.
• Dist metric. Paper: no decoding stated for the table. Code: Dist-n is computed on the greedy outputs by default evaluate.py evaluate.py.
• Perplexity. Code: exp of the mean per-batch teacher-forced loss on the gold reply common.py model.py. The paper does not say how PPL is computed.

Pitfalls

• The text and the table disagree on perplexity. The text says CEM "achieves the lowest perplexity" §4.3. Table 2 has the ablation "w/o Div" lower at 35.60 tab. 2.
• Equation 18 is misprinted. The denominator sums freq(c_i) over j. Use the total count over all vocabulary tokens, as the code does model.py.
• Diversity weights need a count source. The paper says training corpus. The code uses the model's own predictions, accumulated over training. A reimplementation that counts gold tokens once will get different weights, and possibly different outputs model.py.
• COMET runs on every new last utterance. The code generates commonsense inside data preparation and caches the result in the preprocessed dataset. The README says the preprocessed file ships with the repository loader.py. Serving new dialogues needs the COMET model to produce five beams per relation for each new last turn (5 relations × 5 sequences, beam 5) comet.py.
• Text normalisation is in the code, not the paper. Contractions are expanded, and text is tokenised with NLTK before indexing constants.py loader.py. Results from another tokeniser are not comparable on Dist-n.
• Emotion accuracy is averaged per batch. Training and validation use batches of 16, while test uses batch size 1 common.py loader.py.
• Only the last utterance gets commonsense. Earlier turns are encoded without knowledge vectors. The paper says so §3.2, so an implementer should not add knowledge to every turn.
• Cognitive relations share one encoder. The four cognitive relations use the same cognition encoder and the same cognition-refinement encoder model.py model.py.
• Ablations are flags. Use woEMO, woCOG and woDiv in the code, which match the paper's three ablations config.py.
• Unstated values. Dropout, layer count, heads, vocabulary size and the test-time beam setting are not in the paper. The code's values are listed above and differ from what a reader might assume.

Lineage

• Builds on: Transformer (Vaswani et al., 2017) as the base architecture §3; EmpatheticDialogues (Rashkin et al., 2019) as the dataset and the Multi-TRS baseline §2.3; MIME (Majumder et al., 2020) for the context-refinement and importance-gate design, which the paper says it follows §3.3 §3.4; FACE (Jiang et al., 2019) for the frequency-aware loss §3.6; ATOMIC (Sap et al., 2019), COMET (Bosselut et al., 2019) and COMET-ATOMIC-2020 (Hwang et al., 2020) for commonsense generation §2.2.
• Compared against: Transformer, Multi-TRS (Rashkin et al., 2019), MoEL (Lin et al., 2019), MIME (Majumder et al., 2020) and EmpDG (Li et al., 2020) §4.1. No cards for these papers exist in this wiki, so they are named but not linked.
• Superseded by: not stated in this wiki.