INSPIRED: Toward Sociable Recommendation Dialog Systems

In plain words

When a friend recommends a movie, they rarely just name a title. They give their own opinion, say they liked it too, cheer the other person on, or explain why it is credible. This paper asks whether those social moves make a recommendation more likely to land, and whether a chatbot can learn to make them.

The authors first collected about a thousand text chats between pairs of online workers, one playing the person recommending a movie and one looking for something to watch. Each recommender message was labelled with the social move it made. Recommendations that the other person went on to accept and enjoy were linked with several of these moves, such as sharing an opinion or showing similarity.

They then trained a chatbot on these chats. One version was given the label in front of each reply, and the other was not. The labelled version did better on the automatic scores and was preferred by people who chatted with both versions. The results come from the authors' own tests on a small data set.

TL;DR

• INSPIRED is 1,001 human-human movie-recommendation dialogs with 35,811 utterances. Each utterance carries a strategy label from a scheme with eight sociable strategies, two preference-elicitation inquiries and non-strategy classes §3 §4.1 tab. 2 tab. 5.
• Personal opinion, similarity, encouragement and credibility have positive, significant coefficients in a logistic regression on success §5.2 tab. 6. The model is a correlational analysis, not a causal test.
• The dialog model is two GPT-2 small language models, one per speaker. The strategy label is a special token prepended to the recommender turn §6.1 §6.2 app. C app. D.
• Author-reported: the strategy model has test perplexity 8.93 against 9.28 and BLEU-4 6.63 against 5.11 for the baseline tab. 7. Humans prefer it on all five aspects, with the largest gap on engagingness fig. 5 tab. 10.
• The released code differs from the paper in the number of sampled candidates (3, not 5) and in the selection rule (uniform random, the paper's rule is disabled). See "Paper vs code".

Problem

Recommendation dialog systems need data where a human recommender uses social strategies, because such strategies may drive acceptance. Earlier data sets were built from question-answer templates over reviews, or from a gamified role-play in which the seeker pretends to like the given movies §1 §2. The authors judge that neither gives natural, engaged conversation in which sociable behaviour can be studied §1 §2 tab. 1.

Success is measured in this paper by the seeker's behaviour after the chat: the seeker watches more than half of the recommended trailer and rates it 4 or 5 §3.2 §5.2. Without labelled sociable behaviour, a system cannot be trained to use it, and without the success measure, its effect cannot be estimated.

Method

Dataset collection §3. Recommenders and seekers are paired on Amazon Mechanical Turk and given different roles §3.2. The movie database contains 17,869 movies that have trailers and metadata; trailers come from a trailer channel (2008 to 2020) and the movies from MovieLens §3.1. Recommenders may suggest any number of movies. Seekers accept or reject each suggestion, and the system logs how long they watched the trailer §3.2. Before chatting, each worker answers a personality questionnaire (Big Five, Schwartz values, decision-making style) §3.3. A conversation runs at least 10 turns or until a recommendation is made §3.3. Collection ran from November 2019 to March 2020 with 1,594 workers §3.3.

Strategy definitions §4.1. Eight sociable strategies: personal opinion, personal experience, similarity, encouragement, offering help, preference confirmation, self-modeling and credibility §4.1. Offering help is part of the "transparency" strategy of an earlier study §4.1. Two preference-elicitation inquiries: experience inquiry and opinion inquiry §4.1. Other classes are no strategy, acknowledgment and recommendation. A recommendation is the first time a new movie title is proposed to the seeker §4.1.

Annotation quality §4.2. Two experts annotated 30 conversations with a Kappa of 0.77. Crowd workers had to score at least 0.60 on an evaluation dialog. Five workers labelled each dialog, and disagreements were resolved by experts. The majority agreement across workers is 0.78 §4.2.

Success analysis §5.2. A recommendation is successful when the seeker watched more than 50% of the trailer and rated it 4 or 5. It is unsuccessful when the seeker rejected it or accepted but was uninterested in the trailer. The other accept cases are left out. This gives 532 successful and 135 unsuccessful dialogs §5.2 tab. 3. A logistic regression predicts success from the per-dialog frequency of each strategy §5.2 tab. 6.

Dialog model §6.1 §6.2. Movie titles, actors and genres are replaced with indexed placeholders. The real attribute values are appended to the end of the utterance §6.1. The baseline uses two GPT-2 style language models, one per speaker, and conditions each on the shared history §6.1 app. C. Both are pretrained on WebText and on the ReDial and movie-preference data, then fine-tuned on INSPIRED §6.1 app. D. The strategy-incorporated model writes the strategy label as a special token before the recommender's utterance §6.2.

Candidate selection §6.2. The paper samples five candidates and randomly selects one that either contains encouragement or has the greatest sentence length. A BERT classifier decides whether a candidate is a recommendation; a recommendation triggers a new movie from the movie system §6.2. The authors report 95.4% accuracy and 91.2% F1 for this classifier §6.2.

Training objective. The dialog model's probability is the product, over turns, of each speaker's probability of the turn given the history app. C eq. 1. Training maximises the likelihood of each utterance under the model of its speaker §6.1 training.py.

Pseudocode for fine-tuning, following the released training script. Where paper and code differ, the code is followed and the difference is listed under "Paper vs code".

# Fine-tune two GPT-2 models, one per speaker, on INSPIRED  [§6.1] [app. D]
# Input:  dialogues, each a list of turns tagged "A:" or "B:"
#         [code: code/data_preprocessing.py#L130-L140]
# Output: model_a (recommender), model_b (seeker)

def train_epoch(dialogues, model_a, model_b, optimizer, scheduler):
    # One dialogue per step, in random order  [code: code/training.py#L195]
    for dialogue in shuffle(dialogues):
        # Keep turns only up to a 1024-token total
        # [code: code/training.py#L275-L281]
        turns = truncate_to_tokens(dialogue, max_tokens=1024)
        past = None
        all_logits = []
        # The speaker picks the model; the history cache is shared
        # [§6.1] [code: code/training.py#L98-L104]
        for turn in turns:
            model = model_a if turn.speaker == "A" else model_b
            logits, past = model(turn.token_ids, past=past)
            all_logits.append(logits)
        # Token-level cross-entropy, label smoothing 0.02
        # [code: code/training.py#L113]
        loss = smoothed_cross_entropy(all_logits, turns, smoothing=0.02)
        loss.backward()
        optimizer.step()
        scheduler.step()
        optimizer.zero_grad()

Pseudocode for generating one recommender reply, following the released interactive script:

# Sample candidate recommender replies  [§6.2]
# [code: code/interactive.py#L1582-L1603]
# Input:  model_a, saved history state "past", prompt tokens for "A:"
# Output: a list of candidate token lists

def sample_candidates(model_a, past, prompt, count):
    # The paper samples 5; the code samples 3
    # [§6.2] [code: code/interactive.py#L1539]
    candidates = []
    for _ in range(count):
        # Every candidate starts from the same saved history
        # [code: code/interactive.py#L1583]
        cache = past
        token = prompt
        tokens = []
        # Stop at the turn-end token 628 or after 200 tokens
        # [code: code/interactive.py#L1588-L1597]
        for _ in range(200):
            logits, cache = model_a(token, past=cache)
            # Temperature 0.8, then top-k 400, then top-p 0.9
            # [app. D] [code: code/interactive.py#L1590-L1591]
            logits = top_filter(logits / 0.8, top_k=400, top_p=0.9)
            token = sample_from(softmax(logits))
            if token == 628:
                break
            tokens.append(token)
        candidates.append(tokens)
    return candidates

The selection step follows the sampling:

# Drop any candidate that repeats a bigram
# [code: code/interactive.py#L1635-L1643]
candidates = [c for c in candidates if not has_repeated_bigram(c)]
# Choose one remaining candidate uniformly at random
# [code: code/interactive.py#L1646-L1648]
# (the paper's rule prefers "encouragement" or the longest; see Paper vs code)
chosen = random_choice(candidates)

Hyperparameters

Values in the table are from the paper unless the "Where" column names the code only.

Results

Author-reported. Setup: INSPIRED fine-tuned GPT-2 small; baseline vs strategy-incorporated model; automatic metrics on the held-out split and human pairwise and five-point ratings.

Human pairwise comparison: 28 participants chatted with both models two or three times each. They chose the better model on fluency, consistency, naturalness, persuasiveness and engagingness, with a "can't tell" option. The text reports that the strategy model was preferred on all five aspects, with the largest difference on engagingness, then naturalness §6.3 fig. 5. No counts or statistical test are given in the text.

Five-point ratings, 25 participants per model (50 in total) app. E tab. 10:

Strategy coefficients for success, from the logistic regression §5.2 tab. 6 (* p < 0.05, ** p < 0.01):

Movie attributes added to the same analysis do not reach significance (recent release, popularity and genres) §5.3 tab. 8.

Paper vs code

The code is the official repository linked from the paper's abstract footnote (github.com/sweetpeach/Inspired); the README names the first author as its contact. It contains data preprocessing, training and an interactive script. It does not contain the pretrained weights, the trained model, the two classifiers, the baseline model or the evaluation of BLEU and test perplexity.

• Candidates. Paper samples five candidates §6.2; the code samples three interactive.py.
• Selection. Paper picks at random among candidates with "encouragement" or the greatest length §6.2. The code chooses uniformly at random among candidates that pass the bigram filter interactive.py. The length rule is in a string literal and is disabled interactive.py. The strategy label of each candidate is computed but never used for selection interactive.py interactive.py.
• Temperature. Paper gives 0.82 for the baseline and 0.8 for the strategy model app. D; the code has only 0.8 interactive.py. The baseline code is not in the repository.
• Epochs. Paper trains "until it converges", about two epochs for the baseline and three for the strategy model app. D. The code runs 10 epochs and saves to the same file after each one, so only the last epoch is kept training.py training.py.
• Vocabulary. Paper gives a 50,310-token vocabulary app. D. The code takes the vocabulary size from the pretrained checkpoint and adds two rows training.py. The checkpoint is not in the repository.
• Label set. Paper describes eight sociable strategies and a combined preference-confirmation class §4.1. The classifier's label map has 13 classes, including separate "rephrase_preference" and "preference_confirmation" labels interactive.py.
• Transparency label. Training data uses the "transparency" label, which the preprocessing renames to "offer_help" data_preprocessing.py. This matches the paper's note that offering help belongs to transparency §4.1.
• Split. Paper gives 801 / 100 / 100 §6. The code gives the same counts training.py.
• Preprocessing input. The script reads its input with a comma delimiter data_preprocessing.py. The released files are tab-separated [data/dialog_data/train.tsv]. The script therefore needs the files converted first.
• Column name. The data README calls the main strategy column "first_label" [data/dialog_data/README.md]. The header of train.tsv calls it "expert_label", and the preprocessing reads column index 14 data_preprocessing.py.
• Test split. The code defines a test split training.py but never evaluates it. No BLEU code is in the repository. The test numbers in the paper have no matching script here.
• Length truncation. Paper gives no length limit. Code drops turns once the running total reaches 1024 tokens, and it drops one extra turn training.py.
• Tokenizer maximum. The code sets the tokenizer maximum to 1500 but also truncates at 1024 training.py training.py.

No discrepancies were found for the dataset split size, the learning rate of 3e-5 or the top-k of 400 and top-p of 0.9.

Pitfalls

• Success is a filter. The success analysis uses 532 successful and 135 unsuccessful dialogs. The 45 "rating 3 or lower" dialogs and the 289 "other reasons" dialogs are dropped, so the analysis covers 667 of 1,001 dialogs §5.2 tab. 3. Dropped dialogs may differ systematically from kept ones.
• Numbers that do not reconcile. The text gives 79.7% successful cases §3.3. The counts in the case table add to 1,001 and do not give 79.7% under any simple grouping that I checked tab. 3. The percentages in the annotation table use different denominators tab. 5. Treat both as unreconciled.
• Correlation, not effect. The coefficients come from frequency features on one set of dialogs, with no control set or reported standard errors §5.2 tab. 6. The paper nonetheless calls the result a confirmation of earlier theory §5.2.
• Perplexity definition. The code averages per-dialog perplexities over the validation set training.py. The paper does not say how perplexity is aggregated, so a corpus-level perplexity would not match.
• Small evaluation. Automatic scores come from 100 dialogs. The human study has 28 participants and no reported statistical test §6.3 app. E.
• Sampling is noisy. The reply is drawn at random from candidates, so the same history can give different replies. The bigram filter removes candidates in a loop that changes the list while iterating over it, which can skip a candidate; and when every candidate is removed, the random choice has no valid index interactive.py.
• Strategy is not enforced. The strategy label is generated as part of the reply. The released selection ignores it, and nothing checks that the chosen reply carries the intended strategy interactive.py interactive.py.
• Recommendation trigger. A reply classified as a recommendation forces the system to suggest a new movie §6.2. Movie attribute detection depends on an external movie database and on heuristics that the paper lists in the appendix app. B. For a cold start, the appendix recommends the most recent movie with the last mentioned genre app. B.
• Data and code licence. The repository has no LICENSE file, so the code licence is not stated. The data README does not state a licence either. Do not copy code or data from this repository on the strength of this card.
• Hidden costs. Training two full GPT-2 models with a 1024-token history is the main cost. The paper reports 1.37 hours per pretraining epoch and 16 minutes per fine-tuning epoch on a GTX 1080 Ti §6.1 app. D.

Lineage

• Builds on: the ReDial recommendation dialogs and the movie-preference elicitation data, used for pretraining §6.1.
• Compared against: a baseline without strategy labels, same architecture §6.1 §6.3.
• Related datasets named in the paper: ConvRec, GoRecDial tab. 1 §2. No cards for these yet.
• Superseded by: not known from content in this wiki.