Review-based recommendation

Overview

A review-based recommender predicts how a user would rate an item, using the text of reviews as well as the ratings. The motivation is sparsity: many users and items have only a few ratings, and rating-only models predict poorly for them 1701.04783 §1. A review says why a rating was given, so the text can supply signal where ratings are missing 1701.04783 §1.

Shared structure. The model turns text about the user and text about the item into vectors, then scores the pair. Quality is compared by test MSE 1701.04783 eq. 11 on an 80/10/10 split 1701.04783 §3.3.

Variants

Only one variant has a paper card in the wiki so far. Its ablations show which design choices carry weight, so they are described under it.

Two-tower CNN with a factorization machine (DeepCoNN)

Each side has its own network. The user tower reads all reviews written by the user as one document, and the item tower reads all reviews of the item. Each document becomes a sequence of pretrained word vectors, goes through 1-D convolutions with max-over-time pooling and a fully connected layer, and the two outputs are scored by a factorization machine 1701.04783 §2.2 1701.04783 §2.4 1701.04783 §2.5. Reported settings: 100 kernels, window 3, 300-dimensional word vectors, 50-dimensional latent vectors 1701.04783 §3.3. Card: Joint Deep Modeling of Users and Items Using Reviews for Recommendation.

Reported test MSE, against the best baseline (CDL) 1701.04783 tab. 3:

Ablations on Yelp, Amazon Music Instruments and Beer 1701.04783 tab. 4. Note that the Amazon column here is one category, not the 21-category average above, so the two Amazon figures are not comparable.

The paper reads these as: both towers together beat either tower alone, pretrained embeddings matter most, and the FM beats a dot product 1701.04783 §3.5.

When to use which

Before training, decide whether the scored review is removed from the input documents. The paper does not say, and including it could inflate the reported gains; see the leakage note in Pitfalls on the card.

Implementation sketch

Our own pseudocode, following the paper's description. Items marked "not stated" are choices the implementer must make.

# One document per user and per item, from training reviews only [§2.3]
user_doc = concat(reviews written by u)      # exclude the scored review
item_doc = concat(reviews written for i)      # paper does not say; avoids leakage

# Pretrained 300-d word vectors, order kept [§2.3] [§3.3]
V = [word_vec(w) for w in user_doc]          # n x 300

# 100 kernels, window 3, ReLU, max over positions [§2.4]
o = [max_pos relu(conv(V, K_j, window=3) + b_j) for j in 1..100]

# Fully connected to a 50-d vector; dropout rate not stated [§2.4] [§2.6]
x_u = dropout(relu(W_u @ o + g_u))           # 50
y_i = dropout(relu(W_i @ o_i + g_i))         # 50, separate weights

# Factorization machine over the concatenated vectors [§2.5]
z = concat(x_u, y_i)                         # 100
pred = w0 + w @ z + sum_{a<b} <v_a, v_b> z_a z_b

# RMSprop, learning rate 0.002, batch 100; loss not stated, MSE is the
# metric [§2.6] [§3.3] [eq. 11]

Points the paper leaves open, which an implementation must decide: the max document length and truncation; padding and out-of-vocabulary words; whether the embedding table is fine-tuned; the FM factor size; and the RMSprop epsilon and dropout rate. The card lists these under Pitfalls.

Contradictions

Within the paper, not across papers:

• Headline gains. The conclusion gives 8.5% and 7.6% for Yelp and Beer, and 8.7% for Amazon 1701.04783 §5. The results table gives Yelp 8.5%, Amazon 7.6% and Beer 8.7% 1701.04783 tab. 3.
• Average gain. The table's average row shows 8.3%, the mean of the three per-dataset percentages. The relative drop on the averaged MSE (1.081 to 0.994) is about 8.0% 1701.04783 tab. 3.
• Amazon corpus size. The text describes the full corpus as over 143.7M reviews 1701.04783 §3.1. The dataset table lists 34.7M reviews 1701.04783 tab. 2. The subset selection rule is not stated.

No other paper card in the wiki covers this technique, so no cross-paper disagreement is recorded.

Papers

• Joint Deep Modeling of Users and Items Using Reviews for Recommendation: introduces DeepCoNN, the two-tower CNN with a factorization machine, and the Yelp, Amazon and Beer results above.