Collaborative filtering

Overview

A recommender estimates how a user would rate an item they have not rated yet. Collaborative filtering does this from the ratings matrix alone, the observed ratings of many users on many items 1701.04783 §1. The matrix is very sparse, and users or items with few ratings get poor predictions 1701.04783 §1.

Shared structure. Every variant scores a (user, item) pair from learned representations of the user and the item. Prediction quality is compared by test MSE on an 80/10/10 split 1701.04783 §3.3 1701.04783 eq. 11.

Variants

The variants below differ in what the representations are learned from: ratings only, or ratings plus review text.

Rating-only matrix factorization

Uses only the rating matrix. In 1701.04783 it is a baseline: MF is fit with ALS and tuned over latent-factor sizes {25, 50, 100, 150, 200} and regularisation {0.001, 0.01, 0.1, 1.0} 1701.04783 §3.2 1701.04783 §3.3. Reported test MSE is 1.792 (Yelp), 1.471 (Amazon) and 0.612 (Beer). PMF reaches 1.783, 1.460 and 0.527 1701.04783 tab. 3. Card: Joint Deep Modeling of Users and Items Using Reviews for Recommendation.

Review-augmented two-tower model (DeepCoNN)

Learns each user's and each item's representation from text. All reviews a user wrote form one document, and all reviews of an item form another. Each document goes through its own CNN: pretrained word vectors, 1-D convolution, max-over-time pooling and a fully connected layer 1701.04783 §2.3 1701.04783 §2.4. A factorization machine scores the concatenated user and item vectors 1701.04783 §2.5. Both networks are trained together 1701.04783 §2.6.

Reported test MSE is 1.441 (Yelp), 1.268 (Amazon) and 0.273 (Beer), against 1.574, 1.372 and 0.299 for the best baseline, CDL 1701.04783 tab. 3. Ablations on Yelp show that a learned item-embedding table in place of the item CNN gives 1.577, a dot product in place of the FM gives 1.491, and random word vectors give 1.799 1701.04783 tab. 4. Card: Joint Deep Modeling of Users and Items Using Reviews for Recommendation.

When to use which

Implementation sketch

Review-augmented variant, following 1701.04783 §2.2–§2.6. Our own pseudocode; the paper gives no code.

# Build one document per user and per item from training-split reviews
# (drop the review of the pair being scored; see Papers)            [§2.3]
for each user u:  D_u = all training reviews written by u
for each item i:  D_i = all training reviews written for i

# Encode one document with one tower (Net_u or Net_i)
def encode(D, params):
    # one pretrained vector per word, order kept (dimension 300)     [eq. 1]
    V = [embed(w) for w in D]               # n x 300
    # 100 kernels, window 3 words, bias, ReLU                        [eq. 2]
    Z = [relu(conv1d(V, K_j) + b_j) for j in 1..100]
    # keep the largest value of each feature map                     [eq. 4]
    O = [max(Z_j) for Z_j in Z]
    # fully connected layer to a 50-dim vector, ReLU, dropout        [eq. 6]
    return dropout(relu(W · O + g))

# Score one (user, item) pair
x = encode(D_u, user_params)                # 50-dim
y = encode(D_i, item_params)                # 50-dim
pred = factorization_machine(concat(x, y))  # eq. 7

# Train: RMSprop, batch 100, learning rate 0.002 [§3.3]
# Loss is not stated in the paper; squared error is the likely choice

Contradictions

Only one paper covers this concept, so no disagreement between papers is known. Within 1701.04783 the reported percentages do not agree. The conclusion gives 8.5% for Yelp and 7.6% for Beer, and 8.7% for Amazon 1701.04783 §5. The table gives Yelp 8.5%, Amazon 7.6% and Beer 8.7% 1701.04783 tab. 3. The "8.3% average" is the mean of the three percentages. The relative gain on the averaged MSE row is about 8% 1701.04783 tab. 3.

Papers

• Joint Deep Modeling of Users and Items Using Reviews for Recommendation: DeepCoNN, a two-tower CNN over review text with a factorization machine that predicts the rating.