---
kind: concept
slug: ridge-regression
title: Ridge regression
summary: >-
  Linear regression with an L2 penalty on the weights, which shrinks them towards zero. The penalty strength is the main choice. Shown here on predicting self-rated emotions from text features: worry MAE 1.26 on long texts, while short-text models explain almost nothing.
tags: [regression, regularisation, linear-model]
papers: ["2004.04225"]
related: [lasso-regression, sentiment-analysis, topic-modelling]
status: draft
updated_at: 2026-11-22
---

## Overview

Ridge regression fits a linear model and adds a penalty on the squared size of
its weights, so that large weights cost more. The penalty strength is the
setting that decides how far the weights shrink. In the only card linked here,
ridge is the regression model for predicting a participant's emotion rating
from text features, with the penalty not stated [arxiv:2004.04225 §3.3]. This
page therefore covers one documented use; other cards may use ridge but do not
list this concept yet.

## Variants

### Ridge on reduced text features

Features are built per corpus, either TF-IDF over the 1,000 most frequent words
or counts of 53 spaCy part-of-speech tags. Principal component analysis reduces
them before the ridge fit. Performance is measured with mean absolute error
(MAE) and R-squared under five-fold cross-validation [arxiv:2004.04225 §3.3].
The number of components, the ridge penalty and the fold split are not stated
[arxiv:2004.04225 §3.3]. For worry, TF-IDF gives MAE 1.26 and R-squared 0.16 on
long texts, against MAE 1.38 and R-squared -0.03 on short texts
[arxiv:2004.04225 tab. 4]. Card: [RWWD emotion study](/papers/2004.04225).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Many correlated text features and a linear, inspectable model is wanted | Ridge on PCA-reduced features, tuned by cross-validation | The paper uses this pipeline for its emotion predictions [arxiv:2004.04225 §3.3] |
| Predicting a rating from long texts | Expect some signal, with TF-IDF ahead of POS features | Worry R-squared 0.16 for TF-IDF vs 0.03 for POS on long texts [arxiv:2004.04225 tab. 4] |
| Predicting from short texts | Expect near-zero gain over a mean predictor | Every short-text R-squared in the table lies between -0.03 and 0.01 [arxiv:2004.04225 tab. 4] |
| Judging whether a model is good | Compare against a baseline you compute yourself | The table has no baseline row [arxiv:2004.04225 tab. 4] |

## Implementation sketch

The steps below follow the paper's described pipeline [arxiv:2004.04225 §3.3].
The penalty and component count are inputs you must set; the paper gives
neither.

```pseudocode
# Ridge prediction of one rating from text features  [arxiv:2004.04225 §3.3]
# Input:  texts (N,); ratings (N,) on a 9-point scale
# Output: mean MAE and mean R-squared over five folds

def predict_rating(texts, ratings, feature_type):
    # TF-IDF over the 1,000 most frequent words, or counts of 53 POS tags
    features = build_features(texts, feature_type)     # (N, D)
    maes, r2s = [], []
    # Five folds; the split and seed are not stated  [arxiv:2004.04225 §3.3]
    for train_idx, test_idx in five_folds(len(ratings)):
        # Reduce dimension; component count not stated  [arxiv:2004.04225 §3.3]
        pca = fit_pca(features[train_idx])
        train_x = pca.transform(features[train_idx])
        test_x = pca.transform(features[test_idx])
        # Fit with an L2 penalty; its value is not stated  [arxiv:2004.04225 §3.3]
        model = fit_ridge(train_x, ratings[train_idx])
        pred = model.predict(test_x)
        maes.append(mae(ratings[test_idx], pred))
        r2s.append(r_squared(ratings[test_idx], pred))
    return mean(maes), mean(r2s)
```

## Contradictions

None known. Only one card links this concept, so there is no second paper to
compare against.

## Papers

- [RWWD emotion study](/papers/2004.04225): releases the Real World Worry Dataset and predicts emotion ratings from TF-IDF or POS features with ridge regression.
