---
kind: concept
slug: fmri-encoding-model
title: fMRI encoding model
summary: >-
  A regression that predicts each voxel's fMRI response from a stimulus feature space, usually with time delays. Used to ask which internal representations of language match brain activity. Trade-off: the fit depends on the chosen feature space and the stimuli. One paper so far (author-reported: a representation embedding predicts which spaces fit, over 90% of the time).
tags: [representation-analysis, encoding-models, transfer-learning, neuroscience]
papers: ["2106.05426"]
related: [linear-probing, transfer-learning, word-embedding]
status: draft
updated_at: 2026-11-05
---

## Overview

An fMRI encoding model predicts the signal of each voxel from a feature
space built from the stimulus. The feature space is the variable under study:
a set of numbers per word or time point, taken from a language model, an
embedding table or a tagger [arxiv:2106.05426 §4.2.2]. The model is then
scored by how well its predictions correlate with the held-out recording. The
shared invariant is that the regression is fitted per voxel, so one feature
space gives one score per voxel [arxiv:2106.05426 §4.2.2].

The one paper on this page uses ridge regression with time delays. Each
feature vector is copied at delays of 2, 4, 6 and 8 seconds, so that the
slow blood-oxygen response can be matched to the words that caused it
[arxiv:2106.05426 §4.2.2]. The regularisation strength is chosen per voxel
by cross-validation [arxiv:2106.05426 §4.2.2]. The setup is in
[Antonello et al. (2021)](/papers/2106.05426).

## Variants

### Per-representation encoding with delays

Fit one ridge encoding model per feature space, with delays, and score each
voxel by the correlation on held-out stories. The paper uses 100 feature
spaces across five subjects, each listening to podcast stories during a scan
with a repetition time of 2.00 s [arxiv:2106.05426 §4.2.1] [arxiv:2106.05426 §4.2.2].
Card: [Antonello et al. (2021)](/papers/2106.05426).

Voxels are restricted to a band near the middle of the cortical surface, and
voxels whose explainable variance falls below a threshold are dropped
[arxiv:2106.05426 §4.2.1]. The paper reports about 90,000 voxels per subject
and a training set of about 5 hours, or 9,189 datapoints per voxel
[arxiv:2106.05426 §4.2.1].

### Mapping the fits across representations

The same per-voxel correlation vectors can be compared across feature spaces.
The paper z-scores each voxel's correlations over the 100 representations, then
projects voxels on the first dimension of a multidimensional scaling of the
representation embedding [arxiv:2106.05426 §4.2.3] [arxiv:2106.05426 fig. 3]. This
variant asks how the feature spaces relate to each other through the brain,
not which one fits a single voxel best.

Using the embedding to predict fit, the paper trains a linear map from each
representation's embedding row to its correlation vector, and tests it on the
held-out pairs. The author-reported result is that the correct pairing
scores above the swapped one for most representations [arxiv:2106.05426 §4.2.4]
[arxiv:2106.05426 fig. 4]. Card: [Antonello et al. (2021)](/papers/2106.05426).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You want the fit of one or a few feature spaces on your own recordings | Per-representation encoding with delays | Only one regression per space; the paper's test uses five subjects and one story [arxiv:2106.05426 §4.2.1] [arxiv:2106.05426 app. B] |
| You want to compare many feature spaces through their brain fit | Mapping the fits across representations | Needs a fit for every space on the same data, plus the representation embedding [arxiv:2106.05426 §4.1] [arxiv:2106.05426 §4.2.3] |
| You want to predict fit for a representation not in the set | Not supported by this paper | The matching test only uses the 100 sampled spaces, and the authors say unsampled parts could change the picture [arxiv:2106.05426 §5] |
| You have little compute | Check the cost first | The representation embedding alone needs n² = 10,000 decoders, about one week on 53 GPUs and 64 CPU servers [arxiv:2106.05426 §3.2] [arxiv:2106.05426 §4.1] |

Only one paper is on this page, so no variant is compared against another
method. The paper's own comparison is between feature spaces, not between
encoding models or regression choices [arxiv:2106.05426 §2].

## Implementation sketch

Language-agnostic pseudocode for the per-representation variant. The
regularisation search and the exact correlation criterion are not stated in
the paper [arxiv:2106.05426 §4.2.2].

```pseudocode
# Per-representation fMRI encoding with time delays  [arxiv:2106.05426 §4.2.2]
# Input:  feature vectors X (T x d) per word, BOLD responses Y (T x V)
# Output: correlation r per voxel on held-out stories

def encoding_fit(X, Y, delays=(2, 4, 6, 8)):
    # Shift each feature vector by the delays in seconds  [§4.2.2]
    Xd = concat([shift(X, d) for d in delays], axis=1)
    # Hold out one part of the data for validation  [§4.2.2]
    train, val = split(Xd, Y, validation=0.2)
    # Pick the ridge parameter per voxel by repeated CV  [§4.2.2]
    alpha = pick_alpha_per_voxel(train, val, folds=50)
    W = ridge_fit(train, alpha)
    # Score each voxel on the held-out story  [§4.2.2]
    return corr(predict(W, test), Y_test)  # one value per voxel
```

## Contradictions

- **First-dimension variance.** The text reports 22% of variance for the first
  dimension of the representation embedding [arxiv:2106.05426 §4.1.1]. The
  brain figure labels the first MDS dimension as about 20% [arxiv:2106.05426 fig. 3].
  The two figures are not the same quantity, and the paper does not reconcile them.
- **Scree analysis.** The paper says the scree plot comes from exploratory
  factor analysis [arxiv:2106.05426 §4.1.1]. The authors' released notebook
  plots the eigenvalues of a classical MDS instead, so the 22% and 10% figures
  may not be reproducible from it. This is a code-versus-paper gap, noted on the card.

## Papers

- [Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses](/papers/2106.05426): fits 100 language feature spaces to five subjects' fMRI data with ridge encoding models and delays, then maps the feature spaces with a representation embedding and tests whether the map predicts brain fit.