fMRI encoding model

Overview

An fMRI encoding model predicts the signal of each voxel from a feature space built from the stimulus. The feature space is the variable under study: a set of numbers per word or time point, taken from a language model, an embedding table or a tagger 2106.05426 §4.2.2. The model is then scored by how well its predictions correlate with the held-out recording. The shared invariant is that the regression is fitted per voxel, so one feature space gives one score per voxel 2106.05426 §4.2.2.

The one paper on this page uses ridge regression with time delays. Each feature vector is copied at delays of 2, 4, 6 and 8 seconds, so that the slow blood-oxygen response can be matched to the words that caused it 2106.05426 §4.2.2. The regularisation strength is chosen per voxel by cross-validation 2106.05426 §4.2.2. The setup is in Antonello et al. (2021).

Variants

Per-representation encoding with delays

Fit one ridge encoding model per feature space, with delays, and score each voxel by the correlation on held-out stories. The paper uses 100 feature spaces across five subjects, each listening to podcast stories during a scan with a repetition time of 2.00 s 2106.05426 §4.2.1 2106.05426 §4.2.2. Card: Antonello et al. (2021).

Voxels are restricted to a band near the middle of the cortical surface, and voxels whose explainable variance falls below a threshold are dropped 2106.05426 §4.2.1. The paper reports about 90,000 voxels per subject and a training set of about 5 hours, or 9,189 datapoints per voxel 2106.05426 §4.2.1.

Mapping the fits across representations

The same per-voxel correlation vectors can be compared across feature spaces. The paper z-scores each voxel's correlations over the 100 representations, then projects voxels on the first dimension of a multidimensional scaling of the representation embedding 2106.05426 §4.2.3 2106.05426 fig. 3. This variant asks how the feature spaces relate to each other through the brain, not which one fits a single voxel best.

Using the embedding to predict fit, the paper trains a linear map from each representation's embedding row to its correlation vector, and tests it on the held-out pairs. The author-reported result is that the correct pairing scores above the swapped one for most representations 2106.05426 §4.2.4 2106.05426 fig. 4. Card: Antonello et al. (2021).

When to use which

Only one paper is on this page, so no variant is compared against another method. The paper's own comparison is between feature spaces, not between encoding models or regression choices 2106.05426 §2.

Implementation sketch

Language-agnostic pseudocode for the per-representation variant. The regularisation search and the exact correlation criterion are not stated in the paper 2106.05426 §4.2.2.

# Per-representation fMRI encoding with time delays  [arxiv:2106.05426 §4.2.2]
# Input:  feature vectors X (T x d) per word, BOLD responses Y (T x V)
# Output: correlation r per voxel on held-out stories

def encoding_fit(X, Y, delays=(2, 4, 6, 8)):
    # Shift each feature vector by the delays in seconds  [§4.2.2]
    Xd = concat([shift(X, d) for d in delays], axis=1)
    # Hold out one part of the data for validation  [§4.2.2]
    train, val = split(Xd, Y, validation=0.2)
    # Pick the ridge parameter per voxel by repeated CV  [§4.2.2]
    alpha = pick_alpha_per_voxel(train, val, folds=50)
    W = ridge_fit(train, alpha)
    # Score each voxel on the held-out story  [§4.2.2]
    return corr(predict(W, test), Y_test)  # one value per voxel

Contradictions

• First-dimension variance. The text reports 22% of variance for the first dimension of the representation embedding 2106.05426 §4.1.1. The brain figure labels the first MDS dimension as about 20% 2106.05426 fig. 3. The two figures are not the same quantity, and the paper does not reconcile them.
• Scree analysis. The paper says the scree plot comes from exploratory factor analysis 2106.05426 §4.1.1. The authors' released notebook plots the eigenvalues of a classical MDS instead, so the 22% and 10% figures may not be reproducible from it. This is a code-versus-paper gap, noted on the card.

Papers

• Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses: fits 100 language feature spaces to five subjects' fMRI data with ridge encoding models and delays, then maps the feature spaces with a representation embedding and tests whether the map predicts brain fit.