---
kind: concept
slug: partial-information-decomposition
title: Partial information decomposition
summary: >-
  Splits what several inputs say about a target into unique, redundant and synergistic parts. Sensory PID applies it to audio and video answers of omni-modal models, estimated from answer distributions with one stream masked, and can weight training samples.
tags: [interpretability, information-theory, multimodal]
papers: ["2606.00959"]
related: [multimodal-language-model, feature-attribution, causal-inference]
status: draft
updated_at: 2026-10-19
---

## Overview

Partial information decomposition (PID) asks how the information that several
inputs carry about one target is shared between them. Each part is one of
four kinds: information only one input gives, information both give, and
information that appears only when the inputs are used together (synergy)
[arxiv:2606.00959 §2.1]. The paper uses this to ask which sense a multimodal
model relies on when it answers a question about a video with sound.

The invariant all variants share is that the four terms are defined from
three predictive distributions of the target: one from all inputs, and one
from each input alone [arxiv:2606.00959 §2.1] [eq. 2]. Variants differ in how
those distributions are obtained and how the terms are estimated.

## Variants

### Sensory PID for omni-modal models

The target is the model's answer, the inputs are the video stream and the
audio stream, and the text instruction is held fixed as context. This leaves
a four-term split: video-only, audio-only, shared and synergy
[arxiv:2606.00959 §2.1] [eq. 1]. The text is not decomposed, because
treating it as a third source would enlarge the analysis
[arxiv:2606.00959 §2.1]. Card: [Sensory PID for omni-modal models](/papers/2606.00959).

The single-stream distributions are not native to these models. The paper
approximates them by replacing one stream's embeddings with Gaussian noise
matched per dimension to a profiling set, and keeps the text unchanged
[arxiv:2606.00959 §2.2] [eq. 3]. The estimator is BATCH, a learned coupling
fitted under the source-to-answer marginals [arxiv:2606.00959 §2.2]
[app. A.2]. The training-free estimate needs no change to the model.

### PID-guided sample weighting for fine-tuning

The same paper reuses the local terms as per-sample scores for LoRA
fine-tuning of vision-language models. Samples with high fusion potential and
low synergy and shortcut scores get a larger weight, samples that a text-only
path can answer get a smaller one [arxiv:2606.00959 §2.3] [alg. 1]. The
scores are heuristics for ranking, not pointwise PID atoms [arxiv:2606.00959
§2.3] [app. A.3]. Card: the same [paper card](/papers/2606.00959).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You want to know which input a model's answer depends on, and the model returns answer probabilities | Sensory PID, training-free | Needs three forward passes per sample and a profiling set; no training [arxiv:2606.00959 §2.1] [§2.2] |
| The model is a closed API that returns no probabilities | Not covered | The estimator needs answer distributions with one stream masked [arxiv:2606.00959 §2.2] |
| The task is multiple choice with explicit candidates | Restrict to candidates before estimating | Predictions are renormalised over the options and shrunk toward uniform when little mass falls on them [arxiv:2606.00959 §2.2] [eq. 4]-[eq. 6] |
| You fine-tune a vision-language model and want a cheap sample selection signal | PID-guided weighting | Reported +2.3 pp on MMStar for Qwen2.5-VL-7B over uniform LoRA, author-reported [arxiv:2606.00959 §5.1] |
| You fine-tune a model whose answers gain little from fusing inputs | Expect little from PID weighting | Gemma3-12B showed no clear gain [arxiv:2606.00959 §5.3] [tab. 3] |
| You need a training recipe for omni-modal models | Not established | The reweighting was run on vision-language models only [arxiv:2606.00959 §5] |

Constraints to check before choosing: the number of forward passes (three per
sample for Sensory PID) [arxiv:2606.00959 §2.1], the size of the profiling set
(not stated) [arxiv:2606.00959 §2.2], and whether the target answer is
discrete and enumerable.

## Implementation sketch

The common core is: obtain three answer distributions per sample, stabilise
them, and fit a coupling that yields the four terms. Where variants differ is
noted inline. The recovery of the four terms from the coupling under a text
condition is not stated in the paper [arxiv:2606.00959 app. A.2].

```pseudocode
# Sensory PID core, one target and two input streams  [arxiv:2606.00959 §2.1-§2.2]
# Input:  inputs a, b; fixed context t; candidate answers; profiling set
# Output: four terms (unique a, unique b, shared, synergy)

def pid_terms(a, b, t, candidates, profile):
    # Full-input answer distribution  [§2.1] [eq. 2]
    p_ab = stabilise(model(a, b, t), candidates)
    # Single-stream distributions, other stream replaced by noise  [§2.2] [eq. 3]
    p_a = stabilise(model(a, noise_like(b, profile), t), candidates)
    p_b = stabilise(model(noise_like(a, profile), b, t), candidates)
    # Pool each stream to a fixed-size vector before estimation  [§2.2]
    x_a = mean_pool(a)
    x_b = mean_pool(b)
    # Target marginal, averaged over profiling data and then held fixed  [§2.2] [eq. 7]
    marginal = average(p_ab, over=profile)
    # Coupling fitted with BATCH (Sinkhorn-normalised)  [app. A.2]
    coupling = batch_fit(x_a, x_b, p_ab, p_a, p_b, marginal)
    # Read the four terms off the coupling; text-conditioned formulas not stated
    return recover_terms(coupling)
```

```pseudocode
# Shrink a candidate-restricted distribution toward uniform  [§2.2] [eq. 4]-[eq. 6]
# Input:  full-vocabulary probs; candidate ids; reliability threshold tau
# Output: distribution over the candidates

def stabilise(probs, candidates, tau):
    # Mass that falls on the valid options  [eq. 4]
    mass = sum(probs[c] for c in candidates)
    restricted = renormalise(probs, candidates)
    # Reliability in [0, 1]; the omni-modal tau is not stated  [eq. 5]
    rho = min(1, mass / tau)
    return rho * restricted + (1 - rho) / len(candidates)
```

## Contradictions

None known. Only one card covers this concept, so no paper disagreement can be
checked here. Within the card, the paper reports two stabilisation rules: a
hard gate in the vision-language appendix and a continuous one in the main
text for Sensory PID [arxiv:2606.00959 §2.2] [app. A.2]. The paper states the
continuous rule is the one it uses for Sensory PID, so this is not a
disagreement between results.

## Papers

- [Sensory PID for omni-modal models](/papers/2606.00959): defines Sensory PID for omni-modal models and PID-guided LoRA sample weighting; results are author-reported and on MUSIC-AVQA and vision-language benchmarks.