---
kind: concept
slug: partial-multi-view-learning
title: Partial multi-view learning
summary: >-
  Learning from samples where some views (sensors or feature sets) are missing, without grouping samples by missing pattern or imputing first. CPM-Nets learns one shared code per sample from its observed views, with a class-structure loss. A GAN variant imputes missing views for unsupervised use. Results are author-reported; only one paper is covered so far.
tags: [multi-view-learning, missing-views, representation-learning, data-imputation, clustering]
papers: ["2011.06170"]
related: [multi-view-clustering, multi-view-representation, multimodal-fusion, generative-adversarial-network, autoencoder]
status: draft
updated_at: 2026-11-01
---

## Overview

A multi-view sample has several views, for example several sensors or several
feature sets. In partial multi-view data, some views are absent for some
samples. The task is to classify, or to learn representations, from whatever
views each sample has [arxiv:2011.06170 §1, §2].

The usual workarounds are to fill in the missing views, or to split samples by
their missing pattern and fit one model per group. The first needs paired
samples to learn the fill-in. The second does not scale well as the number of
views grows [arxiv:2011.06170 §1].

The shared idea in the one paper covered here is to keep every sample and every
observed view in a single model. Each sample gets one latent code. A network per
view decodes that code, and the reconstruction error counts only the views that
are observed [arxiv:2011.06170 §2.1]. A class-structure term makes codes of the
same class close, so classification can run on the codes [arxiv:2011.06170 §2.2].

## Variants

### CPM-Nets (supervised)

The base model. It trains per-view decoders and one code per training sample,
with reconstruction over observed views plus a weighted class loss. A new
sample is handled by fitting a code to its observed views and then classifying
by similarity to the training codes [arxiv:2011.06170 §2.2, §2.3, alg. 1].
Card: [Deep Partial Multi-View Learning](/papers/2011.06170).

The paper reports classification accuracy above nine baselines across missing
rates. On CUB with no missing rate stated for that sentence, it reports 89.48%
against 86.27% for the best baseline, LMNN [arxiv:2011.06170 §4.3]. Gaps on the
other datasets are given only as percentages, and in a figure [arxiv:2011.06170
§4.3, fig. 4].

### CPM-GAN (unsupervised, with imputation)

The same per-view networks also act as generators for the missing entries. One
discriminator per view is trained on observed values, and its adversarial loss
applies only to the missing entries [arxiv:2011.06170 §3, eq. 10]. The paper
uses this variant for unsupervised clustering, and for imputation compared with
mean fill, SVD and a low-rank method [arxiv:2011.06170 §4.4].

The paper's unsupervised figures are averages over several multi-view sets at
a missing rate of 0.5. They are 3.36% higher ACC and 5.11% higher NMI than the
second-best method [arxiv:2011.06170 §4.4.2, fig. 8]. Imputation error is
reported as the lowest at every rate tested [arxiv:2011.06170 §4.4.1, fig. 7].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Labelled data, many views, many missing patterns | CPM-Nets | The paper argues this avoids grouping by pattern and a separate imputation step [arxiv:2011.06170 §1, §2]. The paper reports better accuracy when many views are missing, with its large gap figures given at a missing rate of 0.5 [arxiv:2011.06170 §4.3, fig. 4]. |
| Labelled data with few samples per class | CPM-Nets | The authors argue the non-parametric class loss suits small samples [arxiv:2011.06170 §2.2]. This is their argument, not a tested comparison. |
| Need filled-in views for other models | CPM-GAN imputation | The paper reports lower imputation error than its baselines [arxiv:2011.06170 §4.4.1, fig. 7]. |
| No labels, clustering wanted | CPM-GAN (unsupervised) | Reported clustering gains over other methods at a 0.5 missing rate [arxiv:2011.06170 §4.4.2, fig. 8]. |
| Latency-sensitive inference, one forward pass wanted | Not covered by this paper | Each test sample needs an optimisation loop over its code, after the decoders are re-tuned [arxiv:2011.06170 alg. 1]. The paper gives no runtime figures. |
| Large training set | Plan for re-tuning and per-sample codes | The method keeps one code per training sample [arxiv:2011.06170 §2.1]. Runtime and memory are not reported. The paper uses 10% mini-batches per class for large sets [arxiv:2011.06170 §4.3.1]. |

Caveat: all results are author-reported. The text gives gaps without naming
the metric in every case, and some key numbers are only in figures
[arxiv:2011.06170 §4.3, fig. 4].

## Implementation sketch

The common core of the paper's method is shown below. It is a simplification:
the paper alternates decoder and code updates per iteration, and re-tunes the
decoders before testing [arxiv:2011.06170 alg. 1, §2.3].

```pseudocode
# Common core: one code per sample, observed views only  [arxiv:2011.06170 §2.1]
# mask[n, v] = 1 if view v of sample n is observed, else 0

def train(views, mask, labels, lam):
    # One decoder per view, mapping a code back to that view  [§2.1]
    decoders = [make_decoder(v) for v in view_ids]
    # One free code per training sample  [§2.1]
    codes = random_init(num_samples, K)
    for step in range(num_steps):
        # Reconstruct observed views only; missing views add nothing  [eq. 5]
        recon = sum(mask[n, v] * sq_err(decoders[v](codes[n]), views[n, v])
                    for n in samples for v in view_ids)
        # Class term: mean similarity to each class's codes, hinge margin [eq. 7]
        class_term = hinge_class_loss(codes, labels)
        codes -= alpha * grad(recon + lam * class_term, codes)
        # Decoders are updated on the reconstruction term too  [alg. 1]
        decoders -= alpha * grad(recon, decoders)
    return decoders, codes

def predict(views, mask, decoders, train_codes, train_labels):
    # Fit a new code to the observed views only  [§2.3]
    code = random_init(K)
    for step in range(num_test_steps):
        code -= alpha * grad(masked_recon(code, views, mask, decoders), code)
    # Class score: mean similarity to each class's training codes  [§2.2]
    return argmax_c mean(dot(code, train_codes[train_labels == c]))
```

For CPM-GAN, add one discriminator per view. Its loss covers only missing
entries, weighted by 1 minus the mask [arxiv:2011.06170 eq. 10]. The paper's
algorithm alternates discriminator, generator and code updates
[arxiv:2011.06170 alg. 2].

## Contradictions

No disagreements between papers are known, since only one paper is covered.

Within the paper, the λ grid differs between the hyperparameter section and
the training-set section: {0.1, 1, 10} in one place and {0, 0.1, …, 10} in the
other [arxiv:2011.06170 §4.2; §4.3.1]. The paper does not give a single value
per dataset.

The released code differs from the paper in several settings. These include
the learning rates, the latent size, dropout, and the absence of the GAN and
re-tuning steps. The card lists each difference with a citation
[Deep Partial Multi-View Learning](/papers/2011.06170). Implementations should
follow the paper's text, or state which setting they changed.

## Papers

- [Deep Partial Multi-View Learning](/papers/2011.06170): CPM-Nets for supervised partial multi-view classification, and CPM-GAN for unsupervised imputation and clustering.