Multi-view clustering

Overview

Each sample is described by V views, for example colour and texture features of the same image. The goal is one partition of the N samples into k clusters, using all views 2203.11572 §1. Many methods build an N-by-N affinity matrix and run spectral clustering on it. That costs memory and time that grow quickly with N 2203.11572 §1. The design problem is therefore how to combine views and how to keep the cost low, while still avoiding the per-dataset tuning that many methods need 2203.11572 §1 2203.11572 §2.1.

Two choices shape the methods that assume every view is present:

• Where views are fused. Early fusion merges all views into one model. Late fusion clusters each view and merges the results. The paper argues that multi-stage fusion is less explored 2203.11572 §1 2203.11572 §2.1.
• How the graph is built. Anchor-based methods pick a small set of representative points, so the graph has N-by-p entries instead of N-by-N. Methods differ in whether all views share one anchor set or each view has its own 2203.11572 §1.

A third case is a view that is missing for some samples, for example when different exams are run on different patients 2011.06170 §1. Here the question is how to build one representation per sample from whatever views it has, instead of imputing or grouping by missing pattern 2011.06170 §1.

Variants

Random view-group ensemble (FastMICE)

Draw M random groups of views. Each group yields one anchor-based bipartite graph and one base clustering. The M base clusterings are then fused in one consensus step 2203.11572 §3.1. Early fusion (one group with all views) and late fusion (one group per view) are special cases of this scheme 2203.11572 §3.2. Within a group, each view is also subsampled: a random fraction of its columns is kept, and the anchors and neighbours are shared among the views of the group 2203.11572 §3.3. Card: Fast Multi-view Clustering via Ensembles.

The paper's ensemble-size and anchor results are reported as sensitivity curves in figures, without numbers in the text 2203.11572 §4.4 2203.11572 §4.5 2203.11572 §4.6. The general ensemble idea is covered on Ensemble clustering.

Shared code with missing views (CPM-Nets)

Each sample gets one latent code, built from whichever of its views are observed. A network per view decodes the code back to that view, and the reconstruction error counts only the observed views 2011.06170 §2.1. A class-structure loss pulls codes of the same class together, which also helps the codes separate for clustering 2011.06170 §2.2. For clustering, the paper runs k-means on the codes and reports accuracy (ACC) and normalised mutual information (NMI) on six multi-view sets 2011.06170 §4.4.2. At missing rate η = 0.5, it reports average gains of 3.36% in ACC and 5.11% in NMI over the second-best method. The text does not say whether these gains are absolute or relative 2011.06170 §4.4.2. Filling missing values with the per-view average performs poorly in the same experiment 2011.06170 §4.4.2.

The paper also has a GAN variant, CPM-GAN, that imputes missing views before the codes are clustered 2011.06170 §3. It reports that the version without the GAN is sometimes slightly better at low missing rates, on Football or CUB 2011.06170 §4.4.2. On CUB and Animal, the paper also compares with VIGAN, which needs complete-view samples. The paper gives VIGAN 10% complete data, and reports higher ACC and NMI for CPM-Nets 2011.06170 §4.4.2. Card: Deep Partial Multi-View Learning.

When to use which

Every number is author-reported. The two papers study different settings, so the rows compare the methods by constraint rather than by a single score.

Implementation sketch

The paper has no algorithm box. This sketch is our summary of its four stages 2203.11572 §3. Parameter values are the paper's defaults 2203.11572 §4.2.

# Multi-view clustering, FastMICE-style  [arxiv:2203.11572 §3]
# Input: V views (N x d_v each), k clusters; M = 20, p = 1000, K = 5

def multi_view_cluster(views, k):
    base = []
    # Stage 1: M random groups of views, each with 1..V members  [§3.2]
    for group in random_view_groups(num_views, M):
        blocks = []
        for v in group:
            # Keep a random fraction of columns  [§3.3]
            X = keep_random_columns(views[v], ratio=uniform(0.2, 0.8))
            # Pick about p/|group| representatives, then link each sample
            # to its K/|group| nearest  [§3.3]
            blocks.append(anchor_graph(X, p // len(group), K // len(group)))
        # Stack the member graphs side by side  [§3.3]
        B = hstack(blocks)
        # Partition the group graph into k_m clusters, k_m in [k, 2k]  [§3.4]
        base.append(transfer_cut(B, uniform_int(k, 2 * k)))
    # Stage 4: one bipartite graph over all base clusters  [§3.5]
    return transfer_cut(one_hot_union(base), k)

The official code differs from the text in several details, such as scaling, row normalisation and k-means settings. The paper card lists them, so check them before you copy the sketch 2203.11572 §3.4. For the missing-view variant, the training and test loops are on its card, not in this sketch.

Contradictions

No direct disagreement is recorded. The two papers study different settings: one assumes all views are present, the other handles missing views 2203.11572 §1 2011.06170 §1. They also use different dataset sets and metric sets, so their numbers are not comparable 2203.11572 tab. 3 2011.06170 §4.4.2.

Papers

• Deep Partial Multi-View Learning: one shared code per sample from the observed views, with a reconstruction loss on those views and a class-structure loss; reports clustering gains on six partial-view sets.
• Fast Multi-view Clustering via Ensembles: Towards Scalability, Superiority, and Simplicity: anchor-based ensemble clustering over random view groups, with near-linear cost and no per-dataset tuning, reported on 22 datasets.