--- kind: concept slug: multiple-instance-learning title: Multiple instance learning summary: >- Multiple instance learning trains a classifier on bags of instances that carry one label per bag, such as a whole-slide image split into patches. Pooling methods combine instance scores with a fixed rule. Correlated variants let instances attend to each other with a Transformer, which needs an attention approximation for bags of thousands of patches. tags: [weakly-supervised-learning, pathology, transformer] papers: ["1709.00786", "2106.00908"] related: [weakly-supervised-learning, self-attention, linear-attention, positional-encoding, vision-transformer, attention-mechanism, active-learning, semi-supervised-learning] status: draft updated_at: 2026-10-19 --- ## Overview In multiple instance learning (MIL), the label belongs to a bag, a set of instances. Here a bag is one whole-slide image and its instances are patches. A binary bag is 0 only if every instance is 0, and 1 otherwise. Instance labels are not known [arxiv:2106.00908 §3.1] [arxiv:2106.00908 eq. 1]. The shared rule is that the model must give one bag-level prediction from instance features. The variants differ in how they combine instances before that prediction. A 2017 review of histopathology methods lists MIL as one answer to scarce labels. It uses the same bag rule: a slide is the bag, and a cancer slide contains at least one cancer patch [arxiv:1709.00786 §4.2.2]. The review reports no experiments, so this page takes from it the framing and terminology only, and no result. Many MIL methods treat the instances in a bag as independent and identically distributed. TransMIL argues that this often fails in pathology, because a pathologist uses both the context around a region and the relations between regions [arxiv:2106.00908 §1]. The size of the input is part of the problem: CAMELYON16 has about 8,800 patches per bag on average at x20 magnification [arxiv:2106.00908 §1] [arxiv:2106.00908 §4]. ## Variants ### Independent instances with pooling Each patch is scored on its own, and a fixed rule combines the patch outputs into one bag score. The paper uses mean-pooling and max-pooling as baselines [arxiv:2106.00908 tab. 1]. On CAMELYON16 their AUCs are 0.4647 and 0.8569. On TCGA-RCC they are 0.9786 and 0.9879 [arxiv:2106.00908 tab. 1]. The choice of rule matters on CAMELYON16 and barely on TCGA-RCC, so a pooling rule should be checked on the task at hand. The baselines are reported in [TransMIL](/papers/2106.00908). ### Correlated instances with a Transformer (TransMIL) Instances in one bag interact through self-attention. A two-layer Transformer runs over the patch features with a class token, and a convolutional position encoder (PPEG) sits between the two layers [arxiv:2106.00908 §3.3] [arxiv:2106.00908 alg. 2]. Attention is approximated with the Nystrom method, which the authors say cuts the cost from quadratic to linear in the number of instances [arxiv:2106.00908 §3.3]. In the paper's ablation on CAMELYON16, removing position encoding gives AUC 0.8416 and PPEG gives 0.9309, at 2.625M and 2.669M parameters [arxiv:2106.00908 tab. 2]. The full method is in [TransMIL](/papers/2106.00908). The same table names other MIL methods, ABMIL, DSMIL, CLAM-SB, CLAM-MB, MIL-RNN and PT-MTA [arxiv:2106.00908 tab. 1]. No cards exist for them yet, so this page does not describe how they work. ## When to use which | Situation | Use | Why | |---|---|---| | Bags are large and attention over all instances is too costly | Pooling baseline, or an approximate-attention variant | Nystrom attention is the authors' route to linear cost [arxiv:2106.00908 §3.3]. Pooling needs no attention step | | Only bag labels exist and the task may depend on context between regions | Correlated Transformer (TransMIL) | Reports the highest AUC of the compared methods on CAMELYON16, NSCLC and RCC [arxiv:2106.00908 tab. 1] | | Max-pooling is already close to the best reported AUC, as on TCGA-RCC | Pooling is a serious candidate | On TCGA-RCC, max-pooling AUC is 0.9879 against TransMIL's 0.9882 [arxiv:2106.00908 tab. 1]. Both are averages over 4-fold CV | | Tight GPU memory budget | Plan for batch size 1 with the Transformer | Trained with batch size 1 on one RTX 3090. No memory figures are reported [arxiv:2106.00908 §4] | | Commercial product | Reimplement from the paper, not the official code | Official code is for non-commercial academic use. See [the card's Paper vs code section](/papers/2106.00908) | The author-reported gains come from single splits and report no standard deviations [arxiv:2106.00908 §4]. Treat the margins as indicative only. ## Implementation sketch Pooling and correlated variants share the instance encoder and the bag head. Only the aggregation step differs. ```pseudocode # Shared MIL core: instance features -> one bag prediction [alg. 1] # Input: bag of n patch features, shape (n, d); one label per bag [§3.1] # Output: bag-level probability def encode(bag): # Project each patch to a shared width, e.g. 1024 -> 512 [§4] return linear(bag) # (n, 512) def pooled_mil(bag): # Independent instances: one fixed rule over patch scores [tab. 1] h = encode(bag) pooled = mean_or_max(h, axis=0) # (512,) return softmax(head(pooled)) def correlated_mil(bag): # Class token first, then two attention layers with PPEG between [alg. 2] h = concat(class_token, encode(bag)) # (n+1, 512) h = h + attention(layer_norm(h)) # correlate instances h = ppeg(h) # position encoding [alg. 3] h = h + attention(layer_norm(h)) # second correlation step return softmax(head(layer_norm(h)[0])) # read the class token ``` Train with cross-entropy on the bag label [§4]. The paper's training schedule and the optimizer details are in the card's Hyperparameters section, where values the paper leaves unstated are marked. ## Contradictions None known. The review [arxiv:1709.00786 §4.2.2] gives no numbers, so it cannot conflict with the TransMIL results. Its bag rule matches the one in [arxiv:2106.00908 eq. 1]. ## Papers - [Machine learning methods for histopathological image analysis (Komura and Ishikawa, 2017)](/papers/1709.00786): a mini-review that names MIL as one way to handle scarce slide-level labels. No experiments. - [TransMIL (Shao et al., 2021)](/papers/2106.00908): a two-layer Transformer over all patch features of a slide, with Nystrom attention and a convolutional position encoder.