Machine learning methods for histopathological image analysis

In plain words

Doctors look at tissue slices under a microscope to decide whether they show cancer. Scanners now turn each slice into one enormous digital image, and computers can be trained to help read these images. This short review explains how that works, and why it is harder than recognising cats and dogs in photos: the images are huge, labelled examples are scarce and expensive because only pathologists can label them, the colours vary between labs, and the slides often have damage from preparation.

For each of these problems the review lists the ways researchers have tried to handle it, such as splitting the image into small tiles, learning from labels that only exist for the whole slide, reusing models trained on ordinary photos, and normalising the colours. It also points to open questions for the future.

TL;DR

• A mini-review, not a method paper: it reports no experiments of its own.
• It describes the patch-based pipeline for whole-slide images §2 §4.1.
• It maps recurring problems to proposed solutions, summarised in tab. 1.
• Its own datasets tables list public whole-slide and annotated image sets tab. 2 tab. 3.

Problem

Digital pathology images are very large, and labelled data is scarce §4.1 §4.2. Whole-slide images can reach tens of billions of pixels, and resizing them to a standard network input (about 256 × 256) loses cellular detail §4.1. Most labels exist only at the slide level, while many tasks need patch- or pixel-level labels §4.2. The authors also list colour variation and artifacts from slide preparation and scanning as problems that change how a model should be built §4.5.

Method

This is an overview, not a single method. The card summarises the pipeline and the solution families the review names.

Pipeline. Pre-processing samples local patches from a large slide, for example around 256 × 256 §2. Features are then extracted from each patch, either hand-crafted (grey-level co-occurrence matrix, local binary pattern) or learned by a convolutional network, and a classifier separates cancer from non-cancer §2. The paper notes that deep networks optimise features and classifier jointly §2.

# Patch-based whole-slide analysis  [§2] [§4.1]
# Input:  whole-slide image wsi; patch size (about 256 px)  [§2]
# Output: one slide- or region-level decision

def analyze_slide(wsi, patch_size):
    # 1. Sample local patches from the large image  [§2]
    patches = tile(wsi, patch_size)
    # 2. Extract features per patch: hand-crafted (GLCM, LBP) or learned  [§2]
    features = [extract(patch) for patch in patches]
    # 3. Classify each patch as cancer or non-cancer  [§2]
    patch_scores = [classify(feature) for feature in features]
    # 4. Combine patch decisions into one result; the paper calls this
    #    integration an open problem, e.g. regional averaging  [§4.1]
    return integrate(patch_scores)

Task families. The review lists computer-assisted diagnosis (supervised, mapping a slide or slides to a disease category) §3.1, content-based image retrieval (similar-image search, using dimensionality reduction and fast approximate nearest-neighbour search) §3.2, and discovery of clinicopathological relationships such as correlating image features with prognosis §3.3.

Problems and solution families (the review's own grouping, summarised in tab. 1):

1. Very large image size §4.1: case-level classification that summarises patch- or object-level results, e.g. random forests over patch probabilities §4.1 tab. 1. Larger patches (e.g. 960 × 960) are mentioned as a trend that may help accuracy §4.1.
2. Insufficient labelled images §4.2:
   • efficient labelling with GUI tools §4.2.1;
   • automatic region localisation by tracking pathologists' eye, mouse or viewport movements §4.2.1;
   • active learning, which chooses the most valuable unlabelled sample to label next, with uncertainty sampling, query-by-committee, variance reduction and hypothesis-space reduction as criteria §4.2.1;
   • multiple instance learning, where a slide is a bag and a cancer slide has at least one cancer patch §4.2.2;
   • semi-supervised learning, which uses unlabelled data to estimate the label distribution §4.2.2;
   • transfer learning: use early CNN layers as feature extractors, then fine-tune; ImageNet-pretrained networks are common because no large pathology pretrained network is available §4.2.3.
   • public datasets, some with case-level labels, need preprocessing or weak-label methods to be used for training §4.2 tab. 2 tab. 3.
3. Different magnifications carry different information §4.3: cell shape is best seen at high power, glandular structure at low power. The best magnification is called controversial; combining high and low magnification sometimes helps §4.3.
4. WSI as a texture-like image §4.4: the authors argue that patches are closer to texture than to objects. Traditional texture features (grey-level co-occurrence, local binary pattern, Gabor filter banks) and CNN-based deep texture representations are discussed; the latter place no constraint on image size §4.4.
5. Colour variation and artifacts §4.5: colour normalisation matches the pixel colour distribution to a reference, and needs nuclei or tissue composition to be similar §4.5. Colour augmentation applies random hue, saturation, brightness and contrast changes §4.5. Grey-scale conversion is the simplest option but loses colour §4.5. Blur and tissue folds are artifacts with their own detection methods §4.5 fig. 3.

Outlook (the review's suggested topics, not evaluated) §5: outlier detection for unseen tissue types (one-class kernel PCA; reconstruction error) §5; interpretability (attention with reports, visualising network responses, influence functions) §5; intraoperative analysis of frozen sections, which is little studied §5; and quantifying tumour-infiltrating immune cells from H&E and immunohistochemistry, including graph-based analysis of their spatial relations §5.

Hyperparameters

The review gives no training hyperparameters. The numbers it does state:

Results

No experiments are reported. The review has no results of its own, and no headline numbers from other papers are reported in the text. The closest quantitative material is the dataset summary.

Public whole-slide datasets tab. 2 (author-compiled; the paper states the counts as given):

Hand-annotated public image sets tab. 3 include, for example, BreakHis (7,909 images, 700 × 460 px, breast cancer, H&E, disease classification), MITOS-ATYPIA-14 (4,240 images, mitosis coordinates and nuclear-atypia criteria) and a Janowczyk et al. tumour-detection set (277,524 images of 50 × 50 px, breast cancer) tab. 3. Read the table for the full list.

Paper vs code

Survey paper; there is no implementation to compare. No official code. No GitHub search was run, because surveys have no code of their own (repo convention for survey cards).

Pitfalls

• No method to reproduce. Nothing here is validated by experiments in the paper. Every "solution" is a pointer to other work, and the review says so in its own words: several outlook topics "have not been well studied so far" §5.
• Figures are described from the text only. The paper cites Figures 1 to 4 §2 §4.2.2 §4.3 §4.5. The 20 PDF pages read for this card show no figure graphics, so their content is not verified here. Use the text for what each figure is meant to show.
• Figure 1 is cited for two different things. Section 2 says it shows the typical analysis steps §2; section 4.2.2 says it illustrates the decision boundary of semi-supervised learning §4.2.2. Treat the two references as distinct content until the figure is checked.
• Table reference mismatch. Section 4.2 says public datasets are "summarized in Table 1 and Table 2" §4.2. Table 1 is the problem-to-solution overview tab. 1; the dataset tables are Table 2 and Table 3.
• Citation [43] is used twice in Table 1, once for GUI tools (web server) and once for uncertainty sampling in active learning tab. 1. Reference [43] is a web page on histology phenotyping tools [ref. 43]. The active-learning attribution is therefore ambiguous.
• Dates. The review's references run to 2017 [refs. 1–120]. Later work, including later large pathology datasets, is not covered.
• Authors' opinions, not evidence. "Most of these problems are likely to be solved once a large number of well-annotated WSIs become available" §5 is a prediction. The claim that a pathology-pretrained model "may outperform" an ImageNet-pretrained one is also not tested §4.2.3.
• Untested combinations. The authors suggest combining colour normalisation and colour augmentation may be better than either §4.5. No experiment is given.
• Terminology. In multiple instance learning a whole-slide image is the "bag" and its patches are the "instances" §4.2.2. A slide-level diagnosis is the bag label §4.2.2.

Lineage

• Builds on: none; this is a review. It cites earlier reviews on medical and histopathology image analysis as background §1 [refs. 2–7].
• Compared against: not applicable.
• Superseded by: not known from wiki content.