A 3D Probabilistic Deep Learning System for Detection and Diagnosis of Lung Cancer Using Low-Dose CT Scans

In plain words

Doctors reviewing low-dose chest scans look for small spots (nodules) that might be cancer. There are many scans and many spots to check, so software helps. This paper builds such a helper from three-dimensional neural networks. It first finds candidate spots in the scan, then judges how suspicious each spot is, and finally gives one overall score for the patient.

The software also says how sure it is. Each score is meant to be a real probability, so a doctor can act on it. The authors use a few tricks to get honest probabilities. A high score should mean a high chance of cancer, not just a confident-looking number. They also show that the software can send its least certain cases to a radiologist, which improves the results on the cases it keeps.

The main point is that the spot finder and the cancer judge should be built together. A finder tuned only to find as many spots as possible can flood the judge with false alarms, and the judge's results depend on that finder.

TL;DR

• A 3D CNN pipeline with two stages: a candidate-nodule segmenter and scorer (CADe), then a malignancy ranker and attention-based patient classifier (CADx) §III.
• The segmenter and classifier are developed jointly, because the CADx result depends on the candidates CADe produces §I §VII-C.
• Uncertainty comes from MC dropout, and for patient classification also from averaging five models trained on different splits (deep ensembles) §VI-B §VIII.
• Author-reported: LUNA16 average metric 0.921 and Kaggle Stage-2 AUROC of about 0.87 tab. 1 fig. 8.
• No official code was found; the card relies on the paper text alone.

Problem

Low-dose CT lung screening produces many scans, and reading each one takes time. Automated detectors often report only a score per candidate, and they do not say how reliable that score is §I. Nodule detectors are usually tuned on their own benchmark, which can trade false positives against recall in a way that hurts a downstream diagnosis step §I §VII-C. The datasets used here lack very large nodules, since LUNA16 annotators skipped nodules above 30 mm §IV-A §VI-B.

Method

Section numbers follow the journal layout (Roman numerals for sections, letters for subsections, for example §V-A). Equation numbers are those of the compiled paper.

Overview. A raw 3D CT scan is preprocessed, then passed through CADe (a 3D segmentation network that labels voxels, followed by candidate extraction and a 3D scoring network), then CADx (a malignancy ranking network, then an attention-based multiple-instance classifier). The CADx classifier outputs one malignancy probability per patient and one risk score per candidate §III fig. 1.

Preprocessing §IV-B. Intensities are clipped to a Hounsfield-unit window and the scan is resampled to 1 mm isotropic voxels. Each volume is normalised to zero mean and unit variance. Augmentation uses random 3D rotations and reflections, small random scaling and translation, random gamma, blur or unsharp masking, and additive noise, applied only in training §IV-B fig. 2. The CADe segmenter also uses an extra upscaling augmentation, so it can see nodules larger than any annotated one §IV-B.

CADe segmentation §V-A. A 3D fully-convolutional network based on V-Net §V-A. It has three encoder-decoder pairs with skip connections, plus input and output blocks. Encoder blocks use a stride-2 downsampling convolution, two kernel-3 convolutions and a residual connection to the output. Decoder blocks mirror this with a stride-2 upsampling deconvolution. The innermost pair has channel-wise dropout. Instance normalisation and ReLU are used throughout §V-A. Feature counts were tuned on LUNA16 and are not stated in the text §V-A, footnote.

Training draws 64³ blocks from scans. For a patient with N nodules, a block is drawn near a known nodule with probability 1 − 0.7^N, and drawn at random otherwise §V-A. The loss is voxel cross-entropy with nodule voxels weighted 2× against background §V-A. Each scan is visited once per epoch. Full-scan inference splits each volume into eight overlapping 256³ blocks, and stitches the outputs with edge weights that reflect how much of each voxel's field of view lies inside its block §V-A.

Candidate extraction §V-A fig. 3. Voxel probabilities are thresholded, a nearest-neighbour binary opening is applied, and connected regions (connectivity 1) are labelled. Each candidate's centre is the centre of mass of its thresholded voxel scores §V-A.

CADe scoring network §V-B. A second 3D network scores 32³ blocks around each candidate centre. It uses the base architecture of fig. 4. Monte Carlo dropout is used for uncertainty §V-B §VIII.

Malignancy ranking §VI-A. The ranker has the same basic structure as the scorer §VI-A. It regresses a target derived from the four-reader LIDC-IDRI malignancy ratings: nodules with at least three ratings are averaged, and all other candidates (fewer than three ratings, too small to rate, or false positives) get the value 1 §VI-A. Training uses a curriculum. For the first 50 epochs nodules are drawn at random among those with at least three ratings. Afterwards, 90% of draws come from that set and 10% from the nodules whose target is 1 §VI-A.

Candidate sets for CADx §VI-B. Candidates come from two CADe runs: one on the original 1 mm scans, and one on scans downsampled to 2 mm voxels, to reach large nodules §VI-B. Each set is ranked by the ranker, and the top-k of each is kept. The best model keeps the top 2 from each list, four candidates in total §VI-B §VII-B.

Attention-based multiple-instance classifier §VI-B. A shared CNN (base architecture of fig. 4 without its final dense layer) maps each candidate to a 1024-dimensional feature vector h_i §VI-B. The vectors are combined by attention:

\mathbf{x}_i = \tanh(\mathbf{W}_1 \mathbf{h}_i) eq. 1

\mathbf{y} = \mathrm{softmax}(\mathbf{w}_2^T \mathbf{X}) eq. 2

p = \mathrm{sigmoid}(\mathbf{w}_3^T \mathbf{H} \mathbf{y}) eq. 3

Here W_1 is 128×1024, w_2 is 128×1 and w_3 is 1024×1; X and H stack the x_i and h_i of the k candidates as columns. The softmax in eq. 2 runs over the k candidates, so y is a weighting that sums to one. The text notes that bias terms exist in the implementation but are omitted from the equations §VI-B. Noisy-OR, Leaky Noisy-OR and log-sum-exp pooling were tried and did not work as well in the authors' experiments, and were harder to tune §VI-B.

Uncertainty §VI-B §VIII. For CADe scoring, ten models were trained, each with one LUNA16 subset held out, and dropout is kept on at test time (MC dropout) §VIII. For CADx, five models were trained on different train/development splits, and their MC-dropout predictions are combined. The text calls this a combination of MC dropout and deep ensembles §VI-B §VIII.

Referral §VIII. Cases are ordered by the binary entropy of their probability, eq. 4, and the most uncertain are referred to a radiologist. The area under the precision-recall curve for CADe and the ROC area for CADx are then measured on the remaining cases, as a function of the referral rate §VIII fig. 11 fig. 12.

H(p) = -p \ln(p) - (1-p) \ln(1-p) eq. 4

Training data §IV-A §VI-B. LUNA16 (a curated LIDC-IDRI subset, 888 patients, 1186 consensus nodules) serves CADe and part of CADx. Kaggle Data Science Bowl 2017 Stage-1 (1595 patients) trains CADx and Stage-2 (506 patients) tests it §IV-A. The CADx training set is 2101 patients: 556 from LUNA16 plus 1595 from Kaggle Stage-1 §VI-B. Of the 556 LUNA16 patients, 169 are labelled malignant. Malignant means a nodule with at least three ratings and an average of at least 4; benign means no annotations, or all nodules averaging at most 2 §VI-B. Of the training set, 300 patients were set aside for development §VI-B.

Pseudocode of inference.

# CADe + CADx inference  [§V] [§VI] [§VIII]
# Input:  preprocessed scan, (D, H, W) at 1 mm voxels
# Output: one patient probability, one score per candidate

def diagnose(scan, threshold=0.5, k=2):
    # Segment: voxel nodule probabilities, eight overlapping blocks  [§V-A]
    voxel_probs = segment_in_blocks(scan)           # (D, H, W)
    # Extract candidates: threshold, open, label, centre of mass  [§V-A]
    candidates = extract_candidates(voxel_probs, threshold)
    # Rank candidates by malignancy regression  [§VI-A]
    ranked_full = rank(candidates)
    # Repeat on a 2 mm-downsampled scan for large nodules  [§VI-B]
    ranked_small = rank(candidates_from(downsample(scan)))
    # Keep top-k from each list  [§VI-B] [§VII-B]
    chosen = top(ranked_full, k) + top(ranked_small, k)
    # Shared CNN feature per candidate  [§VI-B]
    features = [shared_cnn(c) for c in chosen]        # (n, 1024)
    # Attention pooling over candidates  [eq. 1] [eq. 2]
    weights = attention_softmax(features)             # (n,), sums to 1
    pooled = weights @ features                       # (1024,)
    # Patient probability  [eq. 3]
    return sigmoid(w3 . pooled)

Hyperparameters

Values in the same row as "not stated" are not given in the paper or the text; no official code was found to fill them in (see Paper vs code).

Results

Author-reported. Setup: LUNA16 with 10-fold cross-validation on the prescribed splits §VII-A for CADe. Kaggle Stage-2 as CADx test set, trained on Stage-1 plus LUNA16 patients §VII-B. Baseline: 3D U-Net in place of V-Net, and the LUNA16 scores reported by others §VII-A.

The CADe referral (precision-recall curve against referral rate) and the calibration curves are shown only as figures fig. 9 fig. 10 fig. 11. The text reports the gains qualitatively, so no numbers are listed for them here.

Paper vs code

No official code. Searches: the LaTeX source has no repository link, and arXiv metadata has no comments field. A GitHub repository search for the title terms and method name returned only unrelated third-party projects. Searches limited to the authors' organisation (Draper) returned errors. No discrepancy check was possible.

Pitfalls

• Best CADx settings are chosen on the test figure. The threshold of 0.5 is called the best performing CADx model, and the figure that shows it is the Stage-2 ROC §VII-B fig. 8. The text does not say which split chose that threshold. k = 4 was chosen on the development set §VII-B. Treat the Stage-2 AUROC of 0.87 as optimistic until the selection split is known.
• "Top-k" is ambiguous. The text says the best CADx model uses the top four candidates per patient, and also top two from each of the two lists §VII-B §VI-B. Keep two per list. Whether candidates from the two lists are ever ranked against each other is not stated.
• Scaling augmentation unit. The scaling is written as N(0, 0.06)% §IV-B. Whether 0.06 is a standard deviation, a variance or a fraction in percent is not stated. Check the unit before implementing.
• Noise sampling sign. Additive noise has σ drawn from N(0, 0.03) §IV-B. A normal draw can be negative and is not clipped in the text. Use the absolute value, or confirm the convention.
• Labels for the classifier are partly undefined. A patient is positive with a rated nodule averaging at least 4. Negative means no annotations, or every nodule averaging at most 2. Patients with a mix of averages (for example one nodule at 3 and one at 5) are not covered §VI-B.
• Ranker target for false positives. Candidates with fewer than three ratings, too small to rate, or false positive get a score of 1 §VI-A. These are numerically the lowest class, so the curriculum matters.
• Voxel size for the second candidate set. The 2 mm scan is passed through the same trained CADe and ranker §VI-B. Whether block sizes or thresholds change for 2 mm voxels is not stated.
• Attention biases are omitted from eq. 1–eq. 3 §VI-B. Include them in an implementation.
• Uncertainty is not fully specified. The dropout rate, the number of MC samples, and where dropout sits in the scorer and the classifier are not stated. The paper says MC dropout is used at test time, so implement it that way, but the numbers must be chosen by you.
• Calibration was measured on one held-out set. The authors expect domain shift to matter more but did not measure it §VIII.
• Stitched inference. The authors state that stitched block inference matches a single pass over the full scan on CPU §V-A. This is their claim and was not tested here.
• Compute. The scorer and segmenter train for 2500 epochs, and the segmenter sees each patient once per epoch §V-A §V-B. Full training used a single GP100-class GPU [fn. in §V-A].
• Large nodules are a known gap. LUNA16 excludes nodules above 30 mm §VI-B, and the paper names missed malignant nodules as the main source of false negatives §IX fig. 13.
• "State of the art" is hedged in the text. The results are "comparable to or better than" the cited benchmark on LUNA16 §VII-A. Do not treat the abstract's wording as a stronger claim than the tables support.

Lineage

• Builds on: V-Net for 3D segmentation §V-A; attention-based multiple-instance learning (Ilse et al., 2018) for the classifier §VI-B; MC dropout (Gal and Ghahramani) for uncertainty §V-B; deep ensembles (Lakshminarayanan et al.) for classification uncertainty §VI-B.
• Also builds on the authors' earlier candidate library for CADe §VI-A.
• Compared against: 3D U-Net in the same pipeline §VII-A; a benchmark on LUNA16 (Khosravan and Bagci, S4ND) §VII-A; the Kaggle 2017 winning solution §VII-B.
• Superseded by: not known from content already in this wiki.