Intraclass correlation coefficient

Overview

The intraclass correlation coefficient (ICC) is a reliability statistic for values measured repeatedly on the same subjects. The paper that uses it reads ICC(1,1), the one-way form of Shrout and Fleiss, and treats a value of at least 0.90 as reliable 1806.06719 §4.5. In that paper, a radiomic feature is "robust" when its ICC reaches this threshold, so the statistic becomes a screen: keep the numbers that agree across repeated measurements, drop the rest 1806.06719 §2.

The same statistic can be computed in two ways. The reference is test-retest: two scans of each patient, taken close in time. The alternative is perturbation: many altered copies of one scan, which replace the second scan 1806.06719 §1 1806.06719 §2.3. Both feed the same 0.90 rule.

Only one paper in the wiki uses this statistic. Its figures are author-reported on two small CT cohorts 1806.06719 §3.

Variants

Test-retest ICC(1,1)

The reference in the paper. Each feature is computed on the test scan and the retest scan of every patient, and ICC(1,1) is taken across patients 1806.06719 §4.5. In the NSCLC cohort (31 patients, two scans within 15 minutes) 73.5% of 4032 features reached 0.90. In the HNSCC cohort (19 patients, two scans within 4 days, different PET protocols) 34.0% did 1806.06719 §2.1 1806.06719 §4.1. Source: Assessing robustness of radiomic features by image perturbation.

Perturbation ICC(1,1)

Used when a second scan is not available. For each image, the feature is computed on many perturbed copies, and ICC(1,1) is taken over those copies. The score for a chain of perturbations is the average of the value for the test image and the value for the retest image 1806.06719 §4.5. The paper compares 18 chains of rotation, noise, translation, volume change and contour randomisation with the test-retest result, feature by feature 1806.06719 §2.3. Card: Assessing robustness of radiomic features by image perturbation.

When to use which

Implementation sketch

Robust flag for one feature, from its ICC. The perturbation ICC averages the ICC of the test image and of the retest image 1806.06719 §4.5.

# Robust flag and perturbation ICC  [arxiv:1806.06719 §4.5]
# Input:  feature values (patients x repeats) for test or retest images
# Output: ICC(1,1) and robust flag

def is_robust(icc):
    # Robust when ICC(1,1) reaches the threshold  [arxiv:1806.06719 §4.5]
    return icc >= 0.90

def perturbation_icc(values_test, values_retest):
    # ICC computed separately on the perturbations of each image
    icc_test = icc_1_1(values_test)
    icc_retest = icc_1_1(values_retest)
    # Average of the two  [arxiv:1806.06719 §4.5]
    return (icc_test + icc_retest) / 2

The paper does not write the ICC formula. It says the code is based on the R package psych 1806.06719 §4.5. Which factor is the subject and which the rater, for the perturbation case, is not stated.

Contradictions

Only one paper uses the statistic, so no disagreement between papers. Inside the paper, one point needs care. The summary of the paper says NTVC gave the fewest false-positive robust features. Its own table gives a lower share for RC in NSCLC, 2.6% against 3.3% for NTVC 1806.06719 §2.3. The paper itself notes this 1806.06719 §3. In HNSCC NTVC is lowest, at 10.0%.

Papers

• Assessing robustness of radiomic features by image perturbation: uses ICC(1,1) at 0.90 as the robustness criterion, against test-retest and against perturbed copies, on 31 NSCLC and 19 HNSCC CT patients.