---
kind: concept
slug: intraclass-correlation-coefficient
title: Intraclass correlation coefficient
summary: >-
  ICC(1,1) scores how much of a measure's variation comes from the subject rather than from repeated measurements. Radiomics uses it as a robustness screen at 0.90, against test-retest scans or against perturbed copies of one scan. Only one paper in the wiki uses it, on CT.
tags: [reliability, agreement, icc, radiomics, feature-robustness, medical-imaging]
papers: ["1806.06719"]
related: [inter-annotator-agreement, data-augmentation]
status: draft
updated_at: 2026-11-08
---

## Overview

The intraclass correlation coefficient (ICC) is a reliability statistic for
values measured repeatedly on the same subjects. The paper that uses it reads
ICC(1,1), the one-way form of Shrout and Fleiss, and treats a value of at least
0.90 as reliable [arxiv:1806.06719 §4.5]. In that paper, a radiomic feature is
"robust" when its ICC reaches this threshold, so the statistic becomes a
screen: keep the numbers that agree across repeated measurements, drop the rest
[arxiv:1806.06719 §2].

The same statistic can be computed in two ways. The reference is test-retest:
two scans of each patient, taken close in time. The alternative is perturbation:
many altered copies of one scan, which replace the second scan
[arxiv:1806.06719 §1] [arxiv:1806.06719 §2.3]. Both feed the same 0.90 rule.

Only one paper in the wiki uses this statistic. Its figures are author-reported
on two small CT cohorts [arxiv:1806.06719 §3].

## Variants

### Test-retest ICC(1,1)

The reference in the paper. Each feature is computed on the test scan and the
retest scan of every patient, and ICC(1,1) is taken across patients
[arxiv:1806.06719 §4.5]. In the NSCLC cohort (31 patients, two scans within
15 minutes) 73.5% of 4032 features reached 0.90. In the HNSCC cohort
(19 patients, two scans within 4 days, different PET protocols) 34.0% did
[arxiv:1806.06719 §2.1] [arxiv:1806.06719 §4.1]. Source:
[Assessing robustness of radiomic features by image perturbation](/papers/1806.06719).

### Perturbation ICC(1,1)

Used when a second scan is not available. For each image, the feature is
computed on many perturbed copies, and ICC(1,1) is taken over those copies.
The score for a chain of perturbations is the average of the value for the
test image and the value for the retest image [arxiv:1806.06719 §4.5]. The
paper compares 18 chains of rotation, noise, translation, volume change and
contour randomisation with the test-retest result, feature by feature
[arxiv:1806.06719 §2.3]. Card: [Assessing robustness of radiomic features by image perturbation](/papers/1806.06719).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Test-retest scans exist for the same patients | Test-retest ICC(1,1), threshold 0.90 | It is the reference the paper scores every perturbation chain against [arxiv:1806.06719 §2.3] |
| Only one scan per patient | Perturbation ICC(1,1) with a chain that gave few false positives | The paper reports 3.3% (NSCLC) and 10.0% (HNSCC) false positives for NTVC against test-retest [arxiv:1806.06719 §2.3] |
| Noise is the only perturbation | Do not rely on it alone | Noise alone marks 96.6% (NSCLC) and 99.3% (HNSCC) of features robust [arxiv:1806.06719 §2.2] |
| Imaging modality other than CT | Not covered by this evidence | The paper studies CT only [arxiv:1806.06719 §4.1] |
| Small cohorts or a few patients | Report the confidence interval | Test-retest CI width averages 0.12 (NSCLC) and 0.35 (HNSCC) [arxiv:1806.06719 §3] |

## Implementation sketch

Robust flag for one feature, from its ICC. The perturbation ICC averages the
ICC of the test image and of the retest image [arxiv:1806.06719 §4.5].

```pseudocode
# Robust flag and perturbation ICC  [arxiv:1806.06719 §4.5]
# Input:  feature values (patients x repeats) for test or retest images
# Output: ICC(1,1) and robust flag

def is_robust(icc):
    # Robust when ICC(1,1) reaches the threshold  [arxiv:1806.06719 §4.5]
    return icc >= 0.90

def perturbation_icc(values_test, values_retest):
    # ICC computed separately on the perturbations of each image
    icc_test = icc_1_1(values_test)
    icc_retest = icc_1_1(values_retest)
    # Average of the two  [arxiv:1806.06719 §4.5]
    return (icc_test + icc_retest) / 2
```

The paper does not write the ICC formula. It says the code is based on the R
package `psych` [arxiv:1806.06719 §4.5]. Which factor is the subject and which
the rater, for the perturbation case, is not stated.

## Contradictions

Only one paper uses the statistic, so no disagreement between papers. Inside
the paper, one point needs care. The summary of the paper says NTVC gave the
fewest false-positive robust features. Its own table gives a lower share for
RC in NSCLC, 2.6% against 3.3% for NTVC [arxiv:1806.06719 §2.3]. The paper
itself notes this [arxiv:1806.06719 §3]. In HNSCC NTVC is lowest, at 10.0%.

## Papers

- [Assessing robustness of radiomic features by image perturbation](/papers/1806.06719): uses ICC(1,1) at 0.90 as the robustness criterion, against test-retest and against perturbed copies, on 31 NSCLC and 19 HNSCC CT patients.