CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients

In plain words

A heart monitor records an electrical signal, and a doctor can read it to spot rhythm problems. Training a computer to read these signals normally needs many recordings with expert labels, and labels cost physician time. A common workaround is to first train a model on unlabelled recordings, using a trick that needs no labels, and then reuse that model for the labelled task.

The usual trick treats two slightly different versions of the same recording as "the same thing" and pushes apart versions of other recordings. This paper widens what counts as "the same thing" for heart signals. Two pieces of signal from the same patient belong together, even when they come from different moments or from different leads, which are different views of the same heart signal. The model learns to make such pieces look alike and to tell pieces from different patients apart. A side effect is that the model's output describes a patient's heart in a way that can be used to compare patients.

The practical point is that the trained network can be frozen and used as a feature extractor, or used as a starting point for fine-tuning. The authors report that it does better than the usual alternatives when labels are scarce.

TL;DR

CLOCS is a family of three contrastive pre-training methods for ECG. Each method defines positives as representations of the same patient: adjacent non-overlapping segments (CMSC), different leads (CMLC), or both (CMSMLC) §4.2. The loss is an InfoNCE-style objective with cosine similarity at temperature 0.1, computed over all pairs of instances from the same patient in a mini-batch §4.3 eq. 1 eq. 4. The authors report that CMSC gives the best linear AUC on Chapman and PhysioNet 2020 at E=128 with 50% labels tab. 1. The repository is public, but it is licensed CC BY-NC-SA 4.0 and omits the BYOL, MT-SSL and supervised baselines LICENSE run_experiments.py.

Problem

Deep networks generalise well only with high-quality labels, and procuring them in clinical settings is hard because physicians have little time §1. Most contrastive methods define shared context as "the same instance" through time, augmentation, or views, which the authors argue does not carry over to physiological time series and misses the shared context at the patient level §1 §2. The approach assumes that segments of a recording that are close in time share the cardiac class, and that leads recorded at the same time reflect the same cardiac function. The authors note that both assumptions can fail, for example over long time spans or for conditions that affect only some leads §4.1 §4.2.

Method

Overview. Given a mini-batch of K instances, two transformation operators T_A and T_B produce 2K transformed instances. A positive pair is any two representations that come from the same patient, not only the same instance §4.1. The authors apply this to three transformation families §4.2.

1. CMSC (contrastive multi-segment coding). Split a recording of S seconds into V non-overlapping segments of S/V seconds. Adjacent non-overlapping segments of the same patient form positive pairs §4.2 fig. 1.
2. CMLC (contrastive multi-lead coding). Take temporally aligned projections of the same recording on different leads. Projections of the same patient form positive pairs §4.2 fig. 1.
3. CMSMLC (contrastive multi-segment multi-lead coding). Combine both: the instances are non-overlapping segments on different leads, and the positives are defined across both §4.2.

Loss. Cosine similarity, divided by the temperature τ, measures agreement between representations eq. 1. Within a mini-batch, the similarity matrix is K by K. The diagonal entries are positives by construction. Off-diagonal entries are positives when they come from the same patient, so the set of positives depends on which patients happen to appear more than once in the batch §4.3 fig. 2. The loss is the sum of a diagonal term eq. 2 and an off-diagonal term eq. 3, computed in both directions (A to B and B to A) eq. 4. The expectation in eq. 4 runs over sampled transformation pairs (T_A, T_B) §4.3.

s(h_{A}^{i},h_{B}^{i}) = \frac{f_{\theta}(x_{A}^{i}) \cdot f_{\theta}(x_{B}^{i})}{\| f_{\theta}(x_{A}^{i}) \| \, \| f_{\theta}(x_{B}^{i}) \|} \cdot \frac{1}{\tau} eq. 1

\mathcal{L}_{diag}^{h_{A},h_{B}} = -\mathbb{E}_{i} \left[ \log \frac{e^{s(h_{A}^{i},h_{B}^{i})}}{\sum_{j} e^{s(h_{A}^{i},h_{B}^{j})}} \right] eq. 2

\mathcal{L}_{off\text{-}diag}^{h_{A},h_{B}} = -\mathbb{E}_{i,k \in P} \left[ \log \frac{e^{s(h_{A}^{i},h_{B}^{k})}}{\sum_{j} e^{s(h_{A}^{i},h_{B}^{j})}} \right] eq. 3

\mathcal{L} = \mathbb{E}_{T_{A},T_{B}} \left[ \mathcal{L}_{diag}^{h_{A},h_{B}} + \mathcal{L}_{diag}^{h_{B},h_{A}} + \mathcal{L}_{off\text{-}diag}^{h_{A},h_{B}} + \mathcal{L}_{off\text{-}diag}^{h_{B},h_{A}} \right] eq. 4

The paper uses the symbol P for the patient set and also writes the sums in eqs. 2 and 3 over the batch index; read eq. 3 as "pairs (i, k) of the same patient". The denominators in eqs. 2 and 3 sum over the other view only eq. 2 eq. 3.

Encoder. Three blocks, each a 1D convolution (kernel 7, stride 3), batch normalisation, ReLU, max-pooling by 2 and dropout 0.1, with 4, 16 and 32 output channels, then a linear layer from 320 to E app. C.1. The paper describes the network as three convolutional layers and two fully connected layers §5.5.

# Encoder f_theta: one single-lead frame in, E-dimensional vector out
# [app. C.1]
def encode(frame, embedding_dim):
    # frame: (2500,) one lead or one segment [app. A.1]
    for channels_in, channels_out in [(1, 4), (4, 16), (16, 32)]:
        # Conv1D (kernel 7, stride 3), batch norm, then ReLU [app. C.1]
        frame = relu(batch_norm(conv1d(frame, channels_in, channels_out)))
        # Max-pool by 2, then dropout with probability 0.1 [app. C.1]
        frame = dropout(max_pool(frame, 2), p=0.1)
    # Flatten: 32 channels x 10 steps = 320 values for S = 2500 (computed
    # from the layer sizes) [code: prepare_network.py#L55-L72]
    flat = flatten(frame)
    # Linear layer to E; the paper's table lists a ReLU here, the code has none
    # [app. C.1] [code: prepare_network.py#L94-L98]
    return linear(flat, embedding_dim)

# Patient-specific NCE loss for one ordered pair of views  [eq. 1-3]
# Input:  view_a, view_b: (K, E) representations of the same K instances
#         patient_ids: (K,) patient of each instance in the mini-batch
#         tau: temperature, 0.1 [§5.5]
# Output: diagonal and off-diagonal loss terms

def patient_nce_terms(view_a, view_b, patient_ids, tau):
    # Cosine similarity of every A row with every B row, divided by tau [eq. 1]
    similarity = cosine_matrix(view_a, view_b) / tau      # (K, K)
    # Log-softmax over each row i, denominator summed over all j [eq. 2]
    log_prob = similarity - logsumexp(similarity, axis=1)[:, None]
    # Diagonal: instance i in view A with instance i in view B [eq. 2]
    diagonal_term = -mean(log_prob.diagonal())
    # Off-diagonal positives: i != k with the same patient id [eq. 3]
    same_patient = patient_ids[:, None] == patient_ids[None, :]
    is_positive = same_patient & ~identity(K)
    # Mean over positive pairs only; undefined if the batch has none
    # (not stated in the paper; see Pitfalls) [eq. 3]
    off_diagonal_term = -mean(log_prob[is_positive])
    return diagonal_term, off_diagonal_term

# Full objective for one sampled view pair  [eq. 4]
# Input:  view_a, view_b, patient_ids, tau: as above
# Output: scalar loss

def clocs_loss(view_a, view_b, patient_ids, tau):
    # A-to-B terms, then the same terms with the views swapped [eq. 4]
    diag_ab, off_ab = patient_nce_terms(view_a, view_b, patient_ids, tau)
    diag_ba, off_ba = patient_nce_terms(view_b, view_a, patient_ids, tau)
    # Sum the four terms; eq. 4 takes the expectation over sampled views
    return diag_ab + diag_ba + off_ab + off_ba

Training procedure. Pre-training uses only the training split of two datasets, PhysioNet 2020 and Chapman, because they contain multi-lead data §5.2. The authors keep the number of patients and instances equal across methods §5.2 (but see Pitfalls for the instance counts). They pre-train with either four leads (II, V2, aVL, aVR) or the full set of twelve §5.2. Downstream evaluation uses the frozen encoder (linear evaluation) or the initialised encoder (fine-tuning) §5.3.

Experimental design

Datasets. Splits are by patient, 60/20/20 for training, validation and test, so a patient appears in one split only §5.1. The four ECG datasets are summarised below.

Frames are 2,500 samples each, consecutive frames do not overlap, and the Cardiology frames are resampled from 256 to 2,500 samples app. A.1. Cardiology excludes sudden bradycardia cases app. A.1.

Downstream tasks and metric. Cardiac arrhythmia classification §5.3. The reported metric is test AUC §5.3 tab. 1. The paper does not state how the AUC is averaged over classes; see Pitfalls.

Scenarios. Linear evaluation of frozen representations with multinomial logistic regression §5.3 app. D. Fine-tuning, where all parameters are updated §5.3 app. E. Labelled fractions F of 0.25, 0.5, 0.75 and 1 §6.5. Embedding dimensions E of 32, 64, 128 and 256 §6.5. Five seeds per configuration tab. 1.

Baselines. Random initialisation; supervised pre-training with all labels (F=1) app. C.3; MT-SSL, the multi-task pre-training of Sarkar et al. §5.4 app. C.3; BYOL with EMA decay τ_d = 0.90 §5.5 app. C.3 app. F; SimCLR with time-series perturbations §5.4 app. C.3.

Perturbations (for SimCLR and the sequential-perturbation study). Gaussian noise, temporal reversal (Flip_Y), negation of the signal (Flip_X), and SpecAugment-style masking of the spectrogram (SA_t, SA_f) with w = 0.2 and R = 1 §5.4 app. B.1 app. B.2.

Hyperparameters

Results

Author-reported. Setup: four ECG datasets, pre-training on PhysioNet 2020 and Chapman, 4 leads unless stated, E=128 and F=0.5 for the headline tables §5.2 §6.2 tab. 1 tab. 2. The paper reports test AUC as mean ± standard deviation over 5 seeds tab. 1.

Linear evaluation, E=128, F=0.5, 4 leads tab. 1

* Pre-trained and evaluated with 4 leads (II, V2, aVL, aVR) tab. 1.

Fine-tuning, E=128, F=0.5, 4 leads tab. 2. Columns are pre-training dataset, then downstream dataset. Random initialisation has no pre-training, so its values repeat across the two pre-training columns.

* PhysioNet 2020 used with 4 leads tab. 2.

Linear evaluation across F, E=128, 4 leads app. D.1. Each cell is test AUC as mean ± standard deviation over 5 seeds; bold marks the best of the rows shown.

Linear evaluation across F, E=128, 12 leads app. D.2.

Fine-tuning, E=128, 4 leads, F=0.25, 0.75 and 1 app. E.1. F=0.5 is in the table above. Column order matches that table.

F=0.25

F=0.75

F=1

Fine-tuning, E=128, 12 leads, all F app. E.2. Random initialisation, SimCLR, CMLC and CMSMLC; no BYOL, MT-SSL or supervised rows appear in this grid.

F=0.25

F=0.5

F=0.75

F=1

Headline claims from the text (author-reported, with checks in Pitfalls):

• Linear evaluation: CMSC beats SimCLR on both datasets at E=128 and F=0.5, and the paper reports a 15.8% gain on Chapman §6.2 tab. 1. This "%" is the absolute AUC difference, 0.896 − 0.738 = 0.158 (see Pitfalls).
• Fine-tuning: CMSMLC on Chapman → Cardiology gives 0.717 against 0.676 for SimCLR, reported as a 4.1% improvement §6.3. Again absolute.
• Data efficiency: a network initialised with CMSC on 25% of labels is said to outperform a random initialisation on 100% of labels, for two downstream tasks §6.4 fig. 4.
• Embedding dimension: at F=0.25 on Chapman → Cardiology, random initialisation and SimCLR are flat across E, while CMSC rises from about 0.66 to 0.69 as E goes from 32 to 128. The authors say this shows CMSC still generalises with few labels §6.5 fig. 5.
• Patient-specific representations: intra-patient distances are lower and better separated from inter-patient distances for CMSC than for SimCLR §6.6 fig. 6 app. G.

Paper vs code

Licence. The repository is licensed CC BY-NC-SA 4.0, which excludes commercial use LICENSE. This card quotes no code. Cited code lines are references, and the pseudocode above is written from the paper.

Source. The paper links the repository in a footnote §5.1. Its owner matches the first author, and the code headers name that author perform_training.py. The commit is e58fb4f0369b086735169d781a9513730fa157d0. The commit's README points the datasets to a separate preprocessing repository README.md, which was not read.

Discrepancies between paper and code:

• Embedding layer activation. The paper's architecture table lists a ReLU after the 320→E layer app. C.1. The forward pass applies no activation after the linear layer prepare_network.py.
• Perturbations. The code implements Gaussian noise and the two flips only run_experiments.py prepare_dataset.py. SpecAugment (STFT masking) is absent from the released code (a search for STFT, mask and spectrogram found nothing), so the SA_t and SA_f results cannot be reproduced from this repository.
• Default SimCLR perturbation. The repository's SimCLR configuration uses Gaussian noise, and CMSC, CMLC and CMSMLC use no perturbation run_experiments.py. The paper does not state which perturbation the main-table SimCLR used §5.4.
• Baselines absent. BYOL, MT-SSL and the supervised pre-training trial do not appear in the released scripts run_experiments.py. Only SimCLR, CMSC, CMLC, CMSMLC, Linear, Fine-Tuning and Random are present, plus a CMC trial that the paper does not report run_experiments.py.
• Embedding sweep. The released sweep includes E=320, which the paper does not report run_experiments.py.
• Loss normalisation. The CMSC-family loss is averaged over the terms that are present in each batch: two diagonal terms, plus each off-diagonal term that exists prepare_miscellaneous.py. Eq. 4 is a plain sum eq. 4.
• Denominators in SimCLR. The SimCLR baseline adds within-view negatives to the denominator prepare_miscellaneous.py, whereas eqs. 2 and 3 sum over the other view only eq. 2 eq. 3. The comparison therefore differs in the denominator as well as in the positives.
• Linear evaluation. The encoder's parameters are frozen [code: prepare_models.py#L36-L38], but the whole model is set to training mode during training run_experiment.py. Dropout stays active and batch normalisation statistics keep updating during the linear probe. The paper says the features are frozen §5.3.
• Shared encoder. The code uses one encoder for all views in every CLOCS variant prepare_network.py. The paper does not state whether the encoder is shared.
• No projection head. The contrastive loss is computed directly on the E-dimensional encoder output prepare_network.py prepare_miscellaneous.py. The paper describes no projection head.
• Gaussian noise scale. The code calls np.random.normal(0, s), so the second argument is a standard deviation, in raw signal units, applied before per-view normalisation prepare_dataset.py prepare_dataset.py. The paper writes ε ∼ N(0, σ), which leaves variance versus standard deviation open app. B.1.
• Early stopping. The code caps training at 400 epochs and stops after 15 epochs without improvement in validation loss, keeping the best checkpoint by validation loss run_experiments.py run_experiment.py run_experiment.py. None of this is stated in the paper.

No discrepancy found for: temperature 0.1 prepare_miscellaneous.py, Adam with learning rate 1e-4 and batch sizes run_experiments.py, dropout 0.1 prepare_models.py, the convolution configuration prepare_network.py, five seeds run_experiments.py, and the four leads II, V2, aVL, aVR run_experiments.py.

Pitfalls

• "Percent" gains are absolute AUC differences. The text reports 15.8% for CMSC against SimCLR on Chapman, which is 0.896 − 0.738 = 0.158 §6.2. The relative gain is 21.4%. The same convention gives "4.1%" for CMSMLC against SimCLR, which is 0.717 − 0.676 §6.3, and "13.2%" for 12-lead E=128 at F=0.25, which is 0.903 − 0.771 app. D.2. Report these as AUC points.
• The 100% claim for linear evaluation does not hold in the tables. The text says all three CLOCS methods beat SimCLR in every linear experiment §6.2. CMLC scores 0.580 against 0.585 for SimCLR on PhysioNet 2020 at E=128, F=0.25 app. D.1. At E=32, F=0.25 it scores 0.563 against 0.564 app. D.1.
• "24/24" fine-tuning claim has a tie. For 12-lead E=128 at F=0.25 on Chapman → PhysioNet 2020*, CMSC and SimCLR both score 0.833 app. E.2. The count is 23 strict wins and one tie. The 21/24 figure for 4 leads app. E.1 counts any CLOCS variant; CMSC alone wins 20 of those 24 cells.
• Two Chapman segments. Chapman recordings are 10 s at 250 Hz, which is one 2,500-sample frame app. A.1. CMSC needs two non-overlapping segments of 2,500 samples §4.2 §5.2. The paper does not say how the second segment is found. The code reads two consecutive 2,500-sample halves of a 5,000-sample frame prepare_dataset.py. Those frames come from pickle files built elsewhere prepare_dataset.py, so this could not be checked from the released material.
• Instance counts differ between methods. The paper says all methods see an equal number of patients and instances §5.2. The table lists 51,880 instances for SimCLR and BYOL on PhysioNet 2020, against 24,080 two-segment instances for CMSC and 6,020 for CMSMLC app. A.2.1. The patient counts may be equal; the instance counts are not.
• Off-diagonal positives depend on the batch. A positive pair exists only if the same patient appears twice in the mini-batch, and the paper notes that the frequency varies with shuffling §4.3. The code shuffles the training loader and does not stratify by patient prepare_dataloaders.py. If a batch has no off-diagonal positive, eq. 3 is undefined, and the paper does not say whether the term is zero or dropped.
• Labelled fraction F subsamples frames, not patients. The code draws a random subset of frames with random.seed(0), so the subset is the same across all five seeds prepare_dataset.py. Seeds vary only the model's initialisation.
• Metric averaging is not stated. The code reports the unweighted mean of one-vs-rest ROC-AUC over classes prepare_miscellaneous.py. PhysioNet 2020 is multi-label, with binary cross-entropy over 9 outputs; the other datasets use cross-entropy prepare_miscellaneous.py prepare_miscellaneous.py.
• Normalisation is per view. Each 2,500-sample view is min-max scaled to [0, 1] separately, after noise is added prepare_dataset.py prepare_dataset.py. Cardiology and PhysioNet 2017 are not scaled prepare_dataset.py. The paper says only that the data are normalised for two datasets app. A.1.
• Gaussian noise values and the PTB-XL mention. The appendix gives σ = 10 for Chapman and σ = 0.01 for "PTB-XL" app. B.1. PTB-XL is not among the four datasets. The code applies 0.01 to PhysioNet 2020 prepare_dataset.py. The noise is in raw signal units and is not scaled to the data, so the value depends on the units of the source data.
• Flip naming. "Flip_Y" means temporal reversal and "Flip_X" means sign negation app. B.1. The naming is the reverse of the usual axis convention. The code uses np.flip and negation prepare_dataset.py.
• Temperature and decay share the symbol τ. The NCE temperature is τ = 0.1 eq. 1. The BYOL EMA decay is τ_d = 0.90 §5.5 app. F. The BYOL decay sweep tested 0.50, 0.90 and 0.99 app. F.
• Encoder output size is tied to S. The 320-value flatten holds only for S = 2,500. A different frame length changes the linear layer's input size app. C.1 prepare_network.py.
• Code compatibility. prepare_miscellaneous.py uses np.object [code: prepare_miscellaneous.py#L56]. That alias was removed from recent NumPy releases, so the code needs an older NumPy or a one-line change.
• Costs the abstract hides. The released sweep runs up to 400 epochs per configuration, for 5 seeds, every embedding size and every labelled fraction run_experiments.py run_experiments.py. The 12-lead CMLC and CMSMLC variants use 12 views per instance run_experiments.py. The paper states no hardware or compute figures.

Lineage

• Builds on: SimCLR, which the paper uses as the main contrastive baseline and whose temperature it adopts §5.4 §5.5. The perturbation protocol (single and sequential perturbations) follows SimCLR §6.1. BYOL is a second baseline §5.4. Contrastive multi-view coding and instance discrimination motivate the views §2.
• Compared against: BYOL, SimCLR, MT-SSL (Sarkar et al.), supervised pre-training and random initialisation §5.4 tab. 1 tab. 2.
• Superseded by: not known from this wiki.