Paper · 2022-08-13 · cs.LG
Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification
TS-TCC pretrains a time-series encoder without labels: a Transformer context from one augmented view predicts the other view's future, plus a contextual contrastive loss. CA-TCC adds few labels, pseudo-labels the data and a class-aware contrastive loss. Author-reported: 77.8% accuracy at 1% labels on 10 datasets, versus 61.8% supervised.
Time series classificationSelf-supervised learningSemi-supervised learningContrastive learningtime-seriesself-supervisedsemi-supervisedcontrastive
arXivPDFCodeMarkdown15 min read
AI-compiled by claude-haiku-5-5 on 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Sensor recordings, such as wearable motion data, brain signals or machine vibrations, are plentiful, but someone has to label them, and that is slow and expensive. This paper asks what a model can learn from the unlabelled recordings alone, and then how little labelled data it needs on top.
The core idea is to show the model two slightly different versions of the same recording and train it to recognise that they come from the same source. One version is gently changed, the other is shuffled and noised more heavily. A small prediction task then asks the model to guess the future of one version from the past of the other, which forces it to learn the time structure of the signal rather than surface details. A second loss pulls together the summaries of the two versions of one recording and pushes apart summaries of different recordings.
With a few labels the authors add a second stage. The pretrained model labels the whole training set, using its best guesses, and a loss then groups recordings that share a guessed label. The reported accuracy with very few labels is far above training a model only on those labels.
TL;DR
- Self-supervised pretraining on unlabelled time series: two augmented views, a cross-view temporal prediction task and a contextual contrastive loss (TS-TCC).
- Semi-supervised variant (CA-TCC): fine-tune on 1% or 5% labels, pseudo-label all training data, then train a class-aware supervised contrastive loss.
- Author-reported: 77.8% average accuracy at 1% labels for CA-TCC versus 61.8% for supervised training, across 10 datasets.
- Relevant if you need a time-series encoder with few labels; check the code-versus-paper differences before reproducing numbers.
Problem
Time-series data are collected far faster than they are labelled, and labelling usually needs specialists §I. Image contrastive methods do not transfer directly: time series are characterised by temporal dependencies rather than spatial ones, and colour-style augmentations fit them poorly §I §II.B. Earlier time-series contrastive methods were tied to specific clinical signals §I §II.B. The paper targets a general framework that works without labels and that can use a few labels when they exist §I.
Method
Augmentations: two views of different strength §III.A. For each input sample x the method draws a weak view x^w and a strong view x^s, from two different augmentation families, instead of two draws from one family.
- Weak: scaling, which keeps the signal shape roughly intact. The paper sets the scaling ratio to 2 §IV.B.
- Strong: permutation (split the signal into M chunks and shuffle their order), then jitter (add random noise) §III.A §V.F. Jitter can count as weak or strong depending on its level §III.A.
Both views go through the same encoder: a three-block 1-D convolutional network following [wang2017time] §III.A. Each sample becomes a sequence z_1 ... z_T of feature vectors of length d §III.A.
Temporal contrasting, cross-view §III.B. An autoregressive model reads the features of one view up to a time t and produces a context vector c_t of size h. A linear head for each future step k = 1..K maps c_t to the feature space, and the predicted future is scored against the true feature of the other view at step t+k with a dot product §III.B. Negatives are the other samples in the mini-batch §III.B. The two directions give two losses:
- L_TC^s: strong context predicts weak future eq. 1.
- L_TC^w: weak context predicts strong future eq. 2.
The autoregressive model is a Transformer §III.B. Its input features are linearly projected to size h, a learned token c is prepended, and L identical pre-norm layers follow, each with multi-head attention then a two-layer MLP with ReLU and dropout eq. 3 eq. 4 §III.B. The output of the token is the context c_t §III.B.
Contextual contrasting §III.C. The context vectors of the two views pass through a learned head that maps them into the space where the contrast is computed; the design follows SimCLR §III.C. A batch of N samples gives 2N contexts. The positive of each context is the other view of the same sample. The loss is an InfoNCE-style softmax over cosine similarities with temperature tau, over all other 2N-1 contexts eq. 5, averaged over the 2N anchors eq. 6.
Self-supervised objective §III.C eq. 7. The total is a weighted sum of the two temporal losses and the contextual loss, with fixed scalar weights lambda_1 and lambda_2 eq. 7 §IV.B.
CA-TCC: four training phases §III.D fig. 3.
- Pretrain the encoder from random initialisation with TS-TCC on all unlabelled data §III.D.
- Fine-tune the pretrained encoder on the few labelled samples §III.D.
- Use the fine-tuned encoder to predict pseudo labels for the whole unlabelled training set §III.D.
- Train with the temporal losses plus a supervised contrastive loss, where samples with the same (pseudo) label are positives §III.D eq. 8 eq. 9. The total is lambda_3 times the temporal losses plus lambda_4 times this class-aware loss eq. 10 §IV.B.
# TS-TCC self-supervised loss for one batch [§III] [eq. 1-7]
# Input: x (B, C, T) raw batch
# Output: scalar loss
def ts_tcc_loss(x):
# Weak view: scaling with random factor, mean 2 [§III.A] [§IV.B]
x_weak = scaling(x, mean=2.0)
# Strong view: shuffle M chunks, then add noise [§III.A] [§IV.B]
x_strong = jitter(permute_chunks(x, num_chunks=M))
# Encode each view to a sequence of T' feature vectors [§III.A]
z_weak = encoder(x_weak) # (B, T', d)
z_strong = encoder(x_strong) # (B, T', d)
# Cross-view temporal contrasting, both directions [§III.B] [eq. 1-2]
# First argument supplies the context, second the future
loss_tc_s, c_s = temporal_loss(z_strong, z_weak)
loss_tc_w, c_w = temporal_loss(z_weak, z_strong)
# Contextual contrasting on the two context vectors [§III.C] [eq. 5-6]
loss_cc = contextual_loss(c_s, c_w)
# Weighted sum; SSL weights are lambda_1 = 1, lambda_2 = 0.7 [eq. 7]
lambda_1 = 1.0
lambda_2 = 0.7
return lambda_1 * (loss_tc_s + loss_tc_w) + lambda_2 * loss_cc
def temporal_loss(z_context, z_future):
# Random cut point t that leaves K future steps [§III.B] [§V.E]
t = sample_cut(T_prime, K) # K = 40% of d [§V.E]
# Project to hidden size h, prepend token c, run L=4 pre-norm
# layers, keep the token [§III.B] [eq. 3-4] [§IV.B]
c_t = transformer(project(z_context[:, : t + 1])) # (B, h)
loss = 0
for k in 1..K:
# Predict the future feature at step t+k [§III.B]
predicted = W[k](c_t) # (B, d)
# Dot-product scores against every sample in the batch.
# The diagonal is the positive pair [§III.B] [eq. 1]
scores = predicted @ z_future[:, t + k].T # (B, B)
loss = loss + cross_entropy(scores, diagonal_index)
return loss / K, c_t
def contextual_loss(c_a, c_b):
# Projection head maps both context sets to the contrast space [§III.C]
h_a = projection_head(c_a) # (B, p)
h_b = projection_head(c_b) # (B, p)
# 2B embeddings; each positive is the other view of its sample
# Cosine similarity, temperature tau = 0.2 [§III.C] [§IV.B]
h = normalize(concat(h_a, h_b)) # (2B, p)
sims = h @ h.T / tau
# Softmax over the other 2B-1 embeddings, positive in numerator
# [eq. 5]
per_anchor = cross_entropy_excluding_self(sims, positive_index)
# Mean over the 2B anchors, equal to (1/2N) times the sum [eq. 6]
return mean(per_anchor)# CA-TCC phase 4 loss, supervised contrastive [§III.D] [eq. 8-10]
# Input: h (2B, p) embeddings of both views; y (2B,) pseudo labels
# Output: scalar loss
def class_aware_loss(h, y, tau):
# Positives for anchor i: every other embedding with the same
# pseudo label, self excluded [§III.D] [eq. 9]
same = (y[:, None] == y[None, :]) & not_diagonal
# Per pair term ell(i, p) as in eq. 5 [§III.D] [eq. 8]
pair_terms = ell(h, tau) # (2B, 2B)
# Mean over the positives of each anchor [eq. 8]
per_anchor = masked_mean(pair_terms, same)
# Paper sums over anchors, with no 1/2B factor [eq. 8]
return sum(per_anchor)Hyperparameters
Values stated in the paper are listed first. Values marked "code only" are not in the paper's text; they come from the official repository.
| Name | Value | Where | Notes |
|---|---|---|---|
| Data split | 60% train, 20% validation, 20% test | §IV.B | Subject-wise split for Sleep-EDF |
| Repeats | 5 seeds, mean and standard deviation | §IV.B | Shipped pipeline runs one seed ca_tcc_pipeline.sh |
| Epochs | 40, pretraining and downstream | §IV.B | Same value in all four code configs |
| Batch size | 128, reduced for few-label runs | §IV.B | Code uses 64 for Fault Diagnosis pFD_Configs.py |
| Optimiser | Adam | §IV.B | main.py |
| Learning rate | 3e-4 | §IV.B | |
| Weight decay | 3e-4 | §IV.B | main.py |
| Betas | 0.9, 0.99 | §IV.B | |
| Weak scaling ratio | 2 | §IV.B | Spread (sigma) not stated; code sets it per dataset |
| Strong segments M | Sleep-EDF 20, Epilepsy 12, others 10 | §IV.B | Code values differ, see Paper vs code |
| Strong jitter level | not stated | §IV.B | Code only: 0.8 HAR HAR_Configs.py, 2 Sleep-EDF sleepEDF_Configs.py, 0.001 Epilepsy Epilepsy_Configs.py, 0.1 Fault Diagnosis pFD_Configs.py |
| Normalisation | 0 to 1; weak jitter in [0, 0.1], strong jitter in [0.1, 1] | §V.F | Recommended practice; no normalisation code in this repository |
| Transformer layers L | 4 | §IV.B | |
| Attention heads | 4 | §IV.B | |
| Transformer dropout | 0.1 | §IV.B | |
| Transformer MLP width | not stated | §III.B | Code only: 64 TC.py |
| Hidden size h | tuned over {32, 50, 64, 100, 128, 200, 256}; 64 for Sleep-EDF, 100 otherwise | §IV.B | Code uses 64 for Fault Diagnosis pFD_Configs.py |
| Future steps K | 40% of feature length d | §V.E | Code K/d: HAR 6/18, Sleep-EDF 50/127, Epilepsy 10/24, Fault Diagnosis 50/162 |
| Temperature tau | 0.2 | §IV.B | |
| lambda_1, lambda_2 (TS-TCC) | 1, 0.7 | §IV.B §V.E | Lambda_2 best near 1; 0.7 used |
| lambda_3, lambda_4 (CA-TCC) | 0.01, 0.7 | §IV.B §V.E | Code uses 0.01 and 0.1, see Paper vs code |
| Label fractions | 1%, 5%, 10%, 50%, 75% | §V.A.2 | 1% and 5% in the main tables |
| Linear evaluation | frozen encoder, one fully connected layer | §V.A | main.py |
| Encoder convolution | not stated | §III.A | Code only: widths 32, 64, final_out_channels (128); kernel sizes 8, 25, 8, 32 by dataset model.py |
| Encoder dropout | not stated | §III.A | Code only: 0.35 after the first block model.py |
| Projection head | not stated | §III.C | Code only: Linear, BatchNorm, ReLU, Linear to d/4 TC.py |
| Hardware and software | PyTorch 1.7, one NVIDIA RTX 2080 Ti | §IV.B | |
| Metrics | accuracy and macro F1, reported as percentages | §V |
Results
Author-reported. Setup: 10 real-world datasets §IV.A tab. I: HAR (9 sensor channels), Sleep-EDF (single EEG channel, 5 stages), Epilepsy, and seven UCR datasets §IV.A.5. Fault Diagnosis is used only for transfer §IV.A.4. The baselines are Random initialisation, Supervised, SSL-ECG, CPC and SimCLR §V.A.1 for self-supervised work, and Supervised, Mean-Teacher, DivideMix, SemiTime and FixMatch §V.B.1 for semi-supervised work. Accuracy is shown; MF1 is in the same tables tab. II tab. IV.
Linear evaluation, accuracy, 1% labels tab. II:
| Dataset | Random init | Supervised | SSL-ECG | CPC | SimCLR | TS-TCC |
|---|---|---|---|---|---|---|
| HAR | 39.8 | 44.9 | 60.0 | 65.4 | 65.8 | 70.5 |
| Sleep-EDF | 20.5 | 34.1 | 70.9 | 74.7 | 58.0 | 75.8 |
| Epilepsy | 70.3 | 76.1 | 89.3 | 88.9 | 88.3 | 91.2 |
| Wafer | 90.6 | 91.9 | 93.4 | 93.5 | 93.8 | 93.2 |
| FordA | 50.6 | 56.4 | 67.9 | 75.8 | 55.9 | 80.6 |
| FordB | 52.5 | 51.9 | 64.4 | 66.8 | 50.9 | 72.7 |
| POC | 61.4 | 62.0 | 62.5 | 64.8 | 61.5 | 63.8 |
| PPOC | 68.4 | 64.3 | 49.8 | 63.3 | 37.6 | 63.4 |
| StarLightCurves | 83.9 | 78.8 | 78.3 | 80.8 | 80.6 | 86.0 |
| ElectricDevices | 50.8 | 57.8 | 60.1 | 59.3 | 62.5 | 63.6 |
| Average | 58.8 | 61.8 | 69.7 | 73.3 | 65.5 | 76.1 |
Linear evaluation, accuracy, 5% labels tab. II:
| Dataset | Random init | Supervised | SSL-ECG | CPC | SimCLR | TS-TCC |
|---|---|---|---|---|---|---|
| HAR | 49.6 | 52.8 | 63.7 | 75.4 | 75.8 | 77.6 |
| Sleep-EDF | 22.8 | 60.5 | 73.4 | 76.3 | 64.2 | 77.0 |
| Epilepsy | 75.5 | 83.4 | 92.8 | 92.8 | 91.3 | 93.1 |
| Wafer | 91.2 | 94.6 | 94.9 | 92.5 | 94.8 | 93.2 |
| FordA | 54.4 | 54.5 | 73.6 | 86.5 | 69.6 | 89.9 |
| FordB | 51.3 | 60.5 | 71.7 | 86.3 | 63.0 | 86.1 |
| POC | 61.6 | 61.4 | 62.9 | 66.9 | 62.7 | 62.6 |
| PPOC | 64.1 | 69.1 | 68.8 | 71.5 | 48.0 | 72.1 |
| StarLightCurves | 74.2 | 81.8 | 82.6 | 89.1 | 84.2 | 89.6 |
| ElectricDevices | 57.4 | 59.7 | 63.7 | 62.4 | 63.9 | 65.1 |
| Average | 60.2 | 67.8 | 74.8 | 80.0 | 71.8 | 80.6 |
Semi-supervised, accuracy, 1% labels tab. IV:
| Dataset | Supervised | Mean-Teacher | DivideMix | SemiTime | FixMatch | CA-TCC |
|---|---|---|---|---|---|---|
| HAR | 44.9 | 75.9 | 76.5 | 77.6 | 76.4 | 77.3 |
| Sleep-EDF | 34.1 | 73.6 | 76.5 | 73.6 | 72.5 | 79.4 |
| Epilepsy | 76.1 | 91.5 | 90.9 | 91.6 | 93.2 | 92.0 |
| Wafer | 91.9 | 94.7 | 93.2 | 94.4 | 95.0 | 95.1 |
| FordA | 56.4 | 71.7 | 73.7 | 75.1 | 74.5 | 82.3 |
| FordB | 51.9 | 65.9 | 54.5 | 67.6 | 56.7 | 73.8 |
| POC | 62.0 | 62.1 | 62.1 | 62.0 | 61.9 | 63.4 |
| PPOC | 64.3 | 65.3 | 56.1 | 64.8 | 63.7 | 63.4 |
| StarLightCurves | 78.8 | 79.4 | 79.0 | 79.5 | 77.2 | 85.8 |
| ElectricDevices | 57.8 | 48.9 | 59.8 | 57.3 | 58.2 | 65.9 |
| Average | 61.8 | 72.9 | 72.2 | 74.4 | 72.9 | 77.8 |
Semi-supervised, accuracy, 5% labels tab. IV:
| Dataset | Supervised | Mean-Teacher | DivideMix | SemiTime | FixMatch | CA-TCC |
|---|---|---|---|---|---|---|
| HAR | 52.8 | 88.2 | 89.1 | 87.6 | 87.6 | 88.3 |
| Sleep-EDF | 60.5 | 75.2 | 75.4 | 76.5 | 75.7 | 82.1 |
| Epilepsy | 83.4 | 94.0 | 93.9 | 94.0 | 93.7 | 94.5 |
| Wafer | 94.6 | 94.4 | 94.7 | 95.0 | 94.9 | 95.8 |
| FordA | 54.5 | 82.6 | 84.0 | 83.8 | 83.8 | 90.9 |
| FordB | 60.5 | 64.6 | 60.2 | 65.0 | 62.7 | 88.2 |
| POC | 61.4 | 62.1 | 62.9 | 62.4 | 63.1 | 66.4 |
| PPOC | 69.1 | 73.4 | 69.4 | 71.7 | 72.9 | 73.7 |
| StarLightCurves | 81.8 | 84.9 | 85.6 | 84.6 | 84.1 | 88.8 |
| ElectricDevices | 59.7 | 70.1 | 72.0 | 71.6 | 62.6 | 66.4 |
| Average | 67.8 | 79.0 | 78.7 | 79.2 | 78.1 | 83.5 |
Transfer, Fault Diagnosis, accuracy, 12 source-to-target scenarios tab. III: TS-TCC fine-tuning averages 67.82, CA-TCC 69.66, and supervised training 63.83 tab. III. The paper reports TS-TCC above supervised in 8 of 12 scenarios and CA-TCC in 9 of 12 §V.C.
Ablation, linear evaluation on 5% labels, accuracy and MF1 means, standard deviations omitted tab. VI:
| Component | HAR acc / MF1 | Sleep-EDF acc / MF1 | Epilepsy acc / MF1 |
|---|---|---|---|
| TC only | 68.16 / 66.89 | 75.55 / 60.19 | 88.29 / 88.00 |
| TC + X-Aug | 74.22 / 72.18 | 77.80 / 61.28 | 90.51 / 89.27 |
| TS-TCC (TC + X-Aug + CC) | 77.58 / 76.66 | 76.98 / 70.94 | 93.12 / 93.67 |
| CA-TCC (TC + X-Aug + SCC) | 88.27 / 88.29 | 82.14 / 74.75 | 94.52 / 94.00 |
| TS-TCC, weak views only | 67.39 / 65.54 | 79.63 / 68.15 | 93.22 / 91.97 |
| CA-TCC, weak views only | 85.68 / 84.77 | 81.62 / 70.10 | 93.84 / 92.19 |
| TS-TCC, strong views only | 50.37 / 43.05 | 74.84 / 64.53 | 92.49 / 90.60 |
| CA-TCC, strong views only | 59.59 / 53.34 | 79.24 / 69.39 | 93.74 / 92.00 |
Pretraining combinations, 1% labels in phase 2, HAR and Sleep-EDF accuracy tab. V: TS-TCC in both phases gives 77.3 (HAR) and 79.4 (Sleep-EDF), the best of the combinations listed. The table labels the SSL-ECG row "SSL-EEG".
Augmentation study, HAR, 5% labels, linear evaluation, accuracy tab. VII: the best pair is weak = scaling plus jitter and strong = jitter plus permutation, at 77.58 tab. VII §V.F. Scaling alone with no strong view gives 46.12 tab. VII.
Sensitivity, HAR §V.E fig. 8: predicting about 40% of the feature length works best. Among the loss weights, the best TS-TCC setting is lambda_1 = 1 and lambda_2 = 0.7 §V.E.
Ensemble of self-supervised pseudo labelers §S.II-A fig. S.2: the paper reports a minor gain and a lower standard deviation on HAR and Sleep-EDF.
Paper vs code
Checked against the official repository at code_commit
5298ee8241d4bd0f4eeff3e31b4e7250f31f0357, linked from the paper's abstract.
The repository has no LICENSE file, so no code is quoted here.
- Phase 4 weights. Paper: lambda_3 = 0.01 on the temporal losses and lambda_4 = 0.7 on the supervised loss §IV.B §V.E. Code: 0.01 on the temporal losses, matching the paper, but 0.1 on the supervised loss trainer.py.
- Supervised loss scale. Paper: sum over anchors, no temperature factor eq. 8. Code: the per-anchor mean is multiplied by tau = 0.2 and averaged over the 2N anchors loss.py trainer.py. The code value equals (tau / 2N) times the paper's L_SCC. This combines with the weight difference above.
- Scaling noise. The paper gives only the mean ratio 2 §IV.B. The code draws one factor per sample and per time step, shared by all channels, with spread set per dataset augmentations.py.
- Permutation on multichannel data. The code shuffles chunks using only channel 0 and writes the result to every channel augmentations.py. HAR has nine channels HAR_Configs.py, so its strong views lose inter-channel differences. The other datasets have one channel.
- Number of segments. Paper: M = 20 for Sleep-EDF, 12 for Epilepsy, 10 for others §IV.B. Code: 12, 5, 8 and 5 for Sleep-EDF, Epilepsy, HAR and Fault Diagnosis sleepEDF_Configs.py Epilepsy_Configs.py HAR_Configs.py pFD_Configs.py. The shuffle draws 1 to M-1 segments augmentations.py.
- Future-step fraction. Paper: K = 40% of d §V.E. Code: K/d is 33% for HAR (6 of 18) and 31% for Fault Diagnosis (50 of 162). Sleep-EDF (50 of 127) and Epilepsy (10 of 24) are close to 40%. The values of d are the config's features_len, which I checked by hand against the convolution arithmetic HAR_Configs.py pFD_Configs.py.
- Fault Diagnosis settings. Paper: batch 128 and hidden size 100 for "other" datasets §IV.B. Code: batch 64 and hidden size 64 pFD_Configs.py pFD_Configs.py. The paper's rule "all other datasets" covers Fault Diagnosis, so this is a discrepancy, not a gap.
- Augmentations are fixed. The code computes augmented views once, when the dataset object is built, so every epoch sees the same views dataloader.py.
- Feature normalisation. Before the temporal module the code L2-normalises the encoder features over the channel dimension trainer.py. The paper does not state this.
- One cut point per batch. The code draws one random time step per batch, shared by all samples TC.py. The paper says "a certain timestep" §III.B.
- Learning-rate schedule. The code uses ReduceLROnPlateau on validation loss for supervised and fine-tuning modes trainer.py trainer.py. The paper does not mention a schedule.
- Pseudo labels drop data. Training loaders use drop_last = True dataloader.py and the pseudo-labelling pass uses the same loader main.py, so up to one batch of samples gets no pseudo label.
- SupCon mode.
main.pyline 155 usesdata_perc, which is never defined in the repository. As shipped, the SupCon mode raises NameError there main.py. - Data preparation is outside this repository. The code reads prepared files such as train.pt and train_1perc.pt dataloader.py; the README points to the TS-TCC repository for datasets and splits README.md. The 60/20/20 split and the few-label subsets are therefore not reproducible from this repository alone.
- Configs cover four datasets. Configuration files exist for HAR, Sleep-EDF, Epilepsy and Fault Diagnosis only. No UCR configuration is in the repository, so the seven UCR results cannot be rerun from it.
- Single seed in the shipped pipeline. The pipeline loops from seed 0 to 0 ca_tcc_pipeline.sh. The paper reports five seeds §IV.B.
No discrepancies were found in: the temporal loss structure eq. 1-2, the Transformer layout and its pre-norm residual form eq. 3-4, the contextual loss normalisation eq. 6, the temperature 0.2 §IV.B, the Adam settings and weight decay §IV.B, and the epoch count §IV.B.
Pitfalls
- Weights for phase 4 are not the paper's. Using the paper's 0.01 and 0.7 gives a different loss scale from the code's 0.01 and 0.1 with the tau factor. Pick one set and report it.
- Scaling and permutation details decide the result. The paper says little about the scaling spread and the chunk rule. The code's choices (per-step factors, channel-0 permutation) affect HAR in particular. Test on single- and multi-channel data before trusting the augmentation study.
- Augmentation strength depends on normalisation. The jitter ranges in §V.F assume data scaled to [0, 1]. Normalisation code is not in this repository; the signals must be normalised before the jitter values make sense.
- Few-label results need a fixed split. The few-label subsets and the subject-wise split for Sleep-EDF are not in this repository. Results depend on them, and the paper gives five seeds with mean and standard deviation.
- Results are averages over datasets with very different difficulty. The averages are dominated by datasets with large gains, such as FordA and FordB at 5% labels. Read per-dataset rows before relying on an average.
- Cost. The temporal module is a Transformer evaluated once per cut point and per future step, and pretraining covers both views. The paper says nothing about wall-clock time or memory; its ensemble section notes higher training cost §S.II-A.
- Two evaluation protocols. Linear evaluation freezes the encoder §V.A, as the code does main.py. Fine-tuning updates all encoder weights. Keep the two separate when comparing numbers.
Lineage
- Builds on: TS-TCC, the authors' conference version (IJCAI 2021), which the paper cites as its earlier work §I. This paper adds the semi-supervised CA-TCC variant, the augmentation study and more experiments §I.
- Compared against: Random initialisation, Supervised training, SSL-ECG, CPC and SimCLR (self-supervised) §V.A.1; Mean-Teacher, DivideMix, SemiTime and FixMatch (semi-supervised) §V.B.1. SimCLR was run with the paper's time-series augmentations §V.A.1.
- Superseded by: not stated.
Source and license
Adapted from “Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification” by Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, Xiaoli Li, Cuntai Guan (arXiv:2208.06616), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/emadeldeen24/CA-TCC at commit 5298ee8241d4 (not stated (no LICENSE file in the repository)). Any quoted code is shown under that license.