Generative Feature Replay For Class-Incremental Learning

In plain words

A model that learns new categories in stages tends to forget the old ones, and it leans toward whatever it saw most recently. The usual fix is to keep some old examples and practise on them again, but storing examples raises privacy and storage concerns. This paper keeps no old examples at all.

The network is split into two parts: one that turns an image into a list of numbers (the feature extractor), and one that turns those numbers into a category label (the classifier). Two things protect the past. First, the extractor is nudged to keep producing nearly the same numbers for new images as the previous version did. Second, a small generator learns to invent plausible number lists for old categories, and the classifier trains on these invented lists next to the real new ones. Inventing short lists of numbers is much easier than inventing whole images, which is why the approach stays practical on larger datasets.

TL;DR

Exemplar-free class-incremental method. Each new step freezes a copy of the previous extractor and generator, distills extractor features on new images, and trains a conditional WGAN on the frozen extractor's features so that old classes can be replayed in feature space. Author-reported: it beats the other exemplar-free methods by a large margin on CIFAR-100 and ImageNet-Subset, and its generator and discriminator take about 4.5 MB, against exemplar sets of 6.2 MB to 3.8 GB tab. 1. Gains are author-reported on these two benchmarks.

Problem

In class-incremental learning the task identity is unknown at test time, so the model must rank all classes learned so far §3.1. Each task introduces disjoint classes, and earlier data is not available during later tasks §3.1. Without old data the classifier becomes biased toward the newest classes §1. Storing exemplars from earlier classes fixes this but is not allowed in many settings, for privacy or storage reasons §1. Image-level generative replay avoids stored images but gives poor results on harder datasets such as CIFAR-100 and ImageNet, because generating realistic full-resolution images is hard §1 §3.2. The paper argues that feature-level generation is much easier than image-level generation §1 §2.2.

Forgetting also depends on depth. Measured with CCA similarity, fine-tuning keeps early features fairly correlated and loses correlation in deeper layers §3.2 fig. 2. Learning without forgetting raises similarity at the classifier but not in the middle or early layers §3.2 fig. 2.

Method

The model is a feature extractor F(x; θ) followed by a classifier H(u; V) = softmax(V u) §3.1. The paper's goal is to keep performance on all earlier tasks while learning each new task, without any stored images §3.1.

1. Single head. The classifier is one linear layer over all classes seen so far, and prediction is the argmax over all of them §4.3 test.py.
2. Feature generator G_t. Two variants are described. Gaussian class prototypes fit one Gaussian per class to the features of that class §4.1. Conditional GAN with replay alignment is a conditional WGAN in feature space, trained with a projection discriminator, and whose generator is also pulled toward the previous generator on old classes §4.1 eq. 1–3.
3. Extractor and head step (task t). Start from F_{t−1} and H_{t−1}, which are kept frozen as copies §4. Train F_t and H_t on the new images with cross-entropy, and add cross-entropy on features of old classes produced by G_{t−1} §4.3. Add an L2 distillation term that ties F_t(x) to F_{t−1}(x) on the new images §4.2 eq. 4.
4. Generator step (task t). Freeze F_t and H_t, then train G_t with the WGAN losses on real features of the new classes eq. 1–2 plus the replay alignment term toward G_{t−1} on old classes eq. 3 §4.1.

The whole procedure is summarised in alg. 1. The paper's pseudocode below follows that description and the text. It leaves two weights unstated; the values shown are assumptions and are marked in the code.

# Class-incremental training, one task at a time  [alg. 1]
# Input: tasks D_1 .. D_T, each (images, labels), disjoint classes
# Output: extractor F, head H, generator G after the last task

def learn_all_tasks(tasks):
    # Task 1: no earlier model, so train F and H on D_1  [alg. 1]
    F, H = train_extractor_and_head(tasks[0])
    # Fit the generator to the features F gives the same data  [alg. 1] [§4.1]
    G = train_generator(F, tasks[0], G_old=None)
    for task in tasks[1:]:
        # Freeze copies of the previous extractor and generator  [§4]
        F_old = frozen_copy(F)
        G_old = frozen_copy(G)
        # Train F and H, starting from the previous head [§4] [§4.3]
        F, H = train_extractor_and_head(task, F_old, G_old, H)
        # Freeze F and H, then train G on new features and alignment
        # [§4.1] [alg. 1]
        G = train_generator(F, task, G_old)
    return F, H, G

# Loss for one batch in the extractor and head step  [§4.2] [§4.3]
def extractor_loss(images, labels, F, H, F_old, G_old):
    # Features of the new real images from the trainable extractor  [§4.2]
    feats = F(images)                                  # (B, d)
    # Cross-entropy on real new samples over the whole head  [§3.1] [§4.3]
    new_loss = cross_entropy(H(feats), labels)
    # Sample old-class labels and generate their features  [§4.1] [§4.3]
    old_labels = sample_old_labels(num_samples=B)      # (B, )
    noise = sample_normal(B, latent_dim=200)           # (B, 200)  [app. D]
    gen_feats = G_old(noise, one_hot(old_labels))      # (B, d)
    # Cross-entropy on generated old-class features  [§4.3]
    # The paper gives no weight for this term; 1 is assumed here.
    old_loss = cross_entropy(H(gen_feats), old_labels)
    # Feature distillation toward the frozen extractor  [§4.2] [eq. 4]
    # Per-sample L2 norm, averaged over the batch; weight 1  [§5]
    with no_grad():
        ref_feats = F_old(images)                      # (B, d)
    distill = mean(l2_norm(feats - ref_feats))
    return new_loss + old_loss + 1.0 * distill

# Generator loss, run after F and H are frozen  [§4.1] [eq. 1–3]
def generator_loss(images, labels, F, G, G_old, critic):
    # Real features of the new images from the frozen extractor  [§4.1]
    real_feats = F(images)                             # (B, d)
    # Generated features for the same labels  [§4.1]
    noise = sample_normal(B, latent_dim=200)           # (B, 200)
    fake_feats = G(noise, one_hot(labels))             # (B, d)
    # Wasserstein critic loss, minimised by the critic  [eq. 1]
    critic_loss = (
        mean(critic(fake_feats, labels))
        - mean(critic(real_feats, labels))
    )
    # Generator adversarial term  [eq. 2]
    adv_loss = -mean(critic(fake_feats, labels))
    # Replay alignment on sampled old classes, against G_old  [eq. 3]
    # Sampled estimate of the sum over old classes and the expectation over z
    old_labels = sample_old_labels(num_samples=B)
    noise_old = sample_normal(B, latent_dim=200)
    align = sum_squared_error(
        G(noise_old, one_hot(old_labels)),
        G_old(noise_old, one_hot(old_labels)),
    )
    return critic_loss, adv_loss + align

Key equations, copied in the paper's form:

\mathcal{L}^{\mathrm{FD}}_{F_t}(\mathcal{X}_t) = \mathbb{E}_{x \sim \mathcal{X}_t}\left[\left\lVert F_t(x) - F_{t-1}(x)\right\rVert_2\right] eq. 4

\mathcal{L}^{\mathrm{WGAN}}_{D_t} = \mathbb{E}_{z,\,c \in C_t}\left[D_t(c, G_t(c,z))\right] - \mathbb{E}_{u \sim \mathcal{D}_t}\left[D_t(c, F_t(x))\right] eq. 1

\mathcal{L}^{\mathrm{WGAN}}_{G_t} = -\mathbb{E}_{z,\,c \in C_t}\left[D_t(c, G_t(c,z))\right] eq. 2

\mathcal{L}^{\mathrm{RA}}_{G_t} = \sum_{j=1}^{t-1}\sum_{c \in C_j} \mathbb{E}_{z}\left[\left\lVert G_t(c,z) - G_{t-1}(c,z)\right\rVert_2^2\right] eq. 3

The equation numbers above follow source order and were not checked against a rendered PDF; see Pitfalls.

Feature generator details. Generator input is the 200-dimensional noise concatenated with a one-hot label, and the discriminator is conditioned by a projection on the label §3.2 app. D resnet.py. Both networks have two hidden layers of width 512 with LeakyReLU (slope 0.2) app. D resnet.py resnet.py.

Hyperparameters

Results

Author-reported. Setup: ResNet-18 trained from scratch, class-incremental single-head evaluation, first task with half the classes, 5/10/25 incremental tasks, CIFAR-100 and ImageNet-Subset (100 classes) §5. The paper also reports ImageNet-1000 in the appendix app. A.

Text claims, because most curve values appear only in plots:

Storage tab. 1 (values as printed in the source):

The source marks one backbone cell with an asterisk but does not explain it tab. 1.

Feature-level ablation, CIFAR-100, 4 tasks tab. 2. Columns T1–T4 are the values as printed; the caption calls them averages over tasks:

† one value shared by the four feature rows in the source.

Regulariser ablation, CIFAR-100, 4 tasks tab. 3:

‡ one value shared by the three rows in the source.

The appendix reports ImageNet-1000 only as plots app. A fig. 6; no table with values is included in the source.

Paper vs code

Official code: xialeiliu/GFR-IL (MIT, Copyright 2020 Xialei Liu), linked from the abstract, at commit 4b33a69 LICENSE. It is trained with CUDA train.py. The release does not resize ImageNet images itself and does not compute forgetting.

1. Distillation norm. The paper averages per-sample L2 norms over the batch §4.2 eq. 4. The code takes one L2 norm of the whole batch difference (torch.dist, p=2), not averaged train.py. If the per-sample differences are similar, the code's value is about the square root of the batch size times the paper's value (batch size 128 by default). This is a derivation from the code, not a number stated in either source.
2. Distillation weight. The paper uses coefficient 1 §5. The code multiplies the distillation term by old_task_factor, which is the number of earlier class groups for the current step: nb_cl_fg // num_class_per_task + current_task - 1 train.py train.py. The factor grows with the task index.
3. Classification weighting. The paper gives no weights for the new-class and replayed-class cross-entropy terms §4.3. The code adds the replayed term with weight old_task_factor and divides by old_task_factor + 1 train.py. This weights the two parts by their class counts.
4. Replay alignment. Eq. 3 has no weight and is a sum of squared L2 distances eq. 3. The code uses MSE(reduction='sum') over a random batch of old labels, scaled by gan_tradeoff (default 1e-3, not changed in the README commands) and by old_task_factor train.py train.py train.py train.py.
5. WGAN form. Eqs. 1–2 have no gradient penalty. The code uses WGAN-GP with weight 10 train.py train.py. The auxiliary classifier losses are computed but left out of both objectives train.py train.py.
6. Critic schedule. The paper says the two losses alternate without a ratio §4.1. The code updates the critic every step and the generator every fifth step train.py.
7. Classifier head size. The paper extends the last linear layer when a task arrives §4.3. The code builds the head with all classes from the start, so the softmax runs over every class from task 1 train.py.
8. Batch-norm mode. For every step after the first, the code sets the extractor to eval mode before training and never switches it back to train mode in the epoch loop, so running statistics stay frozen. The paper says nothing about batch norm train.py train.py.
9. Gaussian prototypes. Samples are drawn as class mean plus one scalar N(0, 1) draw times the per-dimension standard deviation train.py. The paper leaves the covariance structure unstated §4.1. Prototypes are estimated in eval mode from about 200 batches of augmented training images train.py.
10. Mean-replay flag. The release has a -mean_replay flag that switches to the Gaussian prototype path and sets the GAN epochs to 2 train.py train.py. The paper does not name this flag. It may be the "Ours Gaussian" variant §5, but the source does not say so.
11. ImageNet augmentation. The paper resizes to 256 and takes random 224 crops §5. The code takes a RandomResizedCrop(224) from a folder it expects to be pre-resized to 256, and the resize step is commented out train.py train.py. Testing uses a centre crop test.py.
12. Generator output. The generator ends in a ReLU, which the paper does not mention resnet.py. This suits post-ReLU backbone features, but it is an implementation choice.
13. Last task. The GAN is not trained after the final task train.py. The paper does not say this.
14. Evaluation. The code reports the unweighted mean of per-task accuracy and also a weighted mean test.py. Forgetting is not computed anywhere in the release.

No discrepancies found in the parts of the paper not listed above.

Pitfalls

• Distillation scale. Implementing eq. 4 literally, with per-sample norms, gives a different scale from the released code train.py. The coefficient 1 in §5 cannot be used on its own to reproduce the released runs.
• Task-dependent weights. Both the distillation and the classification weights in the code depend on the task index train.py. A paper-only implementation with constant weights will not match the code's dynamics on the 5/10/25 splits.
• Unbound class in eq. 1. The real-feature term of eq. 1 has an unbound c. The code uses each real sample's own label train.py.
• Batch norm. The code freezes the extractor's batch-norm statistics after the first task train.py. An implementer who keeps batch norm in training mode may get different results. The paper is silent here.
• Covariance of the Gaussian variant. The Gaussian prototype paragraph §4.1 writes a covariance Σ per class and never specifies its form. The code's per-dimension standard deviation with one shared scalar draw is an implementation choice the paper does not confirm train.py.
• Memory accounting. The paper's 4.5 MB is not explained tab. 1 app. A. My count from the code's defaults (100 classes, latent 200, width 512) gives about 0.68 M parameters for the generator and 0.53 M for the discriminator without its auxiliary classifier and projection layers. That is about 4.6 MiB (4.8 MB) in float32. Including those layers adds about 0.10 M parameters, so about 5.0 MiB (5.2 MB). This is a derivation from the code, not a figure the paper states.
• Asterisk in Table 1. The source marks one value with an asterisk and never defines it tab. 1.
• Inconsistent ablation numbers. The "Block 4 (Ours)" row in tab. 2 (57.6, 48.2, 41.5 after T2–T4) and the "Feature distillation + GAN" row in tab. 3 (58.4, 48.8, 42.2) describe the same method on the same CIFAR-100 4-task setting. The T1 value also differs (80.7 against 81.9). The source gives no explanation.
• Column meaning in tab. 2. The caption says "average accuracy of all tasks" for values that look like accuracy after each task. The source does not define the columns more precisely.
• Curve values. The per-step numbers for the main comparison exist only in plots fig. 4 fig. 5 fig. 6. Read them from the figures if needed; this card does not transcribe them.
• Algorithm listing. Step 4 of alg. 1 names D_1 for the current extractor's features, where the text describes the current task's data. The listing likely means D_t alg. 1 §4.3.
• Training cost. The paper reports storage but no training time §5 app. A. Each step trains the classifier for 101 or 201 epochs and the GAN for 201 or 501 epochs §5, so per-step training is long.
• Hardware. The code calls .cuda() on its tensors throughout and has no CPU path train.py. It needs PyTorch 1.1 or later, as the README says README.md.
• Checkpoint loading. The code reloads whole pickled modules between tasks train.py, so the same class definitions must be present when a checkpoint is restored.
• Scaling claim. The scaling argument rests on the ImageNet-1000 appendix and the storage comparison app. A §5.2. The paper runs no larger dataset.
• Equation numbering. Equation numbers were taken from source order because no rendered PDF was read. The two WGAN losses share one eqnarray; the numbering treats them as eqs. 1 and 2 and the replay term as eq. 3 §4.1. Check against the PDF before quoting numbers elsewhere.

Lineage

• Builds on: Learning without Forgetting for distillation toward a frozen copy of the previous model §3.2; feature distillation with a single head rather than per-head losses §4.2; Wasserstein GANs (WGAN-GP in the code) for the feature generator §4.1 train.py; a projection discriminator §3.2. None of these has a card in this wiki.
• Compared against: LwF, EWC, MAS, iCaRL, Rebalance and MeRGAN §5 tab. 1. None of these has a card in this wiki.
• Superseded by: not known from wiki content.