--- kind: concept slug: continual-learning title: Continual learning summary: >- Learning new tasks or classes without forgetting old ones. Variants: gradient projection (A-GEM), tiny replay (ER) or frozen columns (progressive nets) for tasks; exemplars (PODNet, ORE), class means (SDC) or expansion (DER) for classes; pseudo-labels (PLOP); not-true distillation (FedNTD); weight blending (PAINT); expert freezing (SAME). Trade-off: stored old data against per-step cost. tags: [continual-learning, catastrophic-forgetting, episodic-memory, distillation, class-incremental, prototypes, dynamic-architecture, semantic-segmentation, federated-learning, benchmark, robot-manipulation, weight-interpolation, mixture-of-experts, open-world-detection, instruction-tuning] papers: ["1812.00420", "1802.07569", "1606.04671", "2004.13513", "2004.00440", "2011.11390", "2203.01054", "2110.11309", "2306.03310", "2103.16788", "2106.03097", "2203.11473", "2204.02296", "2208.05592", "2602.01990", "2103.02603", "2308.08747", "1902.10486"] related: [knowledge-distillation, dynamic-network-expansion, gradient-projection, transfer-learning, domain-adaptation, few-shot-learning, prototype-learning, semantic-segmentation, self-training, imitation-learning, robot-simulation, federated-learning, model-merging, model-editing, mixture-of-experts, low-rank-adaptation, multimodal-language-model, experience-replay, object-detection, out-of-distribution-detection, instruction-tuning, catastrophic-forgetting] status: draft updated_at: 2026-11-01 --- ## Overview Neural networks trained on a new task usually lose performance on earlier tasks. This is catastrophic forgetting. Continual (lifelong, incremental) learning asks for a model that learns a stream of tasks or classes and keeps the earlier ones. The stream arrives in order, and past data is either absent or only partly kept [arxiv:1812.00420 §2]. A review by Parisi et al. groups the fixes into three families: regularisation, dynamic architectures, and replay with complementary memories [arxiv:1802.07569 §3.2] [arxiv:1802.07569 §3.3] [arxiv:1802.07569 §3.4]. The review runs no experiments of its own. Its comparisons are qualitative and come from cited work [arxiv:1802.07569 §3.5]. The cards on this page differ in the setting they assume: - **Task-labelled stream.** The predictor takes a task descriptor `t` next to the input, and the evaluation reports average accuracy over all tasks at the end [arxiv:1812.00420 §2] [arxiv:1812.00420 §3]. - **Single-pass stream with a tiny memory.** Each example is seen once. The learner keeps about one stored example per class, and each example carries a task id that selects its head [arxiv:1902.10486 §3.1] [arxiv:1902.10486 §5.2]. - **Class-incremental stream.** Each step adds new classes. At inference the model must pick among all classes seen so far, with no task id. Old classes keep either a fixed number of stored examples [arxiv:2004.13513 §3], or one stored class mean each, with no images kept [arxiv:2004.00440 §4.1]. DER keeps an exemplar memory and also freezes the features of each step inside a network that grows by one extractor per step [arxiv:2103.16788 §3.1] [arxiv:2103.16788 §3.2]. - **Segmentation stream.** Each step adds new classes to a pixel labeller. In the new training images, pixels of old and future classes are labelled as background, and no old images are stored [arxiv:2011.11390 §3.1]. - **Federated clients.** Clients hold non-IID shards, and the server averages their weights each round. FedNTD treats forgetting across rounds as the reason for poor accuracy, presented as a diagnostic and not a proven cause [arxiv:2106.03097 §1] [arxiv:2106.03097 §2]. In federated class-incremental learning, clients also receive new classes over time, and each keeps an exemplar memory [arxiv:2203.11473 §3]. - **Patch stream for an open-vocabulary model.** Patch tasks arrive one at a time, and a supported task such as ImageNet must not regress. Each patch is a fine-tune followed by a weight blend with one coefficient α [arxiv:2208.05592 §2]. Each step starts from the last patched model [arxiv:2208.05592 §5]. - **Robot imitation stream.** One task-conditioned policy learns tasks one after another from demonstrations. Earlier demonstrations are not fully available when a later task is learned [arxiv:2306.03310 §2.2]. The task is given as a language embedding [arxiv:2306.03310 §4.4]. - **Reinforcement-learning task stream.** Each task is a reinforcement-learning environment. Progressive networks add one column per task and freeze the earlier ones. At test time the caller chooses the column, which needs a task label [arxiv:1606.04671 §2]. - **Stream of edits.** Corrections are applied to one pretrained model. MEND turns one desired input-output pair into a weight edit in a single step [arxiv:2110.11309 §3.1]. The results recorded here for MEND are for several edits applied at once, not for a sequence [arxiv:2110.11309 §4.3]. - **Multimodal instruction stream with experts.** A language model with LoRA experts learns instruction tasks one after another, with no stored samples. The authors trace forgetting to two drifts: the router sends old inputs to other experts, and new tasks overwrite the experts [arxiv:2602.01990 §1]. Each task is trained for one epoch [arxiv:2602.01990 §5.1]. - **Open-world detection stream.** Objects of classes not yet named are first reported as "unknown". When a person labels them, the model is updated on the new labels together with a stored, balanced set of exemplars [arxiv:2103.02603 §3] [arxiv:2103.02603 §5.2]. - **Instruction stream for a language model.** Five instruction tasks are trained one after another in one fixed order, and earlier task data is not available at later stages. Forgetting is read from general benchmarks [arxiv:2308.08747 §3.1] [arxiv:2308.08747 §3.2]. A survey names lifelong learning, with changing label sets, as an open challenge for aspect-based sentiment analysis [arxiv:2203.01054 §7]. It reports no experiments on it. The iSDF card lists this concept, because its network trains online on a stream of posed depth frames [arxiv:2204.02296 §4]. Its metrics are SDF error, collision cost and gradient error [arxiv:2204.02296 §5.3]. None of these measures forgetting, so iSDF is not a variant on this page. ## Variants ### Episodic memory with gradient projection (A-GEM) Keep a small memory of examples from each finished task. On each step, compute the gradient of the new mini-batch and a reference gradient on a random batch from memory. If the two point in conflicting directions, remove the conflicting part of the update. The memory constraint is averaged over all past tasks, so no quadratic program is needed [arxiv:1812.00420 §4] [arxiv:1812.00420 eq. 11]. Card: [Efficient Lifelong Learning with A-GEM](/papers/1812.00420). Reported on Split CIFAR, the average accuracy after the last task is 62.3% for A-GEM and 61.2% for GEM, the method it simplifies. Plain fine-tuning (VAN) gets 42.9% [arxiv:1812.00420 tab. 4]. Training time on the same dataset is 449 s for A-GEM and 5238 s for GEM [arxiv:1812.00420 tab. 7]. ### Replay of a tiny memory without gradient constraints (ER) Each step trains on the current mini-batch stacked with a same-size batch drawn at random from an episodic memory. The memory is written after the step. No gradient is projected. The stacked batches are simply averaged in the loss [arxiv:1902.10486 §4] [arxiv:1902.10486 alg. 1]. The paper compares four writing rules: reservoir sampling, a per-class ring buffer, k-means and mean-of-features [arxiv:1902.10486 alg. 2–5]. Card: [On Tiny Episodic Memories in Continual Learning](/papers/1902.10486). Author-reported, with one example per class and one pass over each stream. The ring-buffer writer averages 70.2 on Permuted MNIST, 56.2 on Split CIFAR, 49.0 on Split miniImageNet and 65.0 on Split CUB. A-GEM gets 62.1, 54.9, 48.2 and 62.1 on the same sets, and fine-tuning gets 53.5, 40.6, 34.7 and 55.7 [arxiv:1902.10486 tab. 3]–[tab. 6]. On Split CIFAR the ring-buffer run takes 116 s, against 87 s for fine-tuning and 230 s for A-GEM. The paper gives no hardware [arxiv:1902.10486 tab. 2]. The released code writes with reservoir sampling only, so the ring-buffer and k-means numbers cannot be reproduced from it; see the card's [Paper vs code section](/papers/1902.10486#paper-vs-code). ### Exemplar distillation for class-incremental learning (PODNet) Keep a fixed number of exemplars per class, here 20, selected by herding. The loss on the new classes is a cosine classifier with several proxies per class. A distillation term compares the current network with a frozen copy of the previous one. It matches width- and height-pooled activation statistics at each stage (POD-spatial) and the normalized final embedding (POD-flat) [arxiv:2004.13513 §3.1] [arxiv:2004.13513 eq. 5] [arxiv:2004.13513 eq. 6]. Card: [PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning](/papers/2004.13513). On CIFAR100 with 50 steps of one class, the average incremental accuracy is 61.40% with NME inference and 57.98% with the classifier, against 49.30% for UCIR with its CNN classifier [arxiv:2004.13513 tab. 1]. The gap narrows as the steps get larger: with 5 steps of 10 classes, PODNet (CNN) reaches 64.83% against 64.01% for UCIR (CNN) [arxiv:2004.13513 tab. 1]. ### Expandable network with exemplars (DER) Each step adds a new feature extractor, initialized from the previous one. All earlier extractors are frozen, and the classifier reads their concatenated features [arxiv:2103.16788 §3.2] [arxiv:2103.16788 eq. 1] [arxiv:2103.16788 eq. 2]. An auxiliary head reads only the new extractor. It is trained to tell new classes from one merged old class [arxiv:2103.16788 §3.2] [arxiv:2103.16788 eq. 4]. At the end of each step the classifier is re-initialized and retrained on a class-balanced subset of new data and memory [arxiv:2103.16788 §3.4]. The memory is 2000 exemplars in total for CIFAR-100 with all classes in steps, and 20 per class in the B50 protocol [arxiv:2103.16788 §4.1]. Card: [DER: Dynamically Expandable Representation for Class Incremental Learning](/papers/2103.16788). Author-reported. On CIFAR-100 with 50 steps, the average incremental accuracy is 72.05% with pruning and 72.41% without, against 64.32% for WA, the best baseline in that column [arxiv:2103.16788 tab. 1]. With 10 steps in the B50 protocol, DER reaches 72.45% against 64.04% for PODNet [arxiv:2103.16788 tab. 2]. In the ablation at 10 steps, the average is 61.84% with no expansion and no auxiliary loss, 73.26% with expansion alone, and 75.36% with both. The ablation is for the un-pruned model [arxiv:2103.16788 tab. 4]. The channel-mask pruning of §3.3 is in the paper but not in the released code, so the pruned rows cannot be reproduced from it; see the card's [Paper vs code section](/papers/2103.16788#paper-vs-code). ### Frozen columns with lateral connections (progressive networks) Add a new column for each task, and freeze all earlier columns. Each layer of the new column takes its own previous layer and the previous layer of every earlier column, through lateral weights [arxiv:1606.04671 §2] [arxiv:1606.04671 eq. 1]. Because the frozen columns are never updated, earlier tasks keep their behaviour by construction [arxiv:1606.04671 §2]. The paper also replaces each lateral connection with an adapter, which projects the lateral inputs down and scales them by a learned scalar [arxiv:1606.04671 §2] [arxiv:1606.04671 eq. 2]. Card: [Progressive Neural Networks](/papers/1606.04671). Author-reported, with asynchronous advantage actor-critic (A3C). On Pong Soup, the 2-column net reaches a mean transfer score of 209% against 181% for full finetuning. On Atari the two are level, 132% against 133% [arxiv:1606.04671 tab. 1]. On Labyrinth the 2-column mean is 491%, while the median is 115% [arxiv:1606.04671 tab. 1]. Each column adds parameters, and the total grows quadratically with the number of columns [arxiv:1606.04671 app. 2]. The paper has no released code; see the card's [Paper vs code section](/papers/1606.04671#paper-vs-code). ### Drift compensation of stored class means (SDC) Use an embedding network with no output head, and classify by nearest class mean. After each task, store one prototype per class, not images. Once the next task is trained, estimate how each stored prototype has moved. The estimate is a Gaussian-weighted average of how the current-task images moved in the embedding, and the prototype is shifted by it [arxiv:2004.00440 §4.1] [arxiv:2004.00440 eq. 10]. The paper combines this correction with the regularisers E-LwF, E-EWC and E-MAS, each adapted from softmax to embeddings [arxiv:2004.00440 §3.3] [arxiv:2004.00440 §4.2]. Card: [Semantic Drift Compensation for Class-Incremental Learning](/papers/2004.00440). Author-reported. On six-task CUB-200-2011, E-EWC+SDC reaches 56.4% against 54.2% for E-EWC, and on six-task Flowers-102 it reaches 83.9% against 79.6% [arxiv:2004.00440 tab. 1]. On eleven-task CIFAR100, E-LwF+SDC equals E-LwF at 46.1%. On eleven-task ImageNet-Subset it reaches 50.5% against 51.5% [arxiv:2004.00440 tab. 5]. The official code applies the correction only at evaluation time; see the card's Paper vs code section. ### Pseudo-labels and pooled distillation for continual segmentation (PLOP) For semantic segmentation, the model gets no old images and no task id. Distillation matches pooled feature statistics over strips and grid cells of the old and new feature maps, at several scales [arxiv:2011.11390 §3.2] [arxiv:2011.11390 eq. 5]. For background pixels, the frozen old model gives a label only when its prediction entropy is below a per-class threshold, which is the median entropy of that class. Other background pixels get no target [arxiv:2011.11390 §3.3] [arxiv:2011.11390 eq. 7]. Card: [PLOP: Learning without Forgetting for Continual Semantic Segmentation](/papers/2011.11390). Author-reported, under the Overlapped protocol. On Pascal VOC 2012, when 15 classes come first and one class is added per step (15-1), PLOP reaches 54.64 mIoU over all classes against 29.29 for the authors' MiB re-implementation [arxiv:2011.11390 tab. voc_sota_full]. On 15-5 the authors report PLOP on par with their MiB re-implementation, 70.09 against 70.08 [arxiv:2011.11390 §4.2]. On ADE20k with 100 classes first and 10 per step, PLOP reaches 31.59 against 29.24 for MiB [arxiv:2011.11390 tab. ade_sota]. ### Not-true distillation across federated clients (FedNTD) This is not a stream of tasks. Clients train on their own shards, and the server averages the uploads each round. Each client adds a distillation term that matches the frozen round-start global model over the not-true classes only. The true-class logit is left out of both softmaxes, so the term has no gradient on it [arxiv:2106.03097 §3] [arxiv:2106.03097 eq. FedNTD]. The paper borrows the stability-plasticity framing from continual learning [arxiv:2106.03097 §3]. The method sends nothing extra to the server [arxiv:2106.03097 §4.2]. Card: [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](/papers/2106.03097). Author-reported. On CIFAR-10 with two classes per client, FedNTD reaches 52.61% against 40.14% for FedAvg [arxiv:2106.03097 tab. 1]. The paper says FedNTD "consistently outperforms" the baselines, but its own table does not support that: SCAFFOLD is higher on CIFAR-100 under LDA, 33.68 against 32.37 [arxiv:2106.03097 §4.2] [arxiv:2106.03097 tab. 1]. The default τ = 1 is not tuned, and too small a τ lowers accuracy [arxiv:2106.03097 app.]. The paper does not report the extra compute of the frozen model's forward pass [arxiv:2106.03097 §4]. ### Gradient-compensated loss with server-side prototype reconstruction (GLFC) This is a federated variant of class-incremental learning. Each client trains on mini-batches that mix new data with its exemplar memory. The loss has two terms. The first re-weights each sample's binary cross-entropy by its gradient magnitude, divided by the mean for its own group, new or old classes [arxiv:2203.11473 §4.1] [arxiv:2203.11473 eq. 4]. The second is a KL term that matches the current outputs to soft targets on the old classes, taken from a previous model [arxiv:2203.11473 §4.1] [arxiv:2203.11473 eq. 5]. A client flags a new task when the average entropy of the global model on its data rises by at least a factor of 1.2 [arxiv:2203.11473 §4.1] [arxiv:2203.11473 eq. 7]. On a new task, a client does not send its samples. For each new class it picks the sample whose feature is closest to the class mean, and optimises a copy of it with noise added to the latent feature [arxiv:2203.11473 §4.2] [arxiv:2203.11473 eq. 10–11]. It sends the gradients of these prototypes. The server rebuilds inputs that match those gradients [arxiv:2203.11473 §4.2] [arxiv:2203.11473 eq. 8–9]. It scores the global model on them, keeps the best model, and sends the best models of the last two tasks to the selected clients [arxiv:2203.11473 §4.2] [arxiv:2203.11473 §4.3]. Card: [Federated Class-Incremental Learning](/papers/2203.11473). Author-reported, with ResNet-18 and the mean of three seeds [arxiv:2203.11473 §5.1]. On CIFAR-100 with ten tasks, average accuracy is 66.9, against 61.5 for the best baseline, GeoDL with iCaRL and FL [arxiv:2203.11473 tab. 1]. On ImageNet-Subset with ten tasks it is 57.0, against 52.6 for PODNet with FL [arxiv:2203.11473 tab. 2]. The abstract's range of 4.4 to 15.1 points spans all settings, and its two ends come from these tables [arxiv:2203.11473 tab. 1] [arxiv:2203.11473 tab. 2]. The released code differs from the paper in the perturbation step, the distillation loss, the reconstruction settings and the best-model rule, and it ships only the ten-task configuration. Read the card's [Paper vs code section](/papers/2203.11473#paper-vs-code) and its [Pitfalls](/papers/2203.11473#pitfalls) before implementing. ### Weight blending after each patch (PAINT) Fine-tune the visual encoder of a zero-shot open-vocabulary model on one patch task. The head stays frozen, taken from the text encoder, so no class-specific parameter is added. Then blend the zero-shot and fine-tuned weights with one coefficient α in [0, 1]. α is chosen on held-out validation data for the supported task and the patch task [arxiv:2208.05592 §2] [arxiv:2208.05592 §3]. For a stream of patches, each step starts from the last patched model [arxiv:2208.05592 §5]. The authors report no extra cost at fine-tuning or inference time over standard fine-tuning [arxiv:2208.05592 §1]. Card: [Patching open-vocabulary models by interpolating weights](/papers/2208.05592). On ViT-B/32, sequential patching without interpolation lowers combined accuracy by 4.6 points, and with interpolation it raises it by 11 points, both against the unpatched model [arxiv:2208.05592 §5]. Interpolation reduces forgetting but does not remove it [arxiv:2208.05592 §5] [arxiv:2208.05592 §9]. The released code covers single-task patching and does not select α; see the card's [Paper vs code section](/papers/2208.05592#paper-vs-code). ### Router and expert freezing for mixture-of-LoRA experts (SAME) The model has LoRA experts and a router in the feed-forward layers of its language part. The vision encoder and projector stay frozen [arxiv:2602.01990 §3] [arxiv:2602.01990 §5.1]. SAME adds three mechanisms to this setup. The router's input covariance is kept as a running estimate, and only its leading directions are used for the router update. Router gradients in those directions are rescaled, and the rest of the gradient is passed through unscaled [arxiv:2602.01990 §4.1] [arxiv:2602.01990 eq. 12]. Expert gradients are multiplied by an approximate inverse of the previous task's covariance [arxiv:2602.01990 §4.2] [arxiv:2602.01990 eq. 16]. An expert is frozen for the current task when its utilisation, minus its historical importance, falls below a threshold. It is trained again at the next task and at inference [arxiv:2602.01990 §4.3] [arxiv:2602.01990 eq. 19]. Card: [SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning](/papers/2602.01990). Author-reported, on one run with no error bars [arxiv:2602.01990 §5.1]. On CoIN, average accuracy is 66.82 against 50.58 for MoELoRA [arxiv:2602.01990 tab. 2]. The ablation adds the three mechanisms in turn, with the router update first: 61.32, then 65.89, then 66.82 [arxiv:2602.01990 tab. 4]. On UCIT it reaches 67.12 against 52.06 for MoELoRA and 65.52 for ModalPrompt, the best baseline in that table [arxiv:2602.01990 tab. 3]. On the ten-task TriGap benchmark, which the paper introduces, it reaches 46.53 against 44.45 for MoE-LoRA [arxiv:2602.01990 tab. 1]. The released code differs from the equations in several places, such as routing once per batch, the expert product and the scale direction. Read the card's [Paper vs code section](/papers/2602.01990#paper-vs-code) before implementing. ### Pseudo-unknowns and exemplar replay for open-world detection (ORE) Detect objects of unseen classes as "unknown", and learn them once they are labelled. ORE builds on Faster R-CNN and adds three parts. A contrastive loss pulls the RoI features of each class toward a prototype and pushes the others away. The top background box by RPN objectness in each image is labelled as unknown [arxiv:2103.02603 §4.1] [arxiv:2103.02603 §4.2] [arxiv:2103.02603 app. F]. An energy score is compared against two fitted Weibull densities, one for known and one for unknown samples [arxiv:2103.02603 §4.3]. Each incremental step fine-tunes on the new labels and a balanced exemplar set [arxiv:2103.02603 §4.4]. Card: [Towards Open World Object Detection](/papers/2103.02603). Author-reported, on the four-task Pascal VOC and MS-COCO protocol. In task 2, mAP over both known sets is 38.98 for ORE against 14.91 for Faster R-CNN [arxiv:2103.02603 tab. 2]. Without replay, the baseline's mAP on earlier classes falls to 4.08 in that task [arxiv:2103.02603 tab. 2]. Going from 50 to 100 exemplars per class raises mAP over both known sets only from 38.98 to 39.24, and raises Wilderness Impact from 0.0154 to 0.0410 [arxiv:2103.02603 tab. 5]. The released code differs from the paper's equations in several places, including the loss margin, the norm, and the energy temperature. Read the card's [Paper vs code section](/papers/2103.02603#paper-vs-code) before implementing. ## Benchmark for lifelong robot imitation (LIBERO) LIBERO is a simulated benchmark, not a method. It has 130 language-conditioned manipulation tasks in four suites, with 50 teleoperated demonstrations per task [arxiv:2306.03310 §4.2] [arxiv:2306.03310 §4.4]. Card: [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](/papers/2306.03310). The paper trains each task with behaviour cloning and compares sequential finetuning (SeqL), experience replay (ER), EWC and PackNet. It uses three policy networks: BC-RNN, BC-T and BC-ViT [arxiv:2306.03310 §4.3] [arxiv:2306.03310 §4.4]. Each task is trained for 50 epochs with a replay buffer of 1000 windows per earlier task for ER, and PackNet keeps 25% of each task's weights [arxiv:2306.03310 app. A.2] [arxiv:2306.03310 app. D]. With BC-T on LIBERO-Long, SeqL has forward transfer 0.54 and negative backward transfer 0.63. ER has 0.48 and 0.32, and PackNet has 0.22 and 0.08. For negative backward transfer, lower means less forgetting [arxiv:2306.03310 tab. 2]. On LIBERO-Goal, PackNet has the best AUC, 0.75, with backward transfer 0.06 [arxiv:2306.03310 tab. 2]. On all four suites with BC-T, EWC has the lowest AUC of the four algorithms [arxiv:2306.03310 tab. 2]. Author-reported, as the mean over three seeds [arxiv:2306.03310 tab. 2]. The authors also report that sequential finetuning has the best forward transfer on all four suites, and that basic supervised pretraining can hurt downstream learning [arxiv:2306.03310 §5.2] [arxiv:2306.03310 fig. 5]. The effect of task order is only plotted, and the text gives no numbers for it [arxiv:2306.03310 fig. 4]. The [card's Pitfalls section](/papers/2306.03310#pitfalls) covers how checkpoints are chosen, and should be read before comparing numbers. ## Forgetting in instruction-tuned language models This is a measurement study, not a method. The authors train BLOOMZ (1.1b to 7.1b), mT0 (1.2b and 3.7b), LLaMA-7b and Alpaca-7b on five instruction tasks in one order, then measure forgetting on MMLU, reasoning, reading comprehension and a bias benchmark [arxiv:2308.08747 §3.1] [arxiv:2308.08747 §3.2] [arxiv:2308.08747 §4.1]. Card: [Forgetting in LLMs under continual fine-tuning](/papers/2308.08747). The score is the relative drop from the initial model, averaged over the stages and then over the elements of each set [arxiv:2308.08747 eq. 1]. The authors report forgetting in every model they tested [arxiv:2308.08747 §5.1]. Author-reported: BLOOMZ-7.1b scores 18.37 on domain knowledge and 26.75 on reading comprehension, and mT0-3.7b scores 20.15 on domain knowledge [arxiv:2308.08747 tab. 4]. Alpaca forgets less than LLaMA on all three sets it reports [arxiv:2308.08747 tab. 6]. The study uses one task order and reports no variance [arxiv:2308.08747 §7] [arxiv:2308.08747 §4.2]. No official code was found [arxiv:2308.08747 §4.1]. ## When to use which | Situation | Use | Why | |---|---|---| | The stream has task boundaries and the model can take a task id | A-GEM-style projection | Single pass and one averaged constraint. Training takes about 2.6× (MNIST) and 4.3× (CIFAR) the time of plain fine-tuning, because each step runs a second backward pass [arxiv:1812.00420 tab. 7] | | Classes arrive in many small steps and the model must predict without a task id, and you can store exemplars | PODNet-style distillation with exemplars | The paper reports the largest gains in 50-step settings where older methods collapse [arxiv:2004.13513 §4.1] [arxiv:2004.13513 tab. 1] | | Exemplar budget is very small (5 per class) | Use NME inference | PODNet with NME reached 48.37% against 35.59% with the CNN head [arxiv:2004.13513 tab. 4] | | Exemplar budget is 50 or more per class | Use CNN inference | At 50 per class the CNN head scored 63.69% against 62.27% for NME [arxiv:2004.13513 tab. 4] | | Sequence of RL tasks, a task label is known at test time, and the model may grow with each task | Progressive networks | Frozen columns cannot forget. On Pong Soup the 2-column net beats finetuning on mean transfer (209% against 181%), but on Atari it only matches it (132% against 133%) [arxiv:1606.04671 tab. 1]. Parameters grow with each column [arxiv:1606.04671 app. 2] | | Unfamiliar with the field and need a map of the method families before choosing | Read the review first | It groups the methods into regularisation, dynamic architectures and replay, and states the trade-offs of each family in prose [arxiv:1802.07569 §3.2–3.4]. It reports no numbers of its own [arxiv:1802.07569 §3.5] | | Class-incremental, embedding network, no images may be stored, only one class mean per class | SDC over E-EWC, E-MAS or E-LwF | Only prototypes are kept, and their drift is estimated from current-task images. On six-task CUB and Flowers the reported gain is 1.3 to 11.9 points [arxiv:2004.00440 tab. 1]. On eleven-task CIFAR100 and ImageNet-Subset it ranges from −1.0 to +7.4 points, so it is not a sure gain [arxiv:2004.00440 tab. 5] | | Continual segmentation, no old images may be stored | PLOP | It needs only the frozen old model and the current images. The authors report large gains on 15-1 and parity with MiB on 15-5 [arxiv:2011.11390 tab. voc_sota_full] [arxiv:2011.11390 §4.2] | | No past examples may be stored and the task is classification | SDC, with the caveat above | Stores only class means. PLOP is for segmentation and is not written for classification [arxiv:2011.11390 §3.1] | | Class-incremental, exemplar memory is allowed, and the model may grow with each step | DER, un-pruned | Old features are frozen, so later training cannot overwrite them. The un-pruned model averages 285.6M parameters at 50 steps, against 11.2M for one ResNet-18 [arxiv:2103.16788 tab. 1] | | Class-incremental with a fixed parameter budget | DER with pruning, only if you write the pruning yourself | The paper reports 10.15M parameters at 50 steps with pruning [arxiv:2103.16788 tab. 1]. The masks, sharpness schedule and sparsity loss are described but not released [arxiv:2103.16788 §3.3], and the card's Paper vs code section lists the gaps | | Federated clients with non-IID shards, and no extra upload is allowed | FedNTD with β = τ = 1 | It adds one loss term and no upload. The paper reports gains over FedAvg in most settings, not all [arxiv:2106.03097 tab. 1] [arxiv:2106.03097 §4.2]. Compute cost is not reported [arxiv:2106.03097 §4] | | Federated clients receive new classes over time, each may keep a small exemplar memory, and sharing gradients of prototype samples is acceptable | GLFC, after resolving the code deviations in its card | Author-reported, 66.9 against 61.5 on CIFAR-100 with ten tasks [arxiv:2203.11473 tab. 1], and 57.0 against 52.6 on ImageNet-Subset [arxiv:2203.11473 tab. 2]. The privacy argument depends on a noise step that the released code does not implement [Pitfalls](/papers/2203.11473#pitfalls). The card reports that its proxy path cannot take 200 classes or 224 px images as written | | Single pass, task ids available, and about one stored example per class | ER with the ring buffer; reservoir once the memory is larger | At one example per class the ring buffer reaches 56.2% on Split CIFAR, against 54.9% for A-GEM and 40.6% for fine-tuning [arxiv:1902.10486 tab. 4]. The paper finds reservoir best for larger memories [arxiv:1902.10486 §5.4] | | Compute per step is tight, and task ids are available | ER over A-GEM | On Split CIFAR, ER with the ring buffer takes 116 s against 230 s for A-GEM, with no hardware stated [arxiv:1902.10486 tab. 2]. The ratio is taken within one table, so it is the reliable comparison | | Robot policy learns tasks in order, and forward transfer matters more than keeping old tasks | Sequential finetuning | The paper reports the best forward transfer on all four suites [arxiv:2306.03310 §5.2]. It has the highest backward transfer on LIBERO-Long (0.63) and LIBERO-Spatial (0.81) [arxiv:2306.03310 tab. 2] | | Robot policy must keep old tasks, and a small buffer of old demonstrations can be stored | Experience replay | The authors report ER as robust across suites [arxiv:2306.03310 §5.2]. With BC-T on LIBERO-Long it has the highest AUC of ER, PackNet and EWC [arxiv:2306.03310 tab. 2] | | Robot policy must keep old tasks, and extra fine-tuning per task is affordable | PackNet | Lowest backward transfer on LIBERO-Long and LIBERO-Goal, and the best AUC on LIBERO-Object and LIBERO-Goal [arxiv:2306.03310 tab. 2]. It adds 50 fine-tuning epochs per task [arxiv:2306.03310 app. A.2] | | Robot policy with EWC as the only regulariser | Prefer ER or PackNet | EWC has the lowest AUC on all four suites with BC-T [arxiv:2306.03310 tab. 2] | | One pretrained model needs several corrections at once | MEND for simultaneous edits, with a limit | Edit success on zsRE with BART-base falls from 0.98 at one edit to 0.67 at 125 simultaneous edits. ENN falls from 0.99 to 0.11 [arxiv:2110.11309 §4.3]. The paper's text and table disagree at 25 edits, and the table is used here [arxiv:2110.11309 §4.3] | | Sequential patches of a zero-shot model, one task at a time, with a supported task that must not regress, and labelled validation data for both | Weight blending after each patch (PAINT), with α chosen on validation | For ViT-B/32 the paper reports −4.6 points for sequential patching without interpolation and +11 points with it [arxiv:2208.05592 §5]. Class names can change, because the head comes from the text encoder [arxiv:2208.05592 §3] | | All patch datasets are available at once | Joint patching, then one blend | With ViT-L/14, joint patching of nine tasks is 0.5 points below nine specialised models in combined accuracy [arxiv:2208.05592 §5] | | Multimodal language model with LoRA experts, a task sequence, and no stored old samples | SAME, after resolving the code deviations in its card | Against MoELoRA the author-reported gains are 16.2 points on CoIN and 15.1 on UCIT [arxiv:2602.01990 tab. 2] [arxiv:2602.01990 tab. 3]. The released code does not match the covariance update [arxiv:2602.01990 eq. 4] or the scale direction [arxiv:2602.01990 eq. 8] | | Old instruction samples may be stored | Measure Replay-LoRA first | Replay-LoRA is within 6 points of SAME on UCIT (61.18 against 67.12) and within 8 on TriGap (39.08 against 46.53) [arxiv:2602.01990 tab. 3] [arxiv:2602.01990 tab. 1]. The gap is small enough that your own stream should decide | | Detector must flag unseen objects and learn them later, and labelled exemplars can be stored | ORE-style pseudo-unknowns, energy test and exemplar replay, read against its Paper vs code section | Author-reported mAP over both known sets is 38.98 against 14.91 for Faster R-CNN in task 2 [arxiv:2103.02603 tab. 2]. Replay is what keeps earlier classes [arxiv:2103.02603 tab. 2] | | Language model is tuned on a sequence of instruction tasks, and no general data is kept | Expect forgetting, and measure it on held-out general benchmarks | Reported in every tested model. The score is an average over stages, so it is not an endpoint drop [arxiv:2308.08747 eq. 1] [arxiv:2308.08747 §5.1] | | Same, and a small set of general instruction data can be stored | Mix it into the stream, after checking on your own tasks | On one MMLU-human split, LLaMA-7b falls from 34.72 to 26.8 with the tasks alone and to 30 with 10,000 Alpaca samples mixed in. The mixing ratio is not stated [arxiv:2308.08747 §5.4] [arxiv:2308.08747 fig. 6] | ## Implementation sketch The A-GEM and PODNet variants share a loop over the stream and a memory of past examples. They differ in how the memory enters the update. SDC and PLOP add a step that the sketch does not show. The LIBERO baselines follow the same pattern: ER replays stored windows, PackNet releases pruned weights for later tasks, and EWC adds a Fisher-weighted penalty [arxiv:2306.03310 app. A.2]. ```pseudocode # Shared loop over the task stream [arxiv:1812.00420 alg. 2] # and class-incremental training [arxiv:2004.13513 §3] def train_stream(model, memory, stream, teacher=None): for task in stream: for x, y in task.batches(): # Loss on the new batch [arxiv:1812.00420 alg. 2 l. 6] g = grad(loss(model, x, y)) if variant == "gradient_projection": # Sample past examples from all earlier tasks [§4] x_ref, y_ref = memory.sample() g_ref = grad(loss(model, x_ref, y_ref)) # Remove the conflicting part of the update [eq. 11] if dot(g, g_ref) < 0: g = g - dot(g, g_ref) / dot(g_ref, g_ref) * g_ref else: # Keep pooled features near the frozen teacher [arxiv:2004.13513 eq. 7] g += grad(distill(teacher, model, x)) apply_update(model, g) # Store exemplars (A-GEM, PODNet) or class means (SDC) for the # finished task [arxiv:1812.00420 §6] [arxiv:2004.13513 §4] # [arxiv:2004.00440 §4.1]. PLOP stores nothing. memory.add(task) # SDC: shift each stored class mean by the estimated drift [eq. 10] # PLOP: label confident background pixels with the old model [§3.3] ``` Where the variants differ: A-GEM needs the task boundary to fill memory and to form its reference batch, and it has no teacher. ER has no teacher either. It adds a same-size memory batch to each step instead of projecting the gradient [arxiv:1902.10486 alg. 1]. PODNet needs the teacher and a fixed number of exemplars per class, and it does not use task ids. SDC keeps one class mean per class; its E-LwF, E-EWC and E-MAS base methods keep their usual regularisers [arxiv:2004.00440 §4.2]. PLOP needs the frozen old model for both its distillation and its pseudo-labels, and it stores no examples. PAINT also stores no examples; its step is a weight blend with coefficient α, described for one patch in [arxiv:2208.05592 §2]. DER differs in the network itself: it adds one extractor per step and keeps the older ones frozen [arxiv:2103.16788 §3.2]. Progressive networks also store no examples. They add one column per task, freeze the earlier columns, and read them through lateral connections [arxiv:1606.04671 §2]. FedNTD's teacher is the global model at the start of the round, and its loss covers only the not-true classes [arxiv:2106.03097 §3]. GLFC distils from the previous task's model, or from the current model when the client has just detected a new task [arxiv:2203.11473 §4.3]. Its loss re-weights samples instead of projecting gradients, and clients share prototype gradients with the server [arxiv:2203.11473 §4.2]. ORE keeps a balanced exemplar set, and each incremental step fine-tunes on the new labels plus that set [arxiv:2103.02603 §4.4]. SAME stores no samples either. It keeps a running covariance of the router input, preconditions expert gradients with it, and freezes experts per task [arxiv:2602.01990 §4.1] [arxiv:2602.01990 §4.2] [arxiv:2602.01990 §4.3]. ## Contradictions A-GEM's Split CIFAR average is 62.3% in its own paper [arxiv:1812.00420 tab. 4] and 54.9% in the ER paper's table [arxiv:1902.10486 tab. 4]. The memory budgets differ (65 per task in the first, one example per class in the second), so the gap may come from the protocol. The reported numbers are not comparable. The settings differ (task-labelled single pass for A-GEM, class-incremental multi-epoch training for PODNet and SDC, segmentation for PLOP, robot behaviour cloning for LIBERO, simultaneous edits for MEND, open-world detection for ORE, and instruction tuning for the language-model study), and so do the metrics. A-GEM and SDC report accuracy after the last task, PODNet reports the average over every incremental step [arxiv:1812.00420 §3] [arxiv:2004.13513 §4] [arxiv:2004.00440 tab. 1], PAINT reports combined accuracy, the mean of supported and patch accuracy, against the unpatched model [arxiv:2208.05592 §3] [arxiv:2208.05592 §5], and LIBERO reports forward transfer, negative backward transfer and AUC on success rates [arxiv:2306.03310 §5.1]. Within the SDC card, the gain is not uniform. It is clear on six-task CUB and Flowers, but on eleven-task CIFAR100 E-LwF+SDC equals E-LwF, and on ImageNet-Subset it is one point lower [arxiv:2004.00440 tab. 1] [arxiv:2004.00440 tab. 5]. Within the LIBERO card, the authors report that task order changes results, significantly for PackNet, but the paper gives no numbers for this [arxiv:2306.03310 fig. 4]. Within the progressive networks card, the Labyrinth mean and median point in different directions: 491% against 115% [arxiv:1606.04671 tab. 1]. The comparison with finetuning depends on which of the two is used. Within the SAME card, the text and the TriGap table disagree. The text calls the FloodNetVQA gain "particularly clear", but MoE-LoRA scores higher there, 90.41 against 81.09, and SAME trails MoE-LoRA on CLEVR, 54.50 against 67.00 [arxiv:2602.01990 §5.2] [arxiv:2602.01990 tab. 1]. The CoIN baselines are taken from the original publications, while the TriGap baselines are reproduced under the same setup [arxiv:2602.01990 tab. 2] [arxiv:2602.01990 tab. 1]. Compare across these tables with care. Within the DER card, the headline numbers use the pruned model, while the ablation [arxiv:2103.16788 tab. 4] uses the un-pruned one. Compare like with like [arxiv:2103.16788 tab. 1]. Within the ORE card, the released code and the paper use different loss margins, norms and energy temperatures, and the Weibull fits use pseudo-labelled boxes as unknowns [arxiv:2103.02603 §4.3] [arxiv:2103.02603 app. D]. The card does not check whether these differences change the reported numbers. Within the GLFC card, the ablation's stated average for the variant without the compensation loss is 60.1, but its ten values average 64.7. The card uses the stated value, so do not rely on that row's average [arxiv:2203.11473 tab. 1]. The GLFC and FedNTD numbers come from different federated settings, one class-incremental with an exemplar memory and one a fixed-class classification task, so they are not comparable [arxiv:2203.11473 §5.1] [arxiv:2106.03097 tab. 1]. Within the language-model study, the forgetting score is an average over stages, so it cannot be recomputed from the start and end scores in the table. The bias column does not match its endpoints either [arxiv:2308.08747 eq. 1] [arxiv:2308.08747 tab. 4] [arxiv:2308.08747 tab. 5]. Compare its values only with each other. The review reports that benchmarks and metrics lack consensus across the field [arxiv:1802.07569 §3.5] [arxiv:1802.07569 §5]. Its qualitative rankings are benchmark-specific: EWC and PathNet change order between the data-permutation and multimodal experiments [arxiv:1802.07569 §3.5]. Do not use them as numbers for the methods above. ## Papers - [Efficient Lifelong Learning with A-GEM](/papers/1812.00420): single-pass gradient projection against an episodic memory, with an averaged constraint and a closed-form update. - [On Tiny Episodic Memories in Continual Learning](/papers/1902.10486): experience replay with a tiny memory, a stacked batch per step and four memory-writing rules; no gradient projection. - [Progressive Neural Networks](/papers/1606.04671): one column per RL task, frozen earlier columns and lateral connections with adapters; compared with finetuning on Pong, Atari and 3D mazes. - [PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning](/papers/2004.13513): class-incremental distillation with pooled feature statistics and a multi-proxy cosine classifier. - [Semantic Drift Compensation for Class-Incremental Learning](/papers/2004.00440): class means stored per task and shifted by a Gaussian-weighted estimate of the embedding drift, with no images kept. - [PLOP: Learning without Forgetting for Continual Semantic Segmentation](/papers/2011.11390): pooled multi-scale feature distillation and entropy-thresholded pseudo-labels for background pixels, with no stored images. - [DER: Dynamically Expandable Representation for Class Incremental Learning](/papers/2103.16788): a new extractor per step, frozen old extractors, an auxiliary new-vs-old loss, and a balanced classifier retrain; the pruned variant is not in the released code. - [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](/papers/2106.03097): distillation over the not-true classes against the round-start global model, with no extra communication. - [Federated Class-Incremental Learning](/papers/2203.11473): GLFC, a class-aware gradient-compensated loss and relation distillation per client, with prototype gradients rebuilt into samples at a proxy server. - [Continual Lifelong Learning with Neural Networks: A Review](/papers/1802.07569): a review of catastrophic forgetting that groups regularisation, dynamic-architecture and replay methods into families; no new experiments. - [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](/papers/2203.01054): names lifelong ABSA with changing label sets as an open challenge; no method or experiments on continual learning. - [MEND: Fast Model Editing at Scale](/papers/2110.11309): a small editor network turns one fine-tuning gradient into a low-rank weight edit; evaluated on simultaneous edits, not on a continual stream. - [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](/papers/2306.03310): a simulated benchmark of 130 robot tasks in four suites, with ER, EWC and PackNet compared against sequential finetuning. - [Patching open-vocabulary models by interpolating weights](/papers/2208.05592): one fine-tune per patch task, then a linear blend with the zero-shot weights; sequential and joint multi-task patching are compared. - [SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning](/papers/2602.01990): rehearsal-free LoRA experts with router updates in leading covariance directions, covariance-scaled expert gradients and per-task freezing of low-score experts. - [iSDF: Real-Time Neural Signed Distance Fields for Robot Perception](/papers/2204.02296): online training of a neural SDF on a stream of posed depth frames; no forgetting measure, so not a variant here. - [Towards Open World Object Detection](/papers/2103.02603): ORE, a Faster R-CNN variant with prototype clustering, pseudo-labelled unknowns, an energy-based unknown test and exemplar replay. - [Forgetting in LLMs under continual fine-tuning](/papers/2308.08747): a measurement of forgetting when five instruction tasks are tuned in sequence; no method is proposed.