---
kind: concept
slug: gradient-projection
title: Gradient projection
summary: >-
  Remove the part of a parameter update that conflicts with a reference gradient, so the update does not increase loss on what must be kept. Covered here: A-GEM, which uses one averaged episodic-memory constraint for weaker per-task guarantees at much lower cost than GEM, and ProGrad, which projects prompt-tuning steps against a KL term toward zero-shot CLIP.
tags: [optimization, continual-learning, episodic-memory, catastrophic-forgetting, prompt-tuning]
papers: ["1812.00420", "2205.14865"]
related: [continual-learning, soft-prompt-tuning, few-shot-learning]
status: draft
updated_at: 2026-10-19
---

## Overview

A gradient step on new data can undo what the model already knows. Gradient
projection keeps the step from undoing it. The method computes a reference
gradient that stands for what must be kept, and if the new update points
against it, it removes the conflicting component. With the full projection,
the update no longer has a negative dot product with the reference, so the
loss on what the reference stands for is not pushed up to first order
[arxiv:1812.00420 §4] [arxiv:1812.00420 eq. 10] [arxiv:2205.14865 §3.2]
[arxiv:2205.14865 eq. 4].

The shared invariant is the constraint `g̃ᵀ g_ref ≥ 0`. The variants differ
in what the reference gradient stands for (examples stored from earlier tasks,
or a frozen zero-shot model), in how many reference gradients they use, and in
how the projection is computed. The paper that introduces the closed-form
version compares it with the per-task constraints it simplifies
[arxiv:1812.00420 §4].

## Variants

### Averaged episodic-memory constraint (A-GEM)

Keep a small memory of examples from each finished task. At each step, draw a
random batch from that memory and compute its gradient `g_ref`. Compare it with
the gradient `g` of the current mini-batch. If `gᵀ g_ref` is negative, replace
`g` by its projection onto the plane orthogonal to `g_ref`. Otherwise use `g`
unchanged [arxiv:1812.00420 eq. 11] [arxiv:1812.00420 alg. 2]. The paper
states the constraint on the average memory loss over all past tasks, not one
constraint per task [arxiv:1812.00420 eq. 9].

Card: [Efficient Lifelong Learning with A-GEM](/papers/1812.00420).

The method is a simplification of Gradient Episodic Memory ([GEM, arXiv:1706.08840](https://arxiv.org/abs/1706.08840),
no card here). GEM keeps one constraint per past task and projects with a
quadratic program over those constraints [arxiv:1812.00420 eq. 7]
[arxiv:1812.00420 eq. 8]. A-GEM replaces these with one constraint and a
closed-form projection [arxiv:1812.00420 §4].

### Teacher-KL projection (ProGrad)

A prompt-tuning variant for a frozen CLIP model. Learned context vectors `v`
are placed before the class names and trained with cross-entropy on a few
labelled images, as in CoOp [arxiv:2205.14865 §3.1]. The reference here is not
stored data. It is the frozen zero-shot model: `L_kl` is a KL-type loss between
the student's predictions and the zero-shot predictions on the same training
images, and `g_kl` is its gradient with respect to `v` [arxiv:2205.14865 §3.2]
[arxiv:2205.14865 eq. 3]. If `g_ce · g_kl` is negative, the method subtracts
`λ` times the component of `g_ce` along `g_kl` [arxiv:2205.14865 eq. 4]. The
paper uses `λ = 1` by default and `λ = 0` gives back CoOp [arxiv:2205.14865 §3.2].

The projection differs from A-GEM's in one way. With `λ = 1` the result is
orthogonal to `g_kl`. With `λ < 1` the dot product becomes `(1 − λ)` times
`g_ceᵀ g_kl`, which is still negative, so only part of the conflict is removed
[arxiv:2205.14865 eq. 4].

Card: [Prompt-aligned Gradient for Prompt Tuning](/papers/2205.14865).

The paper gives no timing or memory figures for the method. Each step adds a
zero-shot forward pass on the batch and needs the gradients of two losses
[arxiv:2205.14865 §3.2].

## When to use which

The two source papers evaluate different settings: A-GEM on continual-learning
benchmarks, ProGrad on few-shot image classification. The conclusions below are
limited to those settings.

| Situation | Use | Why |
|---|---|---|
| Many tasks, or a memory that grows with them, and a per-step budget | A-GEM | Per-step cost is one extra memory gradient and a dot product. Training time on Split CUB is 420 s [arxiv:1812.00420 tab. 7]. |
| Few tasks and a need for a guarantee on every past task | Per-task constraints (GEM), if the QP cost is acceptable | GEM's worst-case forgetting on the MNIST test sets is 0.10, against 0.13 for A-GEM [arxiv:1812.00420 tab. 2]. GEM takes 3442 s against 477 s for A-GEM on MNIST [arxiv:1812.00420 tab. 7]. |
| No episodic memory allowed, continual stream | Not covered by this page | A-GEM needs stored examples. Regularization methods such as EWC need no memory and are compared in [arxiv:1812.00420 tab. 4]. |
| Single pass over the data, task labels available | A-GEM | On Split CIFAR, A-GEM reaches 62.3% average accuracy against 42.4% for EWC and 42.9% for plain fine-tuning [arxiv:1812.00420 tab. 4]. |
| Prompt tuning of a frozen CLIP model with 1 to 4 labelled images per class, and a zero-shot prompt that is a reasonable teacher | ProGrad | Averaged over 11 datasets: 62.61% vs 59.46% for CoOp at 1 shot, and 68.45% vs 66.79% at 4 shots [arxiv:2205.14865 tab. 1]. Author-reported. |
| Prompt tuning with 16 labelled images per class | ProGrad, expect a small gap | 74.28% vs 73.49% for CoOp at 16 shots [arxiv:2205.14865 tab. 1]. |
| Zero-shot teacher is weak on the target domain | Test before using ProGrad | The projection can only protect what the zero-shot model already gets right. Many ProGrad failures are also zero-shot errors [arxiv:2205.14865 §4] [fig. 5]. |

Memory size matters for the projection, since the reference gradient comes from
it. The paper gives per-task budgets of 50 to 250 examples and reference batch
sizes of 128 to 1300 [arxiv:1812.00420 §6]. Those are the values it used, not
recommended settings for other problems.

The λ evidence for ProGrad is thin. In the λ ablation on DTD, `λ = 0.9` is best
at 16 shots (64.28), against 62.89 for `λ = 0` and 61.15 for `λ = 1.0`. On
OxfordPets at 16 shots, `λ = 1.0` is best (89.00) [arxiv:2205.14865 §4, λ
table]. The ablation covers only these two datasets.

## Implementation sketch

Common core of the projection step. A-GEM takes the reference from an episodic
memory batch and uses `λ = 1`. ProGrad takes it from the frozen zero-shot model
on the same images, and uses `λ = 1` by default and `λ = 0` for CoOp. The
projection is applied only when the dot product is negative.

```pseudocode
# Reference gradient: memory batch (A-GEM) or frozen teacher (ProGrad)
# [alg. 2] [arxiv:2205.14865 eq. 3]
g_ref = grad(loss_ref(model, x_ref, y_ref), params)
# Gradient of the training loss on the current mini-batch
# [alg. 2] [arxiv:2205.14865 eq. 2]
g = grad(loss(model, x, y), params)
# Project only where the update conflicts with the reference
# [arxiv:1812.00420 eq. 11] [arxiv:2205.14865 eq. 4]
if dot(g, g_ref) < 0:
    g = g - lam * (dot(g, g_ref) / dot(g_ref, g_ref)) * g_ref
# Plain SGD step with the projected gradient [alg. 2]
params = params - lr * g
```

The paper's own code is the reference for the details the algorithm leaves
open, such as how memory is filled and which output head is used for memory
examples [code: model/model.py#L1164-L1172]. See the card's "Paper vs code"
section before implementing.

## Contradictions

Across cards: none known. The two cards report on different tasks, so their
numbers are not compared.

Within the A-GEM card, the text and the table disagree on speed. The text says
A-GEM is "about 100 times faster" than GEM [arxiv:1812.00420 §6.1], while the
training-time table gives roughly 7× on MNIST and 12× on CIFAR
[arxiv:1812.00420 tab. 7]. The table's numbers are the ones cited above.

Within the ProGrad card, the text sets `λ = 0.8` at 16 shots [arxiv:2205.14865
§4], but the λ ablation has no 0.8 row [arxiv:2205.14865 §4, λ table].

## Papers

- [Efficient Lifelong Learning with A-GEM](/papers/1812.00420): replaces GEM's per-task constraints with one averaged constraint and a closed-form projection, and compares it with GEM and regularization baselines on four continual-learning benchmarks.
- [Prompt-aligned Gradient for Prompt Tuning](/papers/2205.14865): projects the CoOp-style cross-entropy gradient for CLIP prompts against the gradient of a KL term toward zero-shot predictions, and reports gains over CoOp across few-shot shot counts.