Gradient projection

Overview

A gradient step on new data can undo what the model already knows. Gradient projection keeps the step from undoing it. The method computes a reference gradient that stands for what must be kept, and if the new update points against it, it removes the conflicting component. With the full projection, the update no longer has a negative dot product with the reference, so the loss on what the reference stands for is not pushed up to first order 1812.00420 §4 1812.00420 eq. 10 2205.14865 §3.2 2205.14865 eq. 4.

The shared invariant is the constraint g̃ᵀ g_ref ≥ 0. The variants differ in what the reference gradient stands for (examples stored from earlier tasks, or a frozen zero-shot model), in how many reference gradients they use, and in how the projection is computed. The paper that introduces the closed-form version compares it with the per-task constraints it simplifies 1812.00420 §4.

Variants

Averaged episodic-memory constraint (A-GEM)

Keep a small memory of examples from each finished task. At each step, draw a random batch from that memory and compute its gradient g_ref. Compare it with the gradient g of the current mini-batch. If gᵀ g_ref is negative, replace g by its projection onto the plane orthogonal to g_ref. Otherwise use g unchanged 1812.00420 eq. 11 1812.00420 alg. 2. The paper states the constraint on the average memory loss over all past tasks, not one constraint per task 1812.00420 eq. 9.

Card: Efficient Lifelong Learning with A-GEM.

The method is a simplification of Gradient Episodic Memory (GEM, arXiv:1706.08840, no card here). GEM keeps one constraint per past task and projects with a quadratic program over those constraints 1812.00420 eq. 7 1812.00420 eq. 8. A-GEM replaces these with one constraint and a closed-form projection 1812.00420 §4.

Teacher-KL projection (ProGrad)

A prompt-tuning variant for a frozen CLIP model. Learned context vectors v are placed before the class names and trained with cross-entropy on a few labelled images, as in CoOp 2205.14865 §3.1. The reference here is not stored data. It is the frozen zero-shot model: L_kl is a KL-type loss between the student's predictions and the zero-shot predictions on the same training images, and g_kl is its gradient with respect to v 2205.14865 §3.2 2205.14865 eq. 3. If g_ce · g_kl is negative, the method subtracts λ times the component of g_ce along g_kl 2205.14865 eq. 4. The paper uses λ = 1 by default and λ = 0 gives back CoOp 2205.14865 §3.2.

The projection differs from A-GEM's in one way. With λ = 1 the result is orthogonal to g_kl. With λ < 1 the dot product becomes (1 − λ) times g_ceᵀ g_kl, which is still negative, so only part of the conflict is removed 2205.14865 eq. 4.

Card: Prompt-aligned Gradient for Prompt Tuning.

The paper gives no timing or memory figures for the method. Each step adds a zero-shot forward pass on the batch and needs the gradients of two losses 2205.14865 §3.2.

When to use which

The two source papers evaluate different settings: A-GEM on continual-learning benchmarks, ProGrad on few-shot image classification. The conclusions below are limited to those settings.

Memory size matters for the projection, since the reference gradient comes from it. The paper gives per-task budgets of 50 to 250 examples and reference batch sizes of 128 to 1300 1812.00420 §6. Those are the values it used, not recommended settings for other problems.

The λ evidence for ProGrad is thin. In the λ ablation on DTD, λ = 0.9 is best at 16 shots (64.28), against 62.89 for λ = 0 and 61.15 for λ = 1.0. On OxfordPets at 16 shots, λ = 1.0 is best (89.00) [arxiv:2205.14865 §4, λ table]. The ablation covers only these two datasets.

Implementation sketch

Common core of the projection step. A-GEM takes the reference from an episodic memory batch and uses λ = 1. ProGrad takes it from the frozen zero-shot model on the same images, and uses λ = 1 by default and λ = 0 for CoOp. The projection is applied only when the dot product is negative.

# Reference gradient: memory batch (A-GEM) or frozen teacher (ProGrad)
# [alg. 2] [arxiv:2205.14865 eq. 3]
g_ref = grad(loss_ref(model, x_ref, y_ref), params)
# Gradient of the training loss on the current mini-batch
# [alg. 2] [arxiv:2205.14865 eq. 2]
g = grad(loss(model, x, y), params)
# Project only where the update conflicts with the reference
# [arxiv:1812.00420 eq. 11] [arxiv:2205.14865 eq. 4]
if dot(g, g_ref) < 0:
    g = g - lam * (dot(g, g_ref) / dot(g_ref, g_ref)) * g_ref
# Plain SGD step with the projected gradient [alg. 2]
params = params - lr * g

The paper's own code is the reference for the details the algorithm leaves open, such as how memory is filled and which output head is used for memory examples model.py. See the card's "Paper vs code" section before implementing.

Contradictions

Across cards: none known. The two cards report on different tasks, so their numbers are not compared.

Within the A-GEM card, the text and the table disagree on speed. The text says A-GEM is "about 100 times faster" than GEM 1812.00420 §6.1, while the training-time table gives roughly 7× on MNIST and 12× on CIFAR 1812.00420 tab. 7. The table's numbers are the ones cited above.

Within the ProGrad card, the text sets λ = 0.8 at 16 shots [arxiv:2205.14865 §4], but the λ ablation has no 0.8 row 2205.14865 §4, λ table.

Papers

• Efficient Lifelong Learning with A-GEM: replaces GEM's per-task constraints with one averaged constraint and a closed-form projection, and compares it with GEM and regularization baselines on four continual-learning benchmarks.
• Prompt-aligned Gradient for Prompt Tuning: projects the CoOp-style cross-entropy gradient for CLIP prompts against the gradient of a KL term toward zero-shot predictions, and reports gains over CoOp across few-shot shot counts.