---
kind: concept
slug: gradient-conflict-resolution
title: Gradient conflict resolution
summary: >-
  Combines the gradients of several losses into one update that keeps the average loss as the target while protecting the worst task. One card so far: CAGrad, which starts from the average gradient and pushes toward the worst task within a radius c. Author-reported gains over PCGrad and MGDA. Its convergence theorem and its experiments use different update rules.
tags: [optimization, multi-task-learning, gradient-methods, reinforcement-learning]
papers: ["2110.14048"]
related: [multi-task-learning, semi-supervised-learning]
status: draft
updated_at: 2026-10-19
---

## Overview

When one network learns from several losses, their gradients can point in
different directions or differ widely in size. Following their average can then
help one task while hurting another, often the hardest one
[arxiv:2110.14048 §1] [arxiv:2110.14048 §2.1].

The methods on this page change how the task gradients are merged into one
update. The shared idea is to keep the average gradient as the starting point
and move only a bounded distance from it, toward a direction that improves the
worst task [arxiv:2110.14048 §3.1] [arxiv:2110.14048 eq. 3].

Only one paper card covers this slug so far, so the page describes one method,
CAGrad. The other methods the paper compares against (MGDA, PCGrad, GradDrop)
are named in its tables but have no cards here, so they are not described.

## Variants

### CAGrad

The update is the average gradient plus a term of fixed length c‖g_0‖ along the
normalised blend of task gradients [arxiv:2110.14048 §3.1]. The blend weights
are found by a small problem over the tasks, not over the parameters
[arxiv:2110.14048 §3.1] [app. A.3]. The one hyperparameter c lies in [0, 1)
for the convergence guarantee. At c = 0 the rule is plain gradient descent on
the average loss [arxiv:2110.14048 §3.1] [arxiv:2110.14048 alg. 1]. Card:
[Conflict-Averse Gradient Descent for Multi-task Learning](/papers/2110.14048).

Author-reported results, where lower Δm is better [arxiv:2110.14048 tab. 1]:

| Setting | CAGrad | Baselines the paper lists |
|---|---|---|
| NYU-v2, 3 tasks, c = 0.4 [arxiv:2110.14048 tab. 1] | Δm 0.20 | MTAN 5.59, PCGrad 3.97, MGDA 1.38 |
| CityScapes, 2 tasks, c = 0.4 [arxiv:2110.14048 tab. 2] | Δm 11.64 | MTAN 22.60, PCGrad 18.29, MGDA 44.14 |

Results on the RL and semi-supervised benchmarks are in
[arxiv:2110.14048 tab. 3] and [arxiv:2110.14048 tab. 4]. On Meta-World MT10 the
paper reports success 0.83 for CAGrad against 0.72 for PCGrad, and on MT50 0.52
against 0.50 [arxiv:2110.14048 tab. 3]. CAGrad uses no task metadata. CARE,
which does use task language metadata, reaches 0.84 on MT10
[arxiv:2110.14048 tab. 3].

#### CAGrad-Fast (sampled tasks)

Each step, a random subset S of tasks is sampled. Their gradients are computed,
and the unsampled tasks are replaced by their average. The same problem is then
solved [arxiv:2110.14048 §3.3] [arxiv:2110.14048 eq. 4]. The paper uses |S| = 4
on the 10-task RL benchmark and |S| = 8 on the 50-task one [arxiv:2110.14048
app. B.3]. Per-step times are 4.8 s for CAGrad-Fast and 10.3 s for full CAGrad
on the 10-task setup, and 11.4 s against 27.8 s on the 50-task setup
[arxiv:2110.14048 tab. 7].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Several losses, average loss is the target, need a stated convergence result | CAGrad with c < 1 | The theorem covers the plain step with α ≤ 1/H [arxiv:2110.14048 §3.2]. Experiments use Adam or SAC, so the theorem does not carry over directly (see Pitfalls). |
| Few tasks (K < 10), standard compute budget | Full CAGrad | The paper says the task-weight solve takes much less time than the backward passes when K < 10 [arxiv:2110.14048 §3.3]. |
| Many tasks and a tight per-step budget | CAGrad-Fast | Its cost is O((|S|N + T)) rather than one backward pass per task [arxiv:2110.14048 §3.3]. On MT50 this is 11.4 s per step against 27.8 s for full CAGrad [arxiv:2110.14048 tab. 7]. |
| The worst task should dominate and no average-loss target is needed | Large c | Large c approaches the MGDA rule, which can settle at any Pareto point rather than at the average loss [arxiv:2110.14048 §3.1] [arxiv:2110.14048 app. A.1]. |
| You do not want to tune c | Use a reported value only for its own setting | The paper picks c per setting: 0.4 for NYU-v2 and CityScapes, 0.9 and 0.5 for MT10 and MT50, and 0.1 for semi-supervised learning, fixed without tuning [arxiv:2110.14048 app. B.2] [arxiv:2110.14048 app. B.3] [arxiv:2110.14048 app. B.4]. |
| Semi-supervised learning with auxiliary tasks | CAGrad on top of the auxiliary-task method | With ARML, CAGrad gives 68.25 at 500 labels against 67.49 for PCGrad, and 82.81 at 2000 labels against 81.91 [arxiv:2110.14048 tab. 4]. |

The margins are small in places. CityScapes segmentation is 75.16 mIoU for
CAGrad against 75.18 for MTAN [arxiv:2110.14048 tab. 2], and the significance
test uses p = 0.1 with three seeds [arxiv:2110.14048 tab. 1]. Treat the
differences as the authors' result on their own setups, not a general ranking.

## Implementation sketch

Restated from the paper's algorithm, with the citations kept
[arxiv:2110.14048 alg. 1]:

```pseudocode
# Average of the task gradients, the main objective  [§2.1]
g0 = mean(grads)
# Search radius, c times the norm of g0  [§3.1] [eq. 3]
radius = c * norm(g0)
# Task weights on the simplex; the paper leaves the solver open  [§3.1]
w = solve_weights(grads, g0, radius)
# Blended task gradient  [§3.1]
g_w = sum(w_i * g_i for w_i, g_i in zip(w, grads))
# Update direction: average plus a bounded push toward g_w  [§3.1] [alg. 1]
d = g0 + radius * g_w / norm(g_w)
# Plain step, as in the paper's algorithm  [alg. 1]
theta = theta - alpha * d
```

For CAGrad-Fast, replace the full gradient list with the sampled subset and
replace the unsampled part by its average, as in [arxiv:2110.14048 eq. 4].
Guard the division by the norm of g_w with a small constant; the paper does not
address g_w = 0 [arxiv:2110.14048 §3.1] (see Pitfalls).

## Contradictions

- **Convergence for c ≥ 1.** The main text says fixed points are
  Pareto-stationary for c ≥ 1 [arxiv:2110.14048 §3.2]. The appendix says any
  c ≥ 0 [arxiv:2110.14048 app. A.3]. Use the appendix version only with its
  proof in hand.
- **CityScapes c.** The text says c = 0.2 [arxiv:2110.14048 §5.2]. The appendix
  and the reported table row say c = 0.4 [arxiv:2110.14048 app. B.2]
  [arxiv:2110.14048 tab. 2]. The authors' code runs c = 0.4 and c = 0.6
  [code: cityscapes/run.sh#L8-L9]. The repository does not show c = 0.2.
- **Selection of c.** The supervised text picks c by average training
  performance [arxiv:2110.14048 app. B.2]. The appendix sweep on CityScapes
  gives Δm 10.74 at c = 0.8, lower than the 11.64 reported at c = 0.4
  [arxiv:2110.14048 tab. 6] [arxiv:2110.14048 tab. 2].

## Pitfalls

- **The theorem and the experiments differ.** Theorem 1 covers the plain step
  θ − αd with α ≤ 1/H [arxiv:2110.14048 §3.2] [arxiv:2110.14048 alg. 1]. The
  supervised, toy and RL runs feed the direction into Adam or SAC optimisers
  [arxiv:2110.14048 §5.2] [arxiv:2110.14048 §5.3].
- **Undisclosed scaling in the code.** The paper's update has no extra scale
  [arxiv:2110.14048 alg. 1]. The authors' code divides it by 1 + c² in the
  supervised setups [code: nyuv2/utils.py#L399]. At c = 0.4 this shrinks the
  step by about 14%. Results reproduced from the paper alone may differ.
- **Degenerate blend.** If g_w = 0 the update is undefined. The paper does not
  say what to do; the code adds small constants [arxiv:2110.14048 §3.1].
- **Compute per step.** Each step needs one backward pass per task. The RL
  task-weight solve adds up to 20 inner steps [arxiv:2110.14048 app. B.3].
- **Best-epoch reporting in RL.** The RL setup reports the highest average test
  score seen during training [arxiv:2110.14048 app. B.3]. This is optimistic,
  and no held-out selection is reported.
- **Unstated weights.** The semi-supervised auxiliary weights are not given in
  the paper [arxiv:2110.14048 §5.4].

## Papers

- [Conflict-Averse Gradient Descent for Multi-task Learning](/papers/2110.14048):
  introduces CAGrad and its sampled variant CAGrad-Fast.