---
kind: concept
slug: class-balanced-loss
title: Class-balanced loss
summary: >-
  Loss weights set per class from its training count, so rare classes count for more: an effective-number weight, or an inverse-frequency power. Covers classifier-only retraining, training from scratch and per-pixel uses, and where each helped or hurt. Three wiki cards use it; numbers are author-reported, and some card tables and code disagree.
tags: [loss-function, class-imbalance, long-tail]
papers: ["2203.14197", "2103.16370", "1908.08297"]
related: [long-tailed-recognition, focal-loss, salient-object-detection, semantic-segmentation]
status: draft
updated_at: 2026-11-01
---

## Overview

When some classes have many training examples and others have few, plain
cross-entropy gives the frequent classes more influence over training. The
class-balanced loss counters this by multiplying each example's loss by a
weight that depends only on its class. The weight is set from how often that
class, or pixel label, occurs in training. The variants differ in the formula.
The first card uses the effective number of samples, which saturates as the
count grows [arxiv:2203.14197 §3.3]. The second uses inverse frequency raised
to a power rho [arxiv:2103.16370 eq. 11]. The effective-number formula below
is specific to the first variant.

The formula the first card gives for the weight is

$$ w_c = \frac{1-\beta}{1-\beta^{n_c}}, $$

where $n_c$ is the number of training samples in class $c$ and $\beta$ is a
hyperparameter [arxiv:2203.14197 §3.3]. The weights are then rescaled so they
sum to the number of classes [arxiv:2203.14197 §3.3] [code:
utils/class_balanced_loss.py#L73-L75]. The paper's text gives neither a value
for $\beta$ nor the loss form [arxiv:2203.14197 §3.3]. The released code uses
$\beta = 0.9999$ [code: demo2_second-stage-training.py#L340].

Three wiki sources use the technique. In "Long-Tailed Recognition via Weight
Balancing", the class-balanced loss is one component of a two-stage recipe,
and it is used only to retrain the final classifier on frozen features
[arxiv:2203.14197 §3.3]. In "Distribution Alignment", the weight is inverse
frequency raised to a power, and it trains a per-class calibration on frozen
features [arxiv:2103.16370 §3.2.2]. A third card, EGNet, does not use either
weight in its paper. Its paper's edge loss is unweighted, but its released code
weights edge and background pixels by class frequency [arxiv:1908.08297 eq. 4]
[code: solver.py#L223-L227].

## Variants

### Classifier retraining on frozen features

Stage 1 trains the backbone with plain cross-entropy and tuned weight decay.
Stage 2 freezes the features and retrains only the classifier, with the
class-balanced loss, weight decay and a MaxNorm projection [arxiv:2203.14197
§3.3]. The paper's ablation on CIFAR100-LT with imbalance factor 100 reports
53.35% class-averaged accuracy for the full stage-2 recipe, against 46.08% for
weight decay alone [arxiv:2203.14197 tab. 1]. Card: [Long-Tailed Recognition
via Weight Balancing](/papers/2203.14197).

Within that setting the loss matters most when combined with stage-2 weight
decay. On tuned features, the class-balanced loss alone gives 47.09%, against
46.08% for cross-entropy. With stage-2 weight decay, it gives 52.42%, against
47.22% for cross-entropy with the same decay [arxiv:2203.14197 tab. 1]. The
paper does not separate these effects further.

### Training from scratch with the loss

The same loss can also replace cross-entropy during end-to-end training. On
CIFAR-100-LT with imbalance factor 100, the paper reports 39.60% for
cross-entropy plus class-balanced loss, against 38.32% for cross-entropy
[arxiv:2203.14197 tab. 2]. On ImageNet-LT it reports 33.2%, against 44.4% for
cross-entropy, and on iNaturalist2018 54.0%, against 61.7% [arxiv:2203.14197
tab. 3]. The card does not explain the gap between these datasets. Card: [Long-Tailed Recognition via Weight Balancing](/papers/2203.14197).

### Generalised re-weighting with a learnable calibration

Distribution Alignment weights each class by $(1/r_c)^{\rho}$, normalised over
classes, where $r_c$ is the class's training-set frequency and $\rho$ is a
scale hyper-parameter [arxiv:2103.16370 eq. 11]. The paper calls $\rho = 0$
instance-balanced and $\rho = 1$ class-balanced [arxiv:2103.16370 §3.2.2]. The
weight of each example's true class enters a cross-entropy over calibrated
scores [arxiv:2103.16370 eq. 9] [eq. 10]. Stage 2 freezes the backbone and the
original classifier. It learns a per-class scale and offset, and an
input-dependent gate that blends them with the original scores [arxiv:2103.16370
§3.2.1] [eq. 5].

The paper's values for $\rho$ are 1.2 for a dot-product classifier and 1.5 for
a cosine one on classification, 0.3 for segmentation and 0.8 for LVIS
[arxiv:2103.16370 app. A.2] [app. B.2] [app. C.2]. On ImageNet-LT with
ResNet-50 and a cosine head, the full method reports 52.9% top-1, against 48.4%
for the same cosine baseline [arxiv:2103.16370 tab. 1]. That number is for the
whole calibration, not for the weight alone. The paper's component ablation
reports 41.6% with none of the three components, and 50.1%, 49.9% and 51.3%
when the re-weighting is combined with a learnable magnitude, a learnable
margin, or both [arxiv:2103.16370 tab. 5]. The table has no row with the
re-weighting alone, so its own effect is not isolated [arxiv:2103.16370 §4.1].
Card: [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](/papers/2103.16370).

### Class-weighted edge loss in a salient-object detector

EGNet trains its edge branch with a binary cross-entropy whose equation is
unweighted [arxiv:1908.08297 eq. 4]. Its released code instead weights edge
pixels by the fraction of negative pixels, and background pixels by 1.1 times
the fraction of positive pixels, in the EGNet repository [code:
solver.py#L160] [code: solver.py#L223-L227]. This is a per-pixel frequency
weight, not the effective-number weight. The paper does not say whether its
reported results used the weighted or the plain loss [arxiv:1908.08297 §4.3].
Card: [EGNet: Edge Guidance Network for Salient Object Detection](/papers/1908.08297).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You have a trained backbone and can retrain only the classifier | Stage-2 class-balanced loss, with weight decay and MaxNorm | Gains on frozen features are largest with stage-2 weight decay [arxiv:2203.14197 tab. 1] |
| You want a per-class calibration on a frozen two-stage classifier, for classification, segmentation or detection | Distribution Alignment's re-weighted, learnable calibration | The paper reports results for all three tasks, with a learnable scale, offset and gate on top of the weight [arxiv:2103.16370 §4.1] [§4.2] [§4.3] |
| You want to test the weight on its own, without a second stage | Run the weighted and unweighted arms on your data | The Distribution Alignment ablation never isolates the weight [arxiv:2103.16370 tab. 5] |
| You train a per-pixel binary task with rare positives, such as edges | Test a plain and a frequency-weighted loss on your data | The EGNet equation is unweighted, its release weights by frequency, and the paper does not say which its results used [arxiv:1908.08297 eq. 4] [arxiv:1908.08297 §4.3] |
| You train from scratch on a large imbalanced set | Test first on your data | End-to-end class-balanced loss rose on CIFAR-100-LT but fell on ImageNet-LT and iNaturalist [arxiv:2203.14197 tab. 2] [tab. 3] |
| Your data has a clearly balanced test split | Class-averaged accuracy as the metric | Both papers evaluate on balanced test or validation sets [arxiv:2203.14197 §4.1] [arxiv:2103.16370 §3.1] [§4.1] |
| You need stage-2 hyperparameters | Start from the released values, then tune | The 2203.14197 paper tunes them by Bayesian optimisation, but the values are not stated in the text [arxiv:2203.14197 §3.3]. The 2103.16370 paper gives its stage-2 learning rate for classification as "restarted" with no value [arxiv:2103.16370 app. A.2] |

## Implementation sketch

The core is the per-class weight and the weighted loss. Stage-2 projection and
weight decay come from the same recipe and are not shown here. Values are the
released defaults for CIFAR100-LT, not values stated in the text.

```pseudocode
# Per-class weight from effective number of samples  [§3.3]
# Input: class_counts n_c (K,), beta
# Output: weights (K,), summing to K  [code: cb#L73-L75]

def class_balanced_weights(class_counts, beta):
    # Effective number of samples per class, saturating in n_c  [§3.3]
    effective = 1 - beta ** class_counts            # (K,)
    weights = (1 - beta) / effective                # (K,)
    # Rescale so the weights sum to the number of classes
    return weights / weights.sum() * len(class_counts)


# Weighted loss on softmax probabilities  [code: cb#L92-L94]
# Each entry carries the weight of its sample's true class
def class_balanced_loss(logits, labels, class_weights):
    probs = softmax(logits)                         # (B, K)
    targets = one_hot(labels, num_classes)          # (B, K)
    sample_weights = class_weights[labels][:, None] # (B, 1)
    bce = -(targets * log(probs) + (1 - targets) * log(1 - probs))
    # Mean over all B*K entries, as the released code does
    return mean(bce * sample_weights)
```

The power-law variant changes only the weight. It uses $(1/n_c)^{\rho}$,
normalised to sum to one, and a batch-mean loss over the weighted
log-likelihood [arxiv:2103.16370 eq. 9] [eq. 11]. The two normalisations
differ, as the Contradictions section explains.

## Contradictions

- **Labels in the paper's tables.** The paper's "+" notation is inconsistent.
  The row "WD + CB" in the main comparison table reports 52.42%, the same
  number as "WD tuned + CB & WD" in the ablation [arxiv:2203.14197 tab. 2] [tab.
  1]. Read each row's exact definition in the card before comparing.
- **Reproduced headline number.** The paper reports 53.35% for the full
  recipe. The release logs show 51.98% best-epoch accuracy at stage 2
  [code: exp/demo_2/MaxNorm_WD_note_bestModel.log]. The training loop picks the
  best epoch on the test loader [code: utils/trainval.py#L152], although the
  paper says it tunes on a validation set [arxiv:2203.14197 §4.1]. Budget for
  lower accuracy on a held-out split.
- **Loss form.** The first card's method section gives the weight formula with
  a section citation, while its comparison section says the paper does not give
  the loss form. The softmax form above comes from the code only.
- **Same name, different weight.** Both papers use "class-balanced" for their
  $\rho = 1$ or effective-number weights, but the formulas differ. The first
  paper uses $(1-\beta)/(1-\beta^{n_c})$ [arxiv:2203.14197 §3.3]. The second
  uses $(1/r_c)^{\rho}$ and calls $\rho = 1$ class-balanced [arxiv:2103.16370
  §3.2.2]. Results from one do not transfer to the other without a check.
- **Weight normalisation.** The first paper rescales its weights to sum to the
  number of classes [arxiv:2203.14197 §3.3]. The second paper's eq. 11
  normalises them to one [arxiv:2103.16370 eq. 11]. Its segmentation and LVIS
  code rescale to the number of classes instead [code:
  semantic_seg/disalign/models/losses/grw_cross_entropy_loss.py#L34-L35] [code:
  instance_seg/lvis0.5/mask_rcnn/res50/mask_rcnn.res50.fpn.lvis.multiscale.cos_norm.disalign.1x/roi_heads/fast_rcnn.py#L51-L58].
  Neither side is checked against reported numbers.
- **Loss reduction.** The first paper's code averages over all batch-by-class
  entries [code: utils/class_balanced_loss.py#L94]. The second paper's eq. 9
  is a batch mean, but its LVIS code uses a weighted mean that divides by the
  sum of the batch weights [code: instance_seg/lvis0.5/mask_rcnn/res50/mask_rcnn.res50.fpn.lvis.multiscale.cos_norm.disalign.1x/roi_heads/fast_rcnn.py#L73-L76].
  Each changes the effective step size, so copy one reduction and keep it.
- **Segmentation $\rho$.** The second paper states $\rho = 0.3$ for all
  segmentation experiments [arxiv:2103.16370 app. B.2]. Its config sets 0.2
  [code: semantic_seg/configs/ade20k_fcn_disalign/disalign_fcn_r50-d8_512x512_160k_ade20k.py#L22-L23].
- **Edge loss, paper vs code.** The EGNet paper's edge loss is plain
  cross-entropy [arxiv:1908.08297 eq. 4]. Its release weights pixels by class
  frequency, and a comment in that code describes the opposite weighting from
  the one it applies [code: solver.py#L223-L227]. Neither side is checked
  against the reported numbers.

## Papers

- [Long-Tailed Recognition via Weight Balancing](/papers/2203.14197): uses the
  class-balanced loss as the classifier loss in a two-stage recipe, and compares
  it against cross-entropy and other long-tailed methods.
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](/papers/2103.16370):
  uses an inverse-frequency power weight in a stage-2 calibration of frozen
  classifier scores, for classification, segmentation and detection.
- [EGNet: Edge Guidance Network for Salient Object Detection](/papers/1908.08297):
  trains an edge branch with an unweighted cross-entropy in the paper; its
  released code weights edge and background pixels by class frequency.