Paper of the day, 2026-10-27

Long-Tailed Recognition via Weight Balancing

Listen to the episode, or read on. The paper: arXiv:2203.14197.

Long-tailed recognition: the rare classes lose because the common ones have longer weight vectors

A classifier trained on skewed data favours its common classes, and part of the reason is plain arithmetic: their weight vectors in the last layer grow longer. This paper doesn't add a new loss or resample the data. It keeps those vectors in check with two old tools, weight decay and a norm cap. If your labels follow a long tail, and most real labels do, this is a cheap baseline to try before anything fancier.

The problem in one picture

Drawing made for wikillm on 2026-10-27 by the AI editor. It is not taken from the paper.

How it works

Picture the last layer of an image classifier as a row of arrows, one per class. Each class's score is roughly how far the image's features point along that class's arrow, so a longer arrow tends to win. After ordinary training on long-tailed data, the arrows for common classes come out longer. Those classes simply receive many more gradient updates. The rare classes aren't necessarily misunderstood; they are just shouted down.

The authors look at three ways to even out the arrow lengths:

L2-normalisation forces every arrow to unit length. You can also apply it after training by dividing each class vector by its own norm. It helps on some datasets and hurts badly on others.
Weight decay adds a penalty on the squared size of the weights. Big weights are pulled down hardest and small ones stay small.
MaxNorm draws a ball of fixed radius around the origin. Any arrow that pokes out of the ball is rescaled back to its surface. Arrows inside can keep growing.

The paper also frames weight decay as one step toward solving the MaxNorm problem, and reports that combining the two beats using either one alone.

The recipe has two stages. Stage one trains the whole network with plain cross-entropy, and the only special ingredient is a carefully tuned weight decay on all parameters. The authors argue that this single knob is enough for good features. MaxNorm would need a separate threshold per layer here, which they found hard to tune. Stage two freezes the features and retrains only the classifier. This time it uses a class-balanced loss, which weights each class by the inverse of its "effective number" of samples so rare classes count more, and it adds weight decay and MaxNorm. The paper tunes the stage-two settings with Bayesian optimisation. In the paper text, the exact stage-two weight decay, the MaxNorm radius, the stage-two learning rate and the stage-two epoch count are all not stated. The released code fills them in. For example, it sets the radius a fraction of 0.1 of the way from the smallest to the largest initial row norm, and it projects once per epoch instead of after every batch as the paper describes.

Try it yourself

This toy has no network and no dataset. We build a long-tailed class distribution and a fake "naively trained" classifier whose row norms grow with class size. That growth rule is our own toy assumption, chosen to mimic the paper's observation. Then we compute the class-balanced weights and apply the MaxNorm projection, both as described on the card.

Python

import numpy as np
import matplotlib.pyplot as plt
K, IF, beta, thresh = 100, 100, 0.9999, 0.1     # classes, imbalance factor, CB beta, MaxNorm fraction
counts = np.round(500 * IF ** (-np.arange(K) / (K - 1)))  # toy long tail, head to tail
rng = np.random.default_rng(0)

# Toy "naively trained" classifier: common classes get longer rows
W = rng.normal(size=(K, 64))
W *= ((0.5 + 0.3 * np.log(counts)) / np.linalg.norm(W, axis=1))[:, None]

# Class-balanced weights from effective numbers, rescaled to sum to K
cb = (1 - beta) / (1 - beta ** counts)
cb = cb / cb.sum() * K

# MaxNorm: one radius from the row norms, then shrink every row above it
norms = np.linalg.norm(W, axis=1)
radius = norms.min() + thresh * (norms.max() - norms.min())
norms_after = np.linalg.norm(W * np.minimum(1, radius / norms)[:, None], axis=1)
fig, ax = plt.subplots(1, 3, figsize=(12, 3.6))
ax[0].bar(range(K), counts, color="grey"); ax[0].set_title("Images per class")
ax[1].plot(cb, color="tab:green"); ax[1].set_title("Class-balanced loss weight")
ax[2].plot(norms, label="before MaxNorm"); ax[2].plot(norms_after, label="after MaxNorm")
ax[2].set_title("Classifier row norms"); ax[2].legend()
for a in ax: a.set_xlabel("class, common to rare")
plt.tight_layout(); plt.show()
print(f"head / tail images:          {counts[0]:.0f} / {counts[-1]:.0f}")
print(f"CB weight, head vs tail:     {cb[0]:.3f} vs {cb[-1]:.3f}")
print(f"MaxNorm radius:              {radius:.3f}")
print(f"norm ratio head/tail before: {norms[0] / norms[-1]:.2f}, after: {norms_after[0] / norms_after[-1]:.2f}")

What you should see:

Left panel: a steep slope of training images per class, with an imbalance factor of 100 between head and tail.
Middle panel: the class-balanced weight climbing toward the rare end. The head class gets a small fraction of the weight the tail class gets.
Right panel: the "before" curve falls smoothly. The "after" curve is nearly flat, because with a fraction of 0.1 the cap sits close to the smallest norm and almost every row gets trimmed.
Printed numbers: the head-to-tail norm ratio should drop from above 2 to close to 1.

To explore: change thresh to 0.5 and watch the cap rise, so only the head classes get trimmed. Then try beta = 0.9. The class-balanced weights flatten out almost completely, because with so small a beta even the rare classes look "full" to the effective-number formula.

What the authors report, and what they leave open

The numbers are top-1 accuracy averaged over classes, on balanced test sets, as reported by the authors.

On CIFAR100-LT with imbalance factor 100, plain cross-entropy training reaches 38.38%. Tuning weight decay in stage one alone lifts that to 46.08%. The full recipe, a class-balanced loss with weight decay and MaxNorm in stage two, reaches 53.35%. The per-split breakdown is the fun part. Few-shot classes, those with under 20 training images, go from 11.43% to 32.63%. Many-shot classes go from 64.05% to 72.60%, so they do better too. At imbalance factors 50 and 10 the full recipe reports 57.71% and 68.67%. Retraining the last two layers instead of one gives 53.55%.

On ImageNet-LT (ResNeXt50) and iNaturalist2018 (ResNet50), the full recipe reports 53.9% and 70.2%. The authors say plainly that this trails the ensemble and self-supervised methods on those datasets, for example RIDE at 56.1% and ACE at 56.6% on ImageNet-LT. One cautionary result: post-hoc L2-normalisation drops iNaturalist accuracy to 51.3%, down from 65.4% with weight decay alone.

What stays open, according to the card:

The code doesn't match the headline. The released code covers only the CIFAR100-LT demos, and it selects the best epoch on the evaluation split. Its logs record 51.98% best-epoch accuracy for stage two, below the 53.35% in the paper.
The code differs from the paper elsewhere. It uses ResNet34 instead of ResNet32 and 320 stage-one epochs instead of 200.
Weight decay is touchy. The paper says large values destabilise training and that the best value depends on dataset size. No general value is given.
MaxNorm stops at the classifier. Hidden layers also show imbalanced filter norms. The authors call MaxNorm everywhere non-trivial and don't try it.
Only balanced test sets are studied. Skewed test distributions are not discussed.
The full study was expensive. The authors report about 2 GPU-years in total.

Ask yourself

If longer weight vectors explain part of the bias, how much is left once the norms are equal? Is that remainder in the features or in the arrows' directions?
The released code chooses epochs using the test split. How would you restructure a long-tailed benchmark so that model selection can't leak, when the rare classes have almost no images to spare for validation?
MaxNorm in hidden layers was left alone because the per-layer thresholds were hard to tune. Could a rule like the code's "fraction of the norm range" work layer by layer, and what would you measure to find out?