---
kind: concept
slug: adaptive-rounding
title: Adaptive rounding
summary: >-
  Post-training weight quantization where each weight is rounded up or down to reduce a layer's output error, instead of rounding to the nearest grid value. Needs a small unlabelled calibration set and no retraining. Author-reported: 4-bit Resnet18 weights recover most of the accuracy that nearest rounding loses.
tags: [quantization, post-training-quantization, weight-rounding]
papers: ["2004.10568"]
related: [post-training-quantization, weight-quantization, batchnorm-folding, quantization-aware-training]
status: draft
updated_at: 2026-10-24
---

## Overview

Rounding a trained weight to the nearest value on a fixed grid is the default
in post-training quantization. The problem adaptive rounding addresses is that
the nearest value is not always the best one: when many weights are rounded
together, their errors add up through the layer's inputs, so the choice for
one weight depends on the others [arxiv:2004.10568 §1] [arxiv:2004.10568 §2].

The invariant all variants share is that the scale and the grid are fixed,
and only the per-weight floor-or-ceil decision is optimised. The network is
not retrained [arxiv:2004.10568 §3.1] [arxiv:2004.10568 §5].

Only one paper with a card covers this concept, so this page describes one
method. Other rounding-optimisation methods are not covered here.

## Variants

### Layer-wise adaptive rounding (AdaRound)

Each layer is optimised on its own. For every weight the method chooses
between the floor and the ceiling of W / s. The choice is first relaxed to a
continuous value h in [0, 1] using a rectified sigmoid with stretch
parameters 1.1 and -0.1 [arxiv:2004.10568 §3.3]. A regulariser then pushes
each h towards 0 or 1, with its exponent annealed from large to small
[arxiv:2004.10568 §3.3] [arxiv:2004.10568 fig. 2]. After optimisation each
h is taken as its nearest end point [arxiv:2004.10568 §3.3] [arxiv:2004.10568
fig. 3]. The paper derives the per-layer loss from a second-order expansion of
the task loss, with a diagonal Hessian and a mean-squared error on
pre-activations [arxiv:2004.10568 §3.2] [arxiv:2004.10568 app. B]. Card:
[Up or Down? Adaptive Rounding for Post-Training Quantization](/papers/2004.10568).

The reconstruction target is the output of the float layer after its
activation, and the input is the output of the already-quantized earlier
layers. This is the "asymmetric" part of the objective. It is about which
inputs each side sees, not about an asymmetric quantization grid
[arxiv:2004.10568 §3.3].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Nearest rounding is already accurate enough for your target | Nearest rounding | It needs no calibration data or optimisation. The paper's main results are at 4-bit weights, and its sources here report no adaptive-rounding result at 8-bit weights [arxiv:2004.10568 tab. 7] |
| Weights at 4 bits, no retraining allowed, a few hundred to about a thousand unlabelled images available | Adaptive rounding | Nearest rounding gives 23.99% top-1 on Resnet18 at 4 bits, against 68.60% with this method and 69.68% FP32 [arxiv:2004.10568 tab. 6] [arxiv:2004.10568 §5] |
| Fine-tuning or retraining is possible | Not covered by this page | The paper does not compare against retraining-based methods. See [quantization-aware training](/concepts/quantization-aware-training) |
| Very deep models or tight time budgets | Measure first | Each layer is optimised in turn, so cost grows with depth. The paper reports about 10 minutes for Resnet18 on one GTX 1080 Ti and gives no timing for larger models [arxiv:2004.10568 §5] |
| Layer types other than standard convolution and linear | Check support first | The paper covers convolutional and fully connected layers [arxiv:2004.10568 §1] [arxiv:2004.10568 §3.3] |
| Calibration data is scarce | Use at least a few hundred images | Results hold down to 256 images within 2% of FP32. Images from other datasets cost less than 0.2% on average [arxiv:2004.10568 §5.1] [arxiv:2004.10568 fig. 4] |

The gain depends on the grid. With a min-max grid, nearest rounding gives
0.23% top-1 on Resnet18 and adaptive rounding gives 61.96%. With a weight-MSE
grid, the same comparison is 23.99% and 68.60% [arxiv:2004.10568 tab. 6].
Choose the grid before measuring the gain.

Activations are kept in FP32 in most results. The paper also reports 8-bit
activation rows with min/max ranges, which it says rarely hurt
[arxiv:2004.10568 §5.2] [arxiv:2004.10568 tab. 7].

## Implementation sketch

Language-agnostic pseudocode of the layer-wise method. The regulariser weight,
the beta range and any warm-up are not stated in the paper. The
[card](/papers/2004.10568) lists the values the reference implementation
uses, and its "Paper vs code" section lists the differences.

```pseudocode
# Adaptive rounding for one linear or convolutional layer
# Input:  weight W, fixed scale s, unlabelled calibration inputs X
# Output: a 0/1 choice per weight, giving the quantized weights
def adaround_layer(W, s, X, steps):
    # Floor of each weight on the grid  [arxiv:2004.10568 §3.1]
    lo = floor(W / s)
    # Soft variable per weight, initialised from the fractional part  [§3.3]
    V = init_from_fraction(W / s - lo)
    for t in range(steps):
        # Mini-batch of calibration inputs, batch size 32  [arxiv:2004.10568 §5]
        x = sample_batch(X, 32)
        # Rectified sigmoid, zeta - gamma = 1.2 and gamma = -0.1  [§3.3]
        h = clip(1.2 * sigmoid(V) - 0.1, 0, 1)
        # Quantized weights using the relaxed choice  [§3.3]
        soft_W = s * clip(lo + h, qmin, qmax)
        # Output of the float layer and of the soft-quantized layer, fed by
        # quantized earlier layers, after the activation  [§3.3]
        target = act(layer_fp(x))
        pred = act(layer(x, soft_W))
        # Push h towards 0 or 1, with beta annealed over the steps  [§3.3]
        reg = sum(1 - abs(2 * h - 1) ** beta(t, steps))
        loss = mse(pred, target) + lam * reg
        V = optimizer_step(V, loss)
    # Hard decision: round up where h ends above 0.5  [§3.3] [fig. 3]
    return lo + (h >= 0.5)
```

The regulariser weight `lam`, the bounds `qmin` and `qmax`, and the
annealing schedule `beta` are not given in the paper. The
[card](/papers/2004.10568) gives the reference values.

## Contradictions

Only one card covers this concept, so no disagreement between papers is known.

Within the paper, the abstract and conclusion say that 4-bit Resnet18 and
Resnet50 weights are within 1% of FP32 [arxiv:2004.10568 §6]. The main
Resnet18 row (68.60) is 1.08 points below FP32 (69.68). The literature-table
row (68.71), which uses 2048 images and 20k iterations, is 0.97 below
[arxiv:2004.10568 tab. 7]. Quote the number from the row you use.

## Papers

- [Up or Down? Adaptive Rounding for Post-Training Quantization](/papers/2004.10568): the layer-wise method with a rectified-sigmoid relaxation and an asymmetric reconstruction loss.