A White Paper on Neural Network Quantization

In plain words

Neural networks are usually stored and run with 32-bit decimal numbers. Storing them with far fewer bits, such as 8-bit whole numbers, makes them smaller and cheaper to run on phone-class chips. The catch is that the rounding adds noise, and the noise can cost accuracy.

This white paper is a practical guide to keeping that cost low. It explains how the rounding works on the chip, then covers two families of fixes. The first family changes the numbers without retraining, which is quick and needs little data. The second family retrains the network while it already experiences the rounding, which costs more effort and can reach lower bit-widths. The authors also give a step-by-step recipe for each family and a checklist for finding out why a quantized model is less accurate than the original.

TL;DR

• A white paper that presents post-training quantization (PTQ) and quantization-aware training (QAT) as two pipelines for uniform affine quantization §1 §2.2.
• PTQ pipeline: cross-layer equalization, weight range setting with MSE, AdaRound or analytic bias correction, activation range setting §3.5. QAT pipeline: start from a PTQ-initialised, pre-trained model, keep batch norm in the simulation, learn the quantization parameters §4.4.
• Author-reported: W8A8 is close to float accuracy for most tested models, and W4A8 is mostly within about 1 to 2 points with per-channel quantization and QAT tab. 6 tab. 10.
• No official code is published. All values come from the paper's text and tables. See Paper vs code.

Problem

Fixed-point accelerators run multiply-accumulate operations on integers. Storing and moving 8-bit values instead of 32-bit values cuts memory traffic by a factor of 4 and the cost of multiplication by a larger factor §1 §2.1. The rounding noise from low-bit quantization, however, can lower accuracy, and some networks are more sensitive to it than others §1. The paper's stated scope is homogeneous bit-width (the same bit-width for all weights, and the same for all activations), per-tensor or per-channel scales, and uniform grids §2.4.

The paper does not state one failing model or one failing layer type as its motivation. It states two cases where the simple approach is not enough: depth-wise separable convolutions with per-tensor weights, which lose much accuracy §3.2, and 4-bit or lower quantization, which needs more than round-to-nearest §3.4 §4.

Method

Scheme. Uniform affine (asymmetric) quantization has a scale s, an integer zero-point z and a bit-width b. A real value is divided by s, rounded, shifted by z, and clamped to the unsigned grid {0, ..., 2^b − 1}. De-quantization returns s · (x_int − z). The grid limits are q_min = −s·z and q_max = s·(2^b − 1 − z) §2.2 eq. 4 eq. 5 eq. 6 eq. 7.

Symmetric variant. The zero-point is fixed to 0, and the grid is signed or unsigned depending on the distribution §2.2.1 eq. 8. A power-of-two scale (s = 2^−k) turns rescaling into a bit shift, at the cost of a coarser choice of step size §2.2.2.

Granularity. Per-tensor scales are the common default. Per-channel scales for weights are supported by many accelerators and are needed for some networks. Per-channel activation scales are hard to implement in fixed point §2.2.3 §2.4.2.

Simulation. A quantizer block is placed after each weight and after each activation. The bias usually stays unquantized §2.3 fig. 4. Batch normalization is folded into the preceding linear layer. For a weight row W_k and BN parameters γ, β, mean μ and standard deviation σ, the folded weights are γ_k W_k / sqrt(σ_k² + ε) and the folded bias is β_k − γ_k μ_k / sqrt(σ_k² + ε) §2.3.1 eq. 9 eq. 10 eq. 11 eq. 12.

Activation functions. A ReLU can be merged into the requantization step by setting the minimum of the activation grid to 0. Sigmoid and Swish need quantizers before and after them, unless the hardware supports them §2.3.2.

Other layers. Max-pooling needs no extra quantizer. Average-pooling and element-wise addition need care, and concatenation needs matching grids or a requantization step §2.3.3.

Asymmetric weights are costly. Asymmetric weights and activations add a data-dependent term that is expensive at inference. The paper therefore uses asymmetric activations with symmetric weights §2.4.1 eq. 13.

PTQ range setting

The quantization range [q_min, q_max] trades clipping error against rounding error §2.2 §3.1.

• Min-max. q_min is the tensor minimum and q_max its maximum. There is no clipping error, but outliers inflate the rounding error §3.1 eq. 14 eq. 15.
• MSE. Choose q_min and q_max that minimise the squared error between the tensor and its quantized version. The paper states grid search, golden-section search and closed-form approximations as ways to solve it §3.1 eq. 16.
• Cross-entropy. For the last layer's logits, minimise the cross-entropy between the softmax of the float logits and the softmax of the quantized logits, because the order of the largest values matters more than their exact values §3.1 eq. 17.
• BN-based. For an activation after BN, the per-channel mean and standard deviation equal the BN shift β and scale γ. With α = 6, the range is [min(β − αγ), max(β + αγ)], which clips only large outliers. The method needs no data §3.1 eq. 18 eq. 19.

Cross-layer equalization (CLE)

For a ReLU (or piece-wise linear) activation f, f(s·x) = s·f(x) for s ≥ 0. Consecutive layers can therefore be rescaled per channel without changing the output: divide channel i of layer 1 (its weights and bias) by s_i, and multiply the matching input column of layer 2 by s_i §3.2 eq. 20 eq. 21. The scale that equalises the two channel ranges r1_i and r2_i is s_i = sqrt(r1_i · r2_i) / r2_i §3.2 eq. 22.

CLE is applied only inside residual blocks, because skip connections break the equivariance across blocks §3.2.

Bias absorption. After CLE, a high bias can be moved into the next layer, where it becomes a bias term, when the ReLU input is known to be above a threshold c for almost all inputs. The threshold is c_i = max(0, min_x (W_i x

• b_i)) §3.2 eq. 23 eq. 24. Without data it is estimated as c_i = max(0, β_i − 3γ_i), which holds for about 99.865 % of inputs if the pre-activation is normal §3.2.

Bias correction

Quantization shifts the expected output by −ΔW·E[x], where ΔW = Ŵ − W. The correction subtracts this shift from the bias §3.3 eq. 25 eq. 26.

• Empirical. Compare the float and quantized layer outputs on calibration data §3.3 eq. 27.
• Analytic. Estimate E[x] from BN parameters, treating the pre-activation as normal and using the clipped-normal mean after ReLU §3.3 eq. 28. The text of §3.3 says γ and β are the standard deviation and the mean, which is the reverse of the convention used in §3.1 and in eq. 28; eq. 28 is followed here (β is the mean and γ the standard deviation).

AdaRound

Rounding weights to nearest is not the best choice for the task loss when weights are quantized post-training. On ResNet-18's first layer at 4 bits, the best of 100 stochastic roundings beats nearest by more than 10 points §3.4 fig. 7.

AdaRound starts from the task-loss objective eq. 29. A second-order Taylor expansion turns it into a quadratic binary problem eq. 30. Assuming a converged model (the gradient term is dropped) and a block-diagonal Hessian that ignores cross-layer correlations, the problem becomes a local output MSE for each layer eq. 31. A continuous relaxation then optimises soft rounding variables V through a soft-quantized weight eq. 32 eq. 33, with a regulariser that pushes each variable towards 0 or 1 eq. 34. The final objective includes the activation function and uses the already-quantized inputs of earlier layers eq. 35. The paper uses a rectified sigmoid for h and optimises with stochastic gradient descent, using a small unlabeled calibration set. The regulariser is annealed during optimisation §3.4.

Standard PTQ pipeline

The recommended order is §3.5 fig. 8:

1. Cross-layer equalization on the float model (important for depth-wise separable networks and per-tensor quantization).
2. Add quantizers: symmetric weights and asymmetric activations. Per-channel weight quantization if the hardware and software stack support it.
3. Weight range setting with layer-wise MSE. Min-max may be preferable for per-channel weights.
4. AdaRound, if a calibration set of about 500 to 1000 unlabeled images is available (the footnote in §3.5). This step is described as crucial for 4-bit weights.
5. Otherwise, analytic bias correction, if the network uses batch normalization.
6. Activation range setting: MSE on a small calibration set for most layers, or BN-based range setting for a fully data-free pipeline.

The paper says some steps can be dropped or swapped depending on the model §3.5.

PTQ debugging

A seven-step flow §3.7 fig. 9:

1. FP32 sanity check: set bit-widths to 32, or bypass the quantizers, and check that the accuracy matches the float model.
2. Quantize weights only, then activations only, to see which one causes the drop.
3. If weights cause the drop: try CLE, per-channel quantization, and bias correction or AdaRound with calibration data.
4. If activations cause the drop: try other range settings, or CLE that accounts for activation ranges.
5. If the drop remains: per-layer analysis. Quantize one quantizer at a time to the target bit-width while the rest stay at 32 bits.
6. Visualise the distribution of the problematic tensor, per channel or per token, as in fig. 5.
7. Fix the individual quantizer with custom range setting or a higher bit-width. If that fails, use QAT.

QAT forward and backward

QAT inserts the same quantizers as PTQ and trains the network with them in the forward pass §4 fig. 4.

• Straight-through estimator (STE). The gradient of round() is set to 1. With this, the gradient with respect to the input is 1 inside the grid and 0 outside it §4.1 eq. 36 eq. 37.
• Learned scale. The gradient with respect to s is −x/s + round(x/s) inside the grid, and n or p outside it §4.1 eq. 38. Updating s every epoch instead of learning it leads to better results, especially at low bit-width §4.4.
• Learned zero-point. z is kept real and rounded in the forward pass. The STE is applied to the rounding of z §4.1 eq. 39 eq. 40.

# Uniform affine quantization, forward and back  [eq. 4] [eq. 5] [eq. 6]
# Input:  real tensor x; scale s > 0; zero-point z; bit-width b
# Output: x_hat, the value the integer grid represents

def quantize_dequantize(x, s, z, b):
    # Round to the nearest step and shift by the zero-point  [eq. 4]
    shifted = round(x / s) + z
    # Clamp to the unsigned grid {0, ..., 2^b - 1}  [eq. 4] [eq. 5]
    x_int = clamp(shifted, 0, 2**b - 1)
    # De-quantize to the real line  [eq. 6]
    x_hat = s * (x_int - z)
    return x_hat

# Cross-layer equalization for two consecutive layers  [§3.2] [eq. 22]
# Input:  W1 (C, I) and b1 (C,) feeding a ReLU, then W2 (O, C)
# Output: rescaled W1, b1, W2 that compute the same function

def equalize(W1, b1, W2):
    # Per-channel dynamic range of each layer's weights  [§3.2]
    r1 = max(abs(W1), axis=1)            # (C,)
    # r2 is the range of input channel i of W2  [§3.2]
    r2 = max(abs(W2), axis=0)            # (C,)
    # Scale that equalises the two ranges  [eq. 22]
    s = sqrt(r1 * r2) / r2               # (C,)
    # Divide layer 1's channels, multiply layer 2's; output is unchanged  [§3.2]
    W1 = W1 / s[:, None]
    b1 = b1 / s
    W2 = W2 * s[None, :]
    return W1, b1, W2

# Analytic bias correction without data  [§3.3] [eq. 25] [eq. 28]
# Input:  W and W_hat (C_out, C_in), bias (C_out,)
#         beta (mean) and gamma (std) of the BN layer before this one (C_in,)
# Output: corrected bias (C_out,)

def bias_correction(W, W_hat, bias, beta, gamma):
    # Clipped-normal mean of the post-ReLU input  [eq. 28]
    ratio = -beta / gamma
    mean_x = gamma * normal_pdf(ratio) + beta * (1 - normal_cdf(ratio))
    # Expected output shift caused by the weight error  [eq. 25] [eq. 26]
    shift = (W_hat - W) @ mean_x
    # Remove the shift from the bias, so inference cost is unchanged  [§3.3]
    return bias - shift

# Simulated quantization for QAT with straight-through gradient  [§4.1]
# Input:  float tensor x; learnable scale s; learnable zero-point z;
#         integer grid limits n and p  [§4.1]
# Output: fake-quantized tensor; gradients flow to x, s and z

def fake_quant(x, s, z, n, p):
    # Round the zero-point too, so that it can be learned  [eq. 39]
    z_int = round(z)
    # Round to the step grid, shift, and clamp to [n, p]  [eq. 39]
    x_int = clamp(round(x / s) + z_int, n, p)
    # Forward uses rounded values; backward treats round() as identity  [eq. 36]
    return s * (x_int - z_int)

BatchNorm in QAT

Static folding (folding BN into the weights and bias before training, then training the folded model) is the paper's default for per-tensor quantization. It is simpler than the double forward pass of Krishnamoorthi, which updates BN statistics and applies a correction. In the paper's runs, static folding is on par or better §4.2 tab. 7.

For per-channel quantization, BN can stay unfolded during training, and its scale can be merged into the per-channel weight scales afterwards. The paper states this is on par or better than static folding §4.2 eq. 41.

QAT initialization

Starting from a pre-trained float model is common practice. Better range initialization (layer-wise MSE rather than min-max) helps early epochs but the advantage shrinks with longer training §4.3 tab. 8 fig. 11. For MobileNetV2, CLE is needed for QAT to train at all, because per-tensor quantization causes catastrophic drops §4.3 tab. 9.

Standard QAT pipeline

§4.4 fig. 12:

1. Cross-layer equalization on the float model. It is necessary for networks with imbalanced weights, such as MobileNet, and optional otherwise or with per-channel quantization.
2. Add quantizers as in PTQ: symmetric weights, asymmetric activations, per-channel weights if supported. Make batch-norm simulation correct (see BatchNorm in QAT).
3. Range estimation: layer-wise MSE for all quantization parameters. Min-max can sometimes be favourable for per-channel quantization.
4. Learnable quantization parameters. With SGD-type optimizers, the learning rate for quantization parameters should be lower than for weights. Adam or RMSProp avoid this adjustment.

Hyperparameters

Values come from the paper text and table captions. The paper has no appendix, and no official code (see Paper vs code).

Results

All numbers are author-reported. Setups: ImageNet classification on ResNet-18, ResNet-50, MobileNetV2, InceptionV3, EfficientNet-lite; DeepLabV3 (MobileNetV2 backbone) on Pascal VOC (mIoU); EfficientDet-D1 on COCO 2017 (mAP); BERT-base on GLUE. Values are on the validation sets tab. 6 tab. 10.

Range setting, weights only (ResNet-18 FP32 69.68; MobileNetV2 FP32 71.72; 5 runs) tab. 1

Range setting, activations only (weights FP32; 5 runs) tab. 2

CLE on MobileNetV2, INT8 weights and activations tab. 3

Bias correction on MobileNetV2 tab. 4

AdaRound, ResNet-18 weights, 5 runs tab. 5 (the bit-width is not stated in the table caption; the text's first-layer experiment is 4-bit §3.4)

Standard PTQ pipeline, W8A8 and W4A8, 5 runs tab. 6

BERT-base keeps a few activations at 16 bits tab. 6.

BatchNorm handling in QAT, 3 runs tab. 7

Initialization for QAT, 3 runs tab. 8 (PTQ accuracy is before training, QAT after)

CLE and bias correction as QAT initialization, 3 runs tab. 9

See Pitfalls for the label conflict between tab. 8 and tab. 9.

Standard QAT pipeline, 3 runs tab. 10

PT = per-tensor, PC = per-channel. The paper's headline conclusion is that the pipeline reaches 8-bit weights and activations within about 1 % of float accuracy and 4-bit weights with a small extra drop §5.

Paper vs code

No official code. The paper does not link a repository, a project page or a URL for code (the arXiv abstract and the LaTeX source contain none; the arXiv comments field is empty in our metadata). The LaTeX source mentions AIMET only in a commented-out line (sections/04_qat.tex, line 168), and its table comments refer to an internal experiment-manager script that is not public.

I also searched GitHub for the title and for "AIMET". The top result is qualcomm/aimet, owned by the Qualcomm organisation and listed under the license NOASSERTION. It is not linked by the paper. Qualcomm is the authors' employer, but the repository is not clearly the authors' own lab's, so I did not treat it as official and did not clone it or quote it. A human could decide whether it qualifies.

Pitfalls

• No official code, so no code check. Every hyperparameter, layer and algorithm detail above is from the paper alone. Where the paper is silent, an implementation must choose (for example, AdaRound's λ, β schedule and iteration count).
• Equation numbers are counted, not checked. Numbers such as eq. 22 were counted in order from the LaTeX source. The compiled PDF was not read, so check a few against the PDF before relying on them.
• Text says "within 0.7 %" for W8A8 PTQ, the table disagrees. §3.6 says 8-bit quantization is within 0.7 % of float for all models. tab. 6 shows MobileNetV2 per-tensor at 70.99 (a 0.73 drop) and EfficientDet-D1 at 38.29 (a 1.79 drop). The conclusion's "within 1 %" also fails for EfficientDet-D1.
• Text says W4A4 QAT is within 1 %; the table disagrees for ResNet-18. §4.5 says the W4A4 drop on networks without depth-wise convolutions stays within 1 %. tab. 10 shows ResNet-18 per-tensor W4A4 at 68.32 (1.36 below float).
• "Up to 5 %" is not the worst case. §4.5 says per-channel W4A4 drops by up to 5 %. tab. 10 shows BERT-base at 77.63 (5.43 below) and EfficientDet-D1 at 28.68 (11.40 below). EfficientDet-D1 is called harder in the text, but the number is not stated there.
• Table 8 and table 9 share MobileNetV2 values under different labels. The pair 12.99 (PTQ) and 70.13 (QAT) appears as "W4A8 with MSE weight init" in tab. 8 and as "W4A8 with CLE" in tab. 9. The source of table 9 has a TODO update PTQ numbers comment. Which label is correct is not determinable from the paper.
• Table 10's RN-18 W4A8 per-channel QAT is 70.01, above float. The text calls this regularisation or extra fine-tuning, which is an interpretation, not a measurement.
• Edge of the 1.5 % claim. §4.5 says per-channel W4A8 QAT brings MobileNetV2 and EfficientNet-lite to "less than 1.5 %" of float. EfficientNet-lite is 1.50 below float in tab. 10 (75.42 − 73.92), so it is not strictly less than 1.5.
• Double-forward BN method has two attributions. §4.2 credits the double forward pass to Jacob et al. (2018), but tab. 7 labels the same row "Double forward (Krishnamoorthi)". The card follows the table's label in the results and the text's citation in the lineage. Which is correct is not stated.
• BN naming in §3.3. The text says γ and β are the standard deviation and mean. Eq. 28 and §3.1 use β as the mean and γ as the standard deviation. Use the latter.
• Analytic bias correction assumes normal pre-activations. The clipped-normal mean eq. 28 is an approximation. The paper reports that the correction brings MobileNetV2 from 0.12 to 52.02 tab. 4, but gives no error bound.
• Absorption threshold is data-dependent unless estimated. c_i = max(0, β − 3γ) is exact only for normal inputs, with the 99.865 % coverage quoted in §3.2. Using a calibration set is the exact route, but the paper states no calibration size for it.
• Absorbing bias costs FP32 accuracy. MobileNetV2 drops from 71.70 to 71.57 in FP32 tab. 3. This is noted in the text as an approximation.
• Zero-point gradient is zero inside the grid. The paper's gradient for the learned zero-point eq. 40 is 0 inside the grid and −s outside. Implement it as stated, but note it is an STE result, not a measured property.
• Learned quantization parameters need an optimizer decision. SGD requires a lower learning rate for scales and zero-points §4.4. The factor is not given.
• AdaRound is only described for the local objective. §3.4 states that the final objective uses the already-quantized inputs of earlier layers. The learning rate, number of iterations, batch size and λ are not stated.
• Calibration size for AdaRound and activations is a footnote. 500 to 1000 images §3.5 is given only in a footnote. The paper does not say how many batches the activation range calibration uses.
• BERT activations at 16 bits. The PTQ BERT number in tab. 6 keeps a few activations at 16 bits, found by debugging §3.5. It is not pure W8A8 or W4A8. The list of those activations is not stated.
• Unused tables in the source. The LaTeX source contains tables that the compiled text does not include: a stochastic-rounding comparison, an AdaRound extensions table, a CLE table for ShuffleNetV2 and a non-asymmetric activation range table. None were used here. The 10-point claim in §3.4 is cited to the text only.
• Figures are described, not reproduced. Figure numbers follow the source order: 1 accelerator, 2 requantization, 3 quantization grids, 4 simulation, 5 MobileNetV2 channel ranges, 6 CLE rescaling, 7 AdaRound correlation, 8 PTQ pipeline, 9 debugging flow, 10 STE graph, 11 activation init curves, 12 QAT pipeline. Figure 11 shows three runs with standard deviation shading fig. 11.
• Homogeneous bit-width only. Mixed precision is out of scope, and the paper's numbers do not apply to it §2.4.
• Hardware assumptions. Per-channel activation quantization is hard to implement, and the paper assumes accelerators cannot apply per-group scales §2.2.3 §2.4.2. Check the target before choosing granularity.

Lineage

• Builds on: Nagel et al., "Data-Free Quantization through Weight Equalization and Bias Correction" (DFQ), for CLE, bias absorption and analytic bias correction §3.2 §3.3. Meller et al., "Same, Same But Different", is a concurrent method for CLE §3.2. Nagel et al., "Up or Down? Adaptive Rounding for Post-Training Quantization" (AdaRound, ICML 2020), for weight rounding §3.4. Banner et al. (2019) for MSE-style range search §3.1. Krishnamoorthi (arXiv:1806.08342) for BN folding §2.3.1. Jacob et al. (2018) for the double forward pass BN method §4.2. Esser et al. (2020, LSQ) for learned step sizes §4.1. Bengio et al. (arXiv:1308.3432) for the straight-through estimator §4.1.
• Compared against: the paper compares its own variants (min-max, MSE, cross-entropy, BN-based; per-tensor and per-channel) in the ablation tables tab. 1 to tab. 9, and the double forward BN method tab. 7.
• Superseded by: not known from content already in this wiki.