Quantization-aware training

Overview

Quantization-aware training (QAT) trains a network with the same rounding it will see at inference. Each weight and activation tensor is passed through a quantize-then-dequantize operation in the forward pass, so the loss reflects the integer grid. The backward pass treats that rounding as the identity inside the representable range and as zero outside it. This is the straight-through estimator 1806.08342 §2.4 2106.08295 §4.1. A float copy of each weight is kept and updated, so small gradient steps are not lost to rounding 1806.08342 §3.2. The fully quantized variant on microcontrollers, below, does not describe such a copy 2407.10734 §3.1. Learned quantizers go further and train the grid parameters themselves, including an offset for skewed activations 2004.09576 §3.1; see the learned-quantizer variants below.

The invariant is that the training graph mirrors the inference graph. Every fusion made at deployment, and every rescaling of inputs to an add or concat, must also be modeled during training 1806.08342 §3.2.1. Most sources start QAT from a float checkpoint and fine-tune it. The 2018 paper recommends this and reports it beating training from scratch 1806.08342 §3.2, §4. The 2021 white paper also starts from a pre-trained float model, and runs its PTQ steps before QAT 2106.08295 §4.3, §4.4. The binary-network paper does the same, initialising from a full-precision model 1711.11294 §3.3. The on-device paper pretrains in float and quantizes before deployment, then trains its last layers on the device 2407.10734 §4.1. The PACT paper is the exception in this set: its networks are trained from scratch 1805.06085 §5, app. B. The alternative is post-training quantization, which needs only a few calibration batches 1806.08342 §3.1.2. The papers' measurements show where that alternative stops being enough.

Variants

The variants come from eleven sources: the 2018 whitepaper on CNN quantization 1806.08342, the 2021 white paper on neural network quantization 2106.08295, the LSQ+ paper on learned low-bit quantization 2004.09576, a tutorial on spiking networks 2109.12894, a dual-encoder paper that stores embeddings at 8 bits 1911.03688, a study of training for analog in-memory chips 2302.08469, the PACT paper on learned activation clipping 1805.06085, a binary-network paper that uses several weight and activation bases 1711.11294, a remote-sensing embedding model that quantizes its output to 8 bits 2506.20380, a survey of inference quantization 2112.06126, and a study of fully quantized training on microcontrollers 2407.10734. Each variant names its source, and the settings are the ones that source tested. Where the sources disagree, see Contradictions.

Per-layer asymmetric QAT (8 bits)

One scale and zero-point per tensor, with a uniform affine quantizer 1806.08342 §2.1. This is the layout that post-training quantization breaks on MobileNets. With QAT it recovers to 0.70 top-1 on MobileNet-v1 against 0.709 for float, while the same layout post-training gives 0.001 1806.08342 tab. 4. On Nasnet-Mobile it reaches 0.73 against 0.74 1806.08342 tab. 4. Use it when the target kernel only supports per-tensor grids. Card: Quantizing deep convolutional networks for efficient inference: A whitepaper.

Symmetric per-channel QAT (8 bits)

One symmetric grid per output channel of each weight tensor 1806.08342 §2.2, §2.6. Activations stay per-layer 1806.08342 §2.6. On MobileNet-v2 1.4 it reaches 0.745 against 0.749 for float, and on Resnet_v1_50 0.75 against 0.752 1806.08342 tab. 4. For the same weights, post-training quantization gives 0.74 on MobileNet-v2 1.4 and 0.751 on Resnet_v1_50, so QAT adds little in those rows 1806.08342 tab. 4. The per-channel post-training numbers are already close to float for most networks, so QAT matters most for the grid-limited cases above.

The 2021 paper's per-channel W8A8 QAT row, under its full pipeline, is above float for ResNet-18 (70.43 against 69.68) and MobileNetV2 (71.82 against 71.72), and below it for EfficientNet-lite (74.75 against 75.42) 2106.08295 tab. 10.

Symmetric per-channel QAT with 4-bit weights

Weights at 4 bits, activations at 8 bits (written W4A8 below) 1806.08342 §3.2.4. Post-training 4-bit weights give 0.001 top-1 on both MobileNets, while QAT gives 0.65 on MobileNet-v1 and 0.62 on MobileNet-v2, against floats of 0.709 and 0.719 1806.08342 tab. 5. A gap of about 6 to 10 points remains on MobileNets. On Resnet_v1_50 the gap is 0.732 against 0.752 1806.08342 tab. 5. The paper also reports that smaller networks lose the most under quantization 1806.08342 §3.1.3.

The 2021 paper's per-channel W4A8 QAT, after its standard pipeline, is higher: 70.01 on ResNet-18 and 70.48 on MobileNetV2, against floats of 69.68 and 71.72 2106.08295 tab. 10. The two results conflict; see Contradictions.

Low-bit activations

Activations at 4 bits with 8-bit weights (written W8A4 below). QAT reaches 0.64 on MobileNet-v1 and 0.58 on MobileNet-v2 at this setting 1806.08342 tab. 6. In every row of that table, 4-bit weights with 8-bit activations scores higher than the reverse. The paper notes that the comparison mixes granularities 1806.08342 §3.2.4. Treat this as a measured result for these networks, not a general rule.

The 2021 paper's W4A4 per-channel QAT row reaches 68.83 on ResNet-18 and 66.89 on MobileNetV2 2106.08295 tab. 10. Across its five ImageNet CNNs, W4A4 QAT with its pipeline gives 66.43 to 77.74 top-1 over both granularities 2106.08295 tab. 10.

Learned activation clip level (PACT)

Each layer's activations are clipped at a clip level alpha, which is a trainable parameter, and then rounded to k bits. The gradient reaches alpha through a straight-through estimator: the output's derivative with respect to alpha is 1 where the input is at or above alpha, and 0 elsewhere 1805.06085 §4, eq. 3. The paper uses one alpha per layer, which it found the best scope among those tried 1805.06085 §4.2, app. A.1. It leaves the first and last layers at full precision in its main comparisons 1805.06085 §4.2. Its weights are DoReFa-quantized at the same bit-width, so the comparison isolates the activation side. With these weights, PACT activations score above DoReFa activations in every cell of the paper's table, for example 0.765 against 0.714 for ResNet-50 at 4 bits 1805.06085 tab. 1. At 4 bits the paper reports results within about 1 point of full precision on the five networks tested. At 2 bits the gap is larger: 0.644 against 0.702 on ResNet-18, and 0.722 against 0.769 on ResNet-50 1805.06085 tab. 1. These are author-reported results from networks trained from scratch, so the numbers are not a fine-tuning benchmark. Card: PACT: Parameterized Clipping Activation for Quantized Neural Networks.

Batch-norm handling during QAT

The 2018 paper compared four schemes on MobileNet-v1 1806.08342 §4, fig. 14:

• naive folding with batch statistics, which makes the quantized weights change every batch;
• batch renormalization;
• moving averages throughout training without freezing;
• batch-norm correction with freezing after a delay.

Only the last gives higher and steadier evaluation accuracy in the paper 1806.08342 §4, fig. 14. The correction scales the folded weights toward the long-term standard deviation, and freezing then switches to the moving statistics that inference uses 1806.08342 §3.2.2. The freeze delay was about 200000 steps for MobileNet-v1 and 300000 to 400000 for MobileNet-v2 1806.08342 §3.2.2, §4, fig. 14, fig. 15. The paper's text and captions disagree on the MobileNet-v2 value; see Contradictions.

The 2021 paper did not test correction with freezing. It compared other options at W4A8 and W4A4 on MobileNetV2 and ResNet-18 2106.08295 tab. 7. For per-tensor quantization it uses static folding, which it reports as on par with or better than the double-forward method, and better than that method in all four columns of its table 2106.08295 §4.2, tab. 7. The source labels that method "Krishnamoorthi" in tab. 7 and attributes it to Jacob et al. (2018) in the text; the card does not settle which is right. For per-channel quantization, BN can stay unfolded during training, with its scale merged into the per-channel weight scales afterwards 2106.08295 §4.2, eq. 41. That scored highest in three of the four columns; the exception is MobileNetV2 at W4A8, 70.48 against 70.52 for static per-channel folding 2106.08295 tab. 7.

Stochastic rounding during training

Adding uniform noise before rounding during training gave worse results than deterministic rounding 1806.08342 §4, fig. 12. The paper studies it and does not recommend it. Deterministic rounding is the default for the other variants.

Weight averaging

Exponential moving averages of the weights fell behind the live weights late in QAT 1806.08342 §4, fig. 15. Evaluating both, as a check, is a reasonable step. The decay rate is not stated.

Standard pipeline with PTQ initialisation

The 2021 paper's standard QAT pipeline 2106.08295 §4.4, fig. 12:

1. Start from a pre-trained float model. Apply cross-layer equalization (CLE) when weights are imbalanced, as in MobileNets. It is optional otherwise, or with per-channel quantization.
2. Add quantizers: symmetric weights, asymmetric activations, and per-channel weights if the target supports them. Make the batch-norm simulation correct; see the BN variant above.
3. Set all ranges with layer-wise MSE. Min-max can be better for per-channel quantization 2106.08295 §3.1, §4.4.
4. Make the quantization parameters learnable.

Better range initialisation helps early epochs, but the advantage shrinks with longer training. On ResNet-18 at W4A8, QAT reaches 69.61 from min-max initialisation and 69.76 from MSE 2106.08295 §4.3, tab. 8. For MobileNetV2, the paper says CLE is needed for QAT to train at all. Its W4A8 baseline ends at 0.10 top-1 after QAT, and adding CLE gives 70.13 2106.08295 §4.3, tab. 9. The table's labels conflict with tab. 8; see Contradictions. The paper's QAT runs use Adam, with 20 epochs for ImageNet models 2106.08295 §4.5. Its text also says that SGD-type optimizers need a lower learning rate for the quantization parameters than for the weights 2106.08295 §4.4.

Learned scale and zero-point

The 2021 paper makes the scale a learnable parameter. Its gradient is −x/s + round(x/s) inside the grid, and the grid limit it saturates to outside it 2106.08295 §4.1, eq. 38. The zero-point is kept real and rounded in the forward pass, with the straight-through rule applied to that rounding 2106.08295 §4.1, eq. 39–40. The paper's zero-point gradient is zero inside the grid 2106.08295 eq. 40, which is a straight-through result rather than a measurement.

Learned scale and offset, asymmetric activations (LSQ+)

The LSQ+ paper learns a scale s and an offset β for activations, so the grid can slide to the actual lowest value. This targets skewed activations such as Swish, whose negative part is bounded near −0.278 while the positive part is unbounded 2004.09576 §3.1. The offset is subtracted before scaling and added back after rounding, and both are trained with straight-through gradients 2004.09576 §3.1 eq. 2–4. Weights stay symmetric and signed 2004.09576 §3.2.1. The paper states that the offset adds a per-output bias at inference, so it has no runtime cost beyond that bias 2004.09576 §3.1 eq. 5.

Author-reported on ImageNet, with pretrained float starts. EfficientNet-B0 with the asymmetric unsigned configuration reaches 73.8% at W4A4, against 71.9% for symmetric unsigned LSQ and 76.1% full precision 2004.09576 tab. 2. At W2A2 the same configuration reaches 48.7% against 43.5% for LSQ 2004.09576 tab. 2. On MixNet-S at W4A4 the gain is 71.7% against 70.4% 2004.09576 tab. 3. On ResNet18, the asymmetric and symmetric configurations match at W4A4 2004.09576 tab. 4. The offset result on EfficientNet-B0 at W4A4 is 71.9% with β fixed at zero, 72.5% with β fixed at the Swish minimum, and 73.8% with β learned 2004.09576 tab. 6.

Initialisation is a second part of the method. The weight scale comes from the layer's mean and standard deviation, taking ±3σ 2004.09576 §3.2.1. The activation scale and offset are fitted by minimising squared error on a few batches 2004.09576 §3.2.2 eq. 8. Over five runs at W4A4 on EfficientNet-B0, the mean is 71.3% ± 2.2 with min-max initialisation, 72.0% ± 1.6 with LSQ's, and 73.0% ± 0.9 with the proposed one 2004.09576 tab. 5. The paper's weight formula divides by 2^(b−1), although the signed range has 2^b − 1 levels, and it does not say whether this is intended 2004.09576 §3.2.1. The optimiser, learning rate, epochs and batch size are not stated 2004.09576 §4, and the paper has no official code.

Quantized spiking networks

A tutorial on spiking networks reports that such networks are robust to quantization-aware training. Its listings quantize weights to 8 bits and membrane states to 4 bits 2109.12894 §4.3.2. It gives no accuracy numbers for this. It suggests clipping negative membrane values to zero, but only from early experiments, and only where inhibitory spikes are not used 2109.12894 §4.3.2 box. Card: Training Spiking Neural Networks Using Lessons From Deep Learning.

Mixed-precision QAT with 8-bit embeddings (ConveRT)

A dual encoder for reply selection keeps its embedding table on an 8-bit grid and its other network weights at 16 bits 1911.03688 §2.1, tab. 2. Training uses the mixed-precision scheme of Micikevicius et al.: FP32 shadow copies, FP16 casts for the forward pass, and layer norm, L2 norm and attention softmax kept in FP32 1911.03688 §2.1. The loss is scaled by 128 and the updates are divided by the same factor 1911.03688 §2.1. The embedding range is refreshed during training, with a margin of 10% of the range or 0.01, whichever is larger 1911.03688 §2.1. The paper says the quantization allowed a doubled batch size, and so more in-batch negatives 1911.03688 §2.1. The refresh cadence is not stated, and the paper gives no separate result for quantization alone: its ablation in tab. 4 varies architecture choices only 1911.03688 tab. 4. The model-size column in tab. 2 is computed with the same bit widths, but its ConveRT figure does not follow from them; see Contradictions. Card: ConveRT: Efficient and Accurate Conversational Representations from Transformers.

Noise- and converter-aware training for analog crossbars (HWA)

The forward pass simulates an analog in-memory crossbar, and the backward pass uses floating-point gradients 2302.08469 Methods: AIMC hardware-aware DNN training. The simulated chip includes 8-bit input and output converters, programming error and drift, read and output noise, and IR-drop 2302.08469 §2.1, eq. 4–6. The input range, the per-column output scales and the input clipping bound are learned as well 2302.08469 Methods: Learning the input range. The weight noise is redrawn once per mini-batch 2302.08469 Methods: Re-training with weight noise injection. This differs from the rounding-only variants above: the main effect studied is analog noise, and the rounding is one part of the chip model.

Without retraining, direct mapping misses the iso-accuracy target for almost every one of the eleven networks. Only LSTM PTB reaches it, at 99.0 normalized at one hour 2302.08469 §2.2, tab. 2. With HWA training, every network reaches at least 96% normalized accuracy at one hour 2302.08469 §2.3, tab. 3, and five reach iso-accuracy: WideResNet-16, BERT, the speech model, the LSTM and RNN-T 2302.08469 §2.3, tab. 3. The worst raw-accuracy gap falls from 21.81% to 2.65% across models 2302.08469 §4. The sensitivity study ranks output-side noise and ADC or DAC resolution as the largest contributors 2302.08469 §2.4, fig. 4. The paper reports that DAC resolution can drop from 8 to 6 bits with little accuracy impact 2302.08469 §2.5. Convolutional networks are the most sensitive, and ResNet-18 degrades to 99% at 1.2 times the standard PCM noise 2302.08469 §2.4, §2.5, fig. 5. These are author-reported results on the paper's own chip model, and the paper states that its hyperparameters were tuned for drift averaged over 1 s to 1 y 2302.08469 tab. 3 caption. Card: Analog in-memory hardware-aware training study.

Multiple binary bases with shifted binary activations (ABC-Net)

Each layer's real weights are written as a linear combination of M binary bases. Each basis is the sign of the centred weights, shifted by a multiple of the weight spread. The scale factors come from least squares on the fixed bases 1711.11294 §3.1.1, eq. 1–3. The activations are written as a combination of N binary activations, each with its own trainable threshold shift and scale 1711.11294 §3.2, eq. 9, eq. 11. A convolution then becomes M×N bitwise convolutions with scale factors 1711.11294 eq. 12. Training keeps real-valued weights and passes gradients through the sign function with the straight-through estimator 1711.11294 §3.1.1, eq. 7. With M = 1 the weight part reduces to the single-scale binary-weight scheme the authors cite as BWN 1711.11294 §3.1.2. The paper reports, for ResNet-18 on ImageNet, 65.0% Top-1 at M = N = 5, against 69.3% full precision and 51.2% for XNOR-Net 1711.11294 tab. 3. With binary weights and full-precision activations, M = 5 gives 68.3% 1711.11294 tab. 1. Smaller grids lose more: M = N = 1 gives 42.7% 1711.11294 tab. 2. The BNN row in that comparison is the authors' own run, not the published figure 1711.11294 §4.1.3. The paper reports no timings or energy measurements, so the speed gain from the M×N convolutions is not measured 1711.11294 §5.2. Card: Towards Accurate Binary Convolutional Neural Network.

Int8 output embeddings (TESSERA)

A pixel-wise self-supervised model quantizes its 128-d output embedding to 8-bit integers, using quantization-aware training. The paper reports about a fourfold storage reduction with negligible loss 2506.20380 §3.2, §5. Its one ablation of this step, on Austrian Crop validation F1, gives 77.3 with quantization and 77.9 without 2506.20380 tab. 3. The released int8 path in the authors' repository uses one symmetric scale per vector, and that scale must be stored to dequantise. The pretraining code is not in the repository, so the quantization used during training cannot be checked against code. Card: TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis.

Fully quantized training on microcontrollers (no float copy)

The on-device paper stores weights, feature maps and errors as 8-bit tensors, with one scale and zero point per tensor, and uses the inference scheme for the forward pass, the backward pass and the update 2407.10734 §3.1. The text describes the stored weights as 8-bit and does not describe a separate float copy. The update is applied to the dequantized values. The scale and zero point are then re-fit from the new minimum and maximum over 255 steps, so the range follows the weights 2407.10734 §3.1, eq. 5–7. The paper accumulates gradients over the minibatch rather than storing every sample, and uses no momentum or adaptive optimizer, to save memory 2407.10734 §3.1.

The same paper also prunes the backward pass per sample. It keeps only the top-k error channels by L1 norm, with the keep rate set by the loss 2407.10734 §3.2, eq. 9. This part is separate from quantization, and its rate formula does not match the text; see Contradictions.

On seven transfer-learning datasets, the uint8 configuration matches or exceeds the GPU baseline on the simpler ones and falls below it on flowers, cifar100 and speech. The mixed configuration, with a float32 classification head, matches float 2407.10734 §4.1, fig. 4. Those three datasets have 102, 100 and 36 classes 2407.10734 tab. 1. In full training on four MNIST-family datasets, float32 is highest and the quantized configurations are close behind, with a larger gap than in transfer learning 2407.10734 §4.4, fig. 7. Against MCUNet-5FPS on CIFAR-10, on the IMXRT2062 (Cortex-M7) board, the paper reports 34.8% less memory and 49.0% lower latency per sample. It does not say whether the memory figure is ROM or RAM 2407.10734 §4.5, fig. 9. The 6.64x average backprop speed-up is for the mixed configuration at lambda_min 0.1. That setting was unstable for uint8 2407.10734 §4.3, fig. 6. All figures are author-reported. Card: On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollers.

Survey-tabulated QAT results (secondhand)

The survey copies QAT rows from the works it cites. Most rows have no float baseline 2112.06126 tab. 4. The rows relevant here are:

2112.06126 tab. 4. In prose, the survey says the ternary ResNet18 row loses "only" about 4 points against float 2112.06126 §6.2, and that 4-bit AlexNet and ResNet50 marginally beat float 2112.06126 §5. The survey does not show those baselines in its table. These numbers are copied from the original papers and should be checked there before use. Card: Neural Network Quantization for Efficient Inference: A Survey.

When to use which

Only what the sources' measurements support. The two whitepapers measure accuracy on ImageNet CNNs and related tasks, and the 2018 paper also measures latency on a Pixel 2. The LSQ+ numbers are author-reported on ImageNet models. The spiking tutorial has no numbers of its own. None says anything about other hardware or domains beyond what the cards record.

Implementation sketch

Pseudocode in our own words. The fake quantizer composes eqs. 1 to 3, not eq. 13 as printed; see Contradictions.

# Fake quantizer: quantize then dequantize, 256 levels at 8 bits  [eq. 1-3]
def fake_quant(x, delta, z, n_levels=256):
    x_int = round(x / delta) + z
    x_q = clamp(0, n_levels - 1, x_int)
    return (x_q - z) * delta

# Straight-through backward: pass the gradient inside the range  [eq. 14-15]
def fake_quant_backward(grad_out, x, x_min, x_max):
    inside = (x >= x_min) & (x <= x_max)
    return grad_out * inside

# One QAT step; the float copy holds the update  [§3.2, eq. 16-17]
def qat_step(batch, w_float, lr):
    w_q = fake_quant(w_float, delta_w, z_w)     # forward uses rounded weights
    y = forward(batch.x, w_q, quantize_acts=True)
    loss = criterion(y, batch.labels)
    grad = backward_with_ste(loss)              # STE at every fake quant
    return w_float - lr * grad                  # update the float copy only

# Conv + batch norm with correction and freezing  [eq. 22-31]
def bn_conv_qat(x, w, gamma, beta, mu_ema, sigma_ema, step, freeze_at):
    mu_b, sigma_b = batch_stats(conv(x, w))
    c = sigma_b / sigma_ema                     # correction factor  [eq. 22]
    w_fold = c * gamma * w / sigma_b            # folded, long-term scale  [eq. 23]
    y = conv(fake_quant(w_fold), x)
    bias = beta - gamma * mu_b / sigma_b
    if step < freeze_at:
        y = y / c                               # same output as batch norm  [eq. 25]
        corr = 0
    else:
        # moving statistics frozen, as at inference  [eq. 29-31]
        corr = gamma * (mu_b / sigma_b - mu_ema / sigma_ema)
    return y + bias + corr

Notes that apply across variants:

• The 2018 paper's learning rate for fine-tuning is 1e-5 with SGD. The schedule is not stated 1806.08342 §3.2.3.
• The example snippet uses quant_delay=2000000. The value used in experiments is not stated 1806.08342 §3.2.
• The 2018 paper does not state how the scale and zero-point are computed from the observed range, the rounding mode for activations, or the moving-average decay for activation ranges 1806.08342 §2.1, §2.5.
• For SIMD kernels, signed symmetric weights are clamped to ±127 at 8 bits, dropping the most negative level 1806.08342 §2.2, eq. 10. Use the same range in training and at deployment.
• How the two batch-norm bias terms combine in phase 2 is implicit in the 2018 paper. The sum above reproduces the inference bias 1806.08342 §3.2.2, eq. 21, fig. 9; this is our reading.
• Before QAT, CLE rescales channel pairs so the float output is unchanged, which spreads the weight ranges evenly 2106.08295 §3.2, eq. 22. Run it on the float model, and only inside residual blocks, since skip connections break the equivariance across blocks 2106.08295 §3.2.
• The 2021 paper's QAT runs use Adam 2106.08295 §4.5. With SGD-type optimizers, the rate for scales and zero-points should be lower than for weights; the factor is not stated 2106.08295 §4.4.
• With a learned offset (LSQ+), the zero-gradient rule for β means it stops moving once it is below the lowest value, since every value is then inside the range 2004.09576 §3.1 eq. 4. Check the learned offsets after training. The paper's learned values stayed above the Swish minimum 2004.09576 §5.1.
• ABC-Net's straight-through mask in eq. 10 is centred on R = v + 0.5, while its forward threshold sits at R = 0.5 − v. The card reads this as an inconsistency in the paper, so decide the mask deliberately and check the gradient 1711.11294 eq. 8–10. Its shift formula is undefined at M = 1, where the supplement uses zero shift 1711.11294 supp. Parameter settings.
• PACT's clip level alpha has an L2 penalty, but the paper names the penalty without writing its exact form, and it does not state the gradient with respect to the input 1805.06085 §4, app. A.2. An implementation has to choose both. Its initial alpha is given only for one network, 10 for ResNet-20 on CIFAR-10 1805.06085 §4, fig. 2.
• The fully quantized variant on microcontrollers stores 8-bit weights, takes the update in floating point, then re-fits the scale and zero point from the new minimum and maximum 2407.10734 §3.1, eq. 5–7. The paper does not say how a zero range is handled when the new minimum equals the maximum 2407.10734 eq. 6, or whether the batch gradient is summed or averaged before the update 2407.10734 §3.1.

Contradictions

• MobileNetV2 at 4-bit weights after QAT. The 2018 paper gives 0.62 against a float of 0.719, a gap of about 10 points, with symmetric per-channel weights and 8-bit activations 1806.08342 tab. 5. The 2021 paper gives 70.48 at W4A8 per-channel against a float of 71.72, a gap of about 1.2 points, after its standard pipeline 2106.08295 tab. 10. The 2018 paper fine-tunes TF-Slim checkpoints with BN freezing 1806.08342 §3.2.3. The 2021 paper's QAT uses Adam and 20 epochs on ImageNet models 2106.08295 §4.5. Neither card states the other paper's setup, so the gap is not explained here.
• Post-training 4-bit weights on MobileNetV2. The 2018 paper gives 0.001 with symmetric per-channel weights 1806.08342 tab. 5. Its weights use the actual min and max 1806.08342 §2.5. The 2021 paper gives 69.79 at W4A8 per-channel for its PTQ pipeline 2106.08295 tab. 6. That pipeline adds CLE, MSE ranges, and AdaRound or bias correction 2106.08295 §3.5. The methods differ, so the two results are not in conflict on the same method.
• Batch-norm schemes. The 2018 paper finds correction with freezing steadier than naive folding, batch renormalization, or moving averages without freezing, on MobileNet-v1 1806.08342 §4, fig. 14. The 2021 paper finds static folding on par with or better than the double-forward method, and better than it in all four columns of its table, on MobileNetV2 and ResNet-18 2106.08295 §4.2, tab. 7. Neither paper tests the other's scheme, so both results stand as stated.
• Freeze step for MobileNet-v2. About 300000 steps in 1806.08342 §3.2.2, and 400000 in 1806.08342 §4 and the fig. 15 caption 1806.08342 fig. 15.
• Eq. 13 sign. The 2018 paper writes the fake-quant with a minus sign on the zero-point inside the clamp. Its eqs. 1 to 3 use a plus sign, and the two agree only when z=0 1806.08342 §2.4, eq. 1-3, eq. 13. The sketch uses the composed form.
• Per-layer weights, weight-only post-training. The 2018 text says asymmetric per-layer is best. Its table 2 shows asymmetric per-channel as best, with per-layer collapsing on MobileNets 1806.08342 §3.1.3, tab. 2. This bears on the Variants above, which use the table values.
• Within the 2021 paper, text and table disagree. Its text says W8A8 PTQ is within 0.7 % of float 2106.08295 §3.6. Its table shows MobileNetV2 per-tensor at 70.99, a 0.73 drop, and EfficientDet-D1 at 38.29, a 1.79 drop 2106.08295 tab. 6. Its text says W4A4 QAT stays within 1 % on networks without depth-wise convolutions 2106.08295 §4.5, but its table shows ResNet-18 per-tensor at 68.32, 1.36 below float 2106.08295 tab. 10.
• Within the 2021 paper, table labels. The MobileNetV2 values 12.99 (PTQ) and 70.13 (QAT) appear as "W4A8 with MSE weight init" in tab. 8 and as "W4A8 with CLE" in tab. 9. The source of tab. 9 has a TODO on its PTQ numbers. Which label is right is not determinable from the paper 2106.08295 tab. 8, tab. 9.
• Within the 2021 paper, learned scales. Its §4.4 says that updating the scale every epoch, rather than learning it, gives better results at low bit-width. Read against the learned-scale setting above, this wording is ambiguous, so no claim is made here 2106.08295 §4.4.
• Within ConveRT, model size. With 16M embedding and 13M network parameters, the stated 8-bit and 16-bit widths give about 42 MB, but tab. 2 lists 59 MB for the quantized ConveRT 1911.03688 tab. 2. The same convention reproduces the 118 MB figure for PolyAI-dual in that table, so the gap is not explained by the paper 1911.03688 tab. 2.
• Within ABC-Net, the full-precision gap. The text gives a Top-1 gap of 0.9 points at M = 5 with full-precision activations 1711.11294 §4.1.1. Table 1 gives 69.3 − 68.3 = 1.0 1711.11294 tab. 1.
• Within the HWA paper, network name. §2.3 says WideResNet-32 on CIFAR-100 reaches iso-accuracy, but tab. 3 lists WideResNet-16 at 100.0 2302.08469 §2.3, tab. 3. The variant above uses the table.
• Within the survey, the zero-point sign. Its eq. 5 subtracts the zero point and its eq. 7 adds it 2112.06126 §3.1, §3.2. An implementation that follows one equation and not the other is off by 2Z on every value. The survey does not say which convention a deployment framework uses.
• Within 2407.10734, the CIFAR-10 accuracy. Section 4.5 gives 89.5% for MCUNet-5FPS and 85.9% for MbedNet 2407.10734 §4.5. Table 4 gives 89.5 for the authors' uint8 method on the MCUNet backbone 2407.10734 tab. 4. The same number appears under two labels, and the paper does not say which is right.
• Within 2407.10734, the sparse-update rate. The text says the rate falls toward lambda_min as the loss falls 2407.10734 §3.2. As written, eq. 9 uses the absolute gap between the loss and the largest loss seen so far. That gives the lowest rate when the loss equals that maximum, which is typical early in training, and the highest rate near zero loss 2407.10734 eq. 9. The paper does not resolve this.

Papers

• Quantizing deep convolutional networks for efficient inference: A whitepaper: defines the uniform affine and symmetric quantizers, post-training and QAT procedures, batch-norm correction with freezing, and measured accuracy and latency on CNNs.
• A White Paper on Neural Network Quantization: defines PTQ and QAT pipelines for uniform affine quantization, with CLE, bias correction, AdaRound, learned scales, and its own batch-norm comparison and results on ImageNet, DeepLabV3, EfficientDet and BERT.
• LSQ+: Improving low-bit quantization through learnable offsets and better initialization: adds a learned offset to learned-scale quantization for activations, with weight and activation initialisation schemes and results at W2A2 to W4A4 on ImageNet.
• Training Spiking Neural Networks Using Lessons From Deep Learning: a tutorial on training spiking networks with surrogate gradients. Its quantization part is brief, with listings and no accuracy numbers.
• ConveRT: Efficient and Accurate Conversational Representations from Transformers: a dual encoder for reply selection, with mixed-precision QAT that keeps embeddings at 8 bits and other weights at 16 bits.
• Analog in-memory hardware-aware training study: retrains eleven networks with a simulated analog crossbar in the forward pass, reporting accuracy at drift times up to one year.
• PACT: Parameterized Clipping Activation for Quantized Neural Networks: replaces the ReLU with a clipped activation whose clip level is learned per layer, with a straight-through gradient, and reports 2 to 5 bit results on CIFAR10, SVHN, AlexNet, ResNet18 and ResNet50 trained from scratch.
• Towards Accurate Binary Convolutional Neural Network: approximates weights with several binary bases and activations with several shifted binary thresholds, reporting ImageNet results for ResNet-18, ResNet-34 and ResNet-50.
• TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis: a self-supervised remote-sensing embedding model whose 128-d output is quantized to int8 with quantization-aware training.
• Neural Network Quantization for Efficient Inference: A Survey: a survey of number formats, QAT and PTQ, and binary, ternary and mixed-precision schemes, with comparison tables copied from cited works and no new method.
• On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollers: fully quantized training on three Cortex-M boards, with 8-bit weights and errors, a scale re-fit after each update, and partial error-channel updates.