--- kind: concept slug: quantization-aware-training title: Quantization-aware training summary: >- Fine-tuning with simulated rounding of weights and activations to low-bit integers, so the network learns to tolerate it. A straight-through gradient makes this trainable, usually with a float master copy. Needed when post-training quantization loses too much accuracy, e.g. per-tensor grids on MobileNets or 4-bit weights. Batch-norm handling differs between sources. Costs a training run. tags: [quantization, efficient-inference, training, edge-deployment, low-bit, spiking-neural-networks, binary-networks, embeddings] papers: ["1806.08342", "2106.08295", "2004.09576", "2109.12894", "1911.03688", "2302.08469", "1805.06085", "1711.11294", "2506.20380", "2112.06126", "2407.10734"] related: [weight-quantization, batchnorm-folding, post-training-quantization, asymmetric-quantization, hardware-aware-training, binary-neural-network, on-device-training, mixed-precision-quantization] status: draft updated_at: 2026-10-25 --- ## Overview Quantization-aware training (QAT) trains a network with the same rounding it will see at inference. Each weight and activation tensor is passed through a quantize-then-dequantize operation in the forward pass, so the loss reflects the integer grid. The backward pass treats that rounding as the identity inside the representable range and as zero outside it. This is the straight-through estimator [arxiv:1806.08342 §2.4] [arxiv:2106.08295 §4.1]. A float copy of each weight is kept and updated, so small gradient steps are not lost to rounding [arxiv:1806.08342 §3.2]. The fully quantized variant on microcontrollers, below, does not describe such a copy [arxiv:2407.10734 §3.1]. Learned quantizers go further and train the grid parameters themselves, including an offset for skewed activations [arxiv:2004.09576 §3.1]; see the learned-quantizer variants below. The invariant is that the training graph mirrors the inference graph. Every fusion made at deployment, and every rescaling of inputs to an add or concat, must also be modeled during training [arxiv:1806.08342 §3.2.1]. Most sources start QAT from a float checkpoint and fine-tune it. The 2018 paper recommends this and reports it beating training from scratch [arxiv:1806.08342 §3.2, §4]. The 2021 white paper also starts from a pre-trained float model, and runs its PTQ steps before QAT [arxiv:2106.08295 §4.3, §4.4]. The binary-network paper does the same, initialising from a full-precision model [arxiv:1711.11294 §3.3]. The on-device paper pretrains in float and quantizes before deployment, then trains its last layers on the device [arxiv:2407.10734 §4.1]. The PACT paper is the exception in this set: its networks are trained from scratch [arxiv:1805.06085 §5, app. B]. The alternative is post-training quantization, which needs only a few calibration batches [arxiv:1806.08342 §3.1.2]. The papers' measurements show where that alternative stops being enough. ## Variants The variants come from eleven sources: the 2018 whitepaper on CNN quantization [arxiv:1806.08342], the 2021 white paper on neural network quantization [arxiv:2106.08295], the LSQ+ paper on learned low-bit quantization [arxiv:2004.09576], a tutorial on spiking networks [arxiv:2109.12894], a dual-encoder paper that stores embeddings at 8 bits [arxiv:1911.03688], a study of training for analog in-memory chips [arxiv:2302.08469], the PACT paper on learned activation clipping [arxiv:1805.06085], a binary-network paper that uses several weight and activation bases [arxiv:1711.11294], a remote-sensing embedding model that quantizes its output to 8 bits [arxiv:2506.20380], a survey of inference quantization [arxiv:2112.06126], and a study of fully quantized training on microcontrollers [arxiv:2407.10734]. Each variant names its source, and the settings are the ones that source tested. Where the sources disagree, see Contradictions. ### Per-layer asymmetric QAT (8 bits) One scale and zero-point per tensor, with a uniform affine quantizer [arxiv:1806.08342 §2.1]. This is the layout that post-training quantization breaks on MobileNets. With QAT it recovers to 0.70 top-1 on MobileNet-v1 against 0.709 for float, while the same layout post-training gives 0.001 [arxiv:1806.08342 tab. 4]. On Nasnet-Mobile it reaches 0.73 against 0.74 [arxiv:1806.08342 tab. 4]. Use it when the target kernel only supports per-tensor grids. Card: [Quantizing deep convolutional networks for efficient inference: A whitepaper](/papers/1806.08342). ### Symmetric per-channel QAT (8 bits) One symmetric grid per output channel of each weight tensor [arxiv:1806.08342 §2.2, §2.6]. Activations stay per-layer [arxiv:1806.08342 §2.6]. On MobileNet-v2 1.4 it reaches 0.745 against 0.749 for float, and on Resnet_v1_50 0.75 against 0.752 [arxiv:1806.08342 tab. 4]. For the same weights, post-training quantization gives 0.74 on MobileNet-v2 1.4 and 0.751 on Resnet_v1_50, so QAT adds little in those rows [arxiv:1806.08342 tab. 4]. The per-channel post-training numbers are already close to float for most networks, so QAT matters most for the grid-limited cases above. The 2021 paper's per-channel W8A8 QAT row, under its full pipeline, is above float for ResNet-18 (70.43 against 69.68) and MobileNetV2 (71.82 against 71.72), and below it for EfficientNet-lite (74.75 against 75.42) [arxiv:2106.08295 tab. 10]. ### Symmetric per-channel QAT with 4-bit weights Weights at 4 bits, activations at 8 bits (written W4A8 below) [arxiv:1806.08342 §3.2.4]. Post-training 4-bit weights give 0.001 top-1 on both MobileNets, while QAT gives 0.65 on MobileNet-v1 and 0.62 on MobileNet-v2, against floats of 0.709 and 0.719 [arxiv:1806.08342 tab. 5]. A gap of about 6 to 10 points remains on MobileNets. On Resnet_v1_50 the gap is 0.732 against 0.752 [arxiv:1806.08342 tab. 5]. The paper also reports that smaller networks lose the most under quantization [arxiv:1806.08342 §3.1.3]. The 2021 paper's per-channel W4A8 QAT, after its standard pipeline, is higher: 70.01 on ResNet-18 and 70.48 on MobileNetV2, against floats of 69.68 and 71.72 [arxiv:2106.08295 tab. 10]. The two results conflict; see Contradictions. ### Low-bit activations Activations at 4 bits with 8-bit weights (written W8A4 below). QAT reaches 0.64 on MobileNet-v1 and 0.58 on MobileNet-v2 at this setting [arxiv:1806.08342 tab. 6]. In every row of that table, 4-bit weights with 8-bit activations scores higher than the reverse. The paper notes that the comparison mixes granularities [arxiv:1806.08342 §3.2.4]. Treat this as a measured result for these networks, not a general rule. The 2021 paper's W4A4 per-channel QAT row reaches 68.83 on ResNet-18 and 66.89 on MobileNetV2 [arxiv:2106.08295 tab. 10]. Across its five ImageNet CNNs, W4A4 QAT with its pipeline gives 66.43 to 77.74 top-1 over both granularities [arxiv:2106.08295 tab. 10]. ### Learned activation clip level (PACT) Each layer's activations are clipped at a clip level alpha, which is a trainable parameter, and then rounded to k bits. The gradient reaches alpha through a straight-through estimator: the output's derivative with respect to alpha is 1 where the input is at or above alpha, and 0 elsewhere [arxiv:1805.06085 §4, eq. 3]. The paper uses one alpha per layer, which it found the best scope among those tried [arxiv:1805.06085 §4.2, app. A.1]. It leaves the first and last layers at full precision in its main comparisons [arxiv:1805.06085 §4.2]. Its weights are DoReFa-quantized at the same bit-width, so the comparison isolates the activation side. With these weights, PACT activations score above DoReFa activations in every cell of the paper's table, for example 0.765 against 0.714 for ResNet-50 at 4 bits [arxiv:1805.06085 tab. 1]. At 4 bits the paper reports results within about 1 point of full precision on the five networks tested. At 2 bits the gap is larger: 0.644 against 0.702 on ResNet-18, and 0.722 against 0.769 on ResNet-50 [arxiv:1805.06085 tab. 1]. These are author-reported results from networks trained from scratch, so the numbers are not a fine-tuning benchmark. Card: [PACT: Parameterized Clipping Activation for Quantized Neural Networks](/papers/1805.06085). ### Batch-norm handling during QAT The 2018 paper compared four schemes on MobileNet-v1 [arxiv:1806.08342 §4, fig. 14]: - naive folding with batch statistics, which makes the quantized weights change every batch; - batch renormalization; - moving averages throughout training without freezing; - batch-norm correction with freezing after a delay. Only the last gives higher and steadier evaluation accuracy in the paper [arxiv:1806.08342 §4, fig. 14]. The correction scales the folded weights toward the long-term standard deviation, and freezing then switches to the moving statistics that inference uses [arxiv:1806.08342 §3.2.2]. The freeze delay was about 200000 steps for MobileNet-v1 and 300000 to 400000 for MobileNet-v2 [arxiv:1806.08342 §3.2.2, §4, fig. 14, fig. 15]. The paper's text and captions disagree on the MobileNet-v2 value; see Contradictions. The 2021 paper did not test correction with freezing. It compared other options at W4A8 and W4A4 on MobileNetV2 and ResNet-18 [arxiv:2106.08295 tab. 7]. For per-tensor quantization it uses static folding, which it reports as on par with or better than the double-forward method, and better than that method in all four columns of its table [arxiv:2106.08295 §4.2, tab. 7]. The source labels that method "Krishnamoorthi" in tab. 7 and attributes it to Jacob et al. (2018) in the text; the card does not settle which is right. For per-channel quantization, BN can stay unfolded during training, with its scale merged into the per-channel weight scales afterwards [arxiv:2106.08295 §4.2, eq. 41]. That scored highest in three of the four columns; the exception is MobileNetV2 at W4A8, 70.48 against 70.52 for static per-channel folding [arxiv:2106.08295 tab. 7]. ### Stochastic rounding during training Adding uniform noise before rounding during training gave worse results than deterministic rounding [arxiv:1806.08342 §4, fig. 12]. The paper studies it and does not recommend it. Deterministic rounding is the default for the other variants. ### Weight averaging Exponential moving averages of the weights fell behind the live weights late in QAT [arxiv:1806.08342 §4, fig. 15]. Evaluating both, as a check, is a reasonable step. The decay rate is not stated. ### Standard pipeline with PTQ initialisation The 2021 paper's standard QAT pipeline [arxiv:2106.08295 §4.4, fig. 12]: 1. Start from a pre-trained float model. Apply cross-layer equalization (CLE) when weights are imbalanced, as in MobileNets. It is optional otherwise, or with per-channel quantization. 2. Add quantizers: symmetric weights, asymmetric activations, and per-channel weights if the target supports them. Make the batch-norm simulation correct; see the BN variant above. 3. Set all ranges with layer-wise MSE. Min-max can be better for per-channel quantization [arxiv:2106.08295 §3.1, §4.4]. 4. Make the quantization parameters learnable. Better range initialisation helps early epochs, but the advantage shrinks with longer training. On ResNet-18 at W4A8, QAT reaches 69.61 from min-max initialisation and 69.76 from MSE [arxiv:2106.08295 §4.3, tab. 8]. For MobileNetV2, the paper says CLE is needed for QAT to train at all. Its W4A8 baseline ends at 0.10 top-1 after QAT, and adding CLE gives 70.13 [arxiv:2106.08295 §4.3, tab. 9]. The table's labels conflict with tab. 8; see Contradictions. The paper's QAT runs use Adam, with 20 epochs for ImageNet models [arxiv:2106.08295 §4.5]. Its text also says that SGD-type optimizers need a lower learning rate for the quantization parameters than for the weights [arxiv:2106.08295 §4.4]. ### Learned scale and zero-point The 2021 paper makes the scale a learnable parameter. Its gradient is −x/s + round(x/s) inside the grid, and the grid limit it saturates to outside it [arxiv:2106.08295 §4.1, eq. 38]. The zero-point is kept real and rounded in the forward pass, with the straight-through rule applied to that rounding [arxiv:2106.08295 §4.1, eq. 39–40]. The paper's zero-point gradient is zero inside the grid [arxiv:2106.08295 eq. 40], which is a straight-through result rather than a measurement. ### Learned scale and offset, asymmetric activations (LSQ+) The LSQ+ paper learns a scale s and an offset β for activations, so the grid can slide to the actual lowest value. This targets skewed activations such as Swish, whose negative part is bounded near −0.278 while the positive part is unbounded [arxiv:2004.09576 §3.1]. The offset is subtracted before scaling and added back after rounding, and both are trained with straight-through gradients [arxiv:2004.09576 §3.1 eq. 2–4]. Weights stay symmetric and signed [arxiv:2004.09576 §3.2.1]. The paper states that the offset adds a per-output bias at inference, so it has no runtime cost beyond that bias [arxiv:2004.09576 §3.1 eq. 5]. Author-reported on ImageNet, with pretrained float starts. EfficientNet-B0 with the asymmetric unsigned configuration reaches 73.8% at W4A4, against 71.9% for symmetric unsigned LSQ and 76.1% full precision [arxiv:2004.09576 tab. 2]. At W2A2 the same configuration reaches 48.7% against 43.5% for LSQ [arxiv:2004.09576 tab. 2]. On MixNet-S at W4A4 the gain is 71.7% against 70.4% [arxiv:2004.09576 tab. 3]. On ResNet18, the asymmetric and symmetric configurations match at W4A4 [arxiv:2004.09576 tab. 4]. The offset result on EfficientNet-B0 at W4A4 is 71.9% with β fixed at zero, 72.5% with β fixed at the Swish minimum, and 73.8% with β learned [arxiv:2004.09576 tab. 6]. Initialisation is a second part of the method. The weight scale comes from the layer's mean and standard deviation, taking ±3σ [arxiv:2004.09576 §3.2.1]. The activation scale and offset are fitted by minimising squared error on a few batches [arxiv:2004.09576 §3.2.2 eq. 8]. Over five runs at W4A4 on EfficientNet-B0, the mean is 71.3% ± 2.2 with min-max initialisation, 72.0% ± 1.6 with LSQ's, and 73.0% ± 0.9 with the proposed one [arxiv:2004.09576 tab. 5]. The paper's weight formula divides by 2^(b−1), although the signed range has 2^b − 1 levels, and it does not say whether this is intended [arxiv:2004.09576 §3.2.1]. The optimiser, learning rate, epochs and batch size are not stated [arxiv:2004.09576 §4], and the paper has no official code. ### Quantized spiking networks A tutorial on spiking networks reports that such networks are robust to quantization-aware training. Its listings quantize weights to 8 bits and membrane states to 4 bits [arxiv:2109.12894 §4.3.2]. It gives no accuracy numbers for this. It suggests clipping negative membrane values to zero, but only from early experiments, and only where inhibitory spikes are not used [arxiv:2109.12894 §4.3.2 box]. Card: [Training Spiking Neural Networks Using Lessons From Deep Learning](/papers/2109.12894). ### Mixed-precision QAT with 8-bit embeddings (ConveRT) A dual encoder for reply selection keeps its embedding table on an 8-bit grid and its other network weights at 16 bits [arxiv:1911.03688 §2.1, tab. 2]. Training uses the mixed-precision scheme of Micikevicius et al.: FP32 shadow copies, FP16 casts for the forward pass, and layer norm, L2 norm and attention softmax kept in FP32 [arxiv:1911.03688 §2.1]. The loss is scaled by 128 and the updates are divided by the same factor [arxiv:1911.03688 §2.1]. The embedding range is refreshed during training, with a margin of 10% of the range or 0.01, whichever is larger [arxiv:1911.03688 §2.1]. The paper says the quantization allowed a doubled batch size, and so more in-batch negatives [arxiv:1911.03688 §2.1]. The refresh cadence is not stated, and the paper gives no separate result for quantization alone: its ablation in tab. 4 varies architecture choices only [arxiv:1911.03688 tab. 4]. The model-size column in tab. 2 is computed with the same bit widths, but its ConveRT figure does not follow from them; see Contradictions. Card: [ConveRT: Efficient and Accurate Conversational Representations from Transformers](/papers/1911.03688). ### Noise- and converter-aware training for analog crossbars (HWA) The forward pass simulates an analog in-memory crossbar, and the backward pass uses floating-point gradients [arxiv:2302.08469 Methods: AIMC hardware-aware DNN training]. The simulated chip includes 8-bit input and output converters, programming error and drift, read and output noise, and IR-drop [arxiv:2302.08469 §2.1, eq. 4–6]. The input range, the per-column output scales and the input clipping bound are learned as well [arxiv:2302.08469 Methods: Learning the input range]. The weight noise is redrawn once per mini-batch [arxiv:2302.08469 Methods: Re-training with weight noise injection]. This differs from the rounding-only variants above: the main effect studied is analog noise, and the rounding is one part of the chip model. Without retraining, direct mapping misses the iso-accuracy target for almost every one of the eleven networks. Only LSTM PTB reaches it, at 99.0 normalized at one hour [arxiv:2302.08469 §2.2, tab. 2]. With HWA training, every network reaches at least 96% normalized accuracy at one hour [arxiv:2302.08469 §2.3, tab. 3], and five reach iso-accuracy: WideResNet-16, BERT, the speech model, the LSTM and RNN-T [arxiv:2302.08469 §2.3, tab. 3]. The worst raw-accuracy gap falls from 21.81% to 2.65% across models [arxiv:2302.08469 §4]. The sensitivity study ranks output-side noise and ADC or DAC resolution as the largest contributors [arxiv:2302.08469 §2.4, fig. 4]. The paper reports that DAC resolution can drop from 8 to 6 bits with little accuracy impact [arxiv:2302.08469 §2.5]. Convolutional networks are the most sensitive, and ResNet-18 degrades to 99% at 1.2 times the standard PCM noise [arxiv:2302.08469 §2.4, §2.5, fig. 5]. These are author-reported results on the paper's own chip model, and the paper states that its hyperparameters were tuned for drift averaged over 1 s to 1 y [arxiv:2302.08469 tab. 3 caption]. Card: [Analog in-memory hardware-aware training study](/papers/2302.08469). ### Multiple binary bases with shifted binary activations (ABC-Net) Each layer's real weights are written as a linear combination of M binary bases. Each basis is the sign of the centred weights, shifted by a multiple of the weight spread. The scale factors come from least squares on the fixed bases [arxiv:1711.11294 §3.1.1, eq. 1–3]. The activations are written as a combination of N binary activations, each with its own trainable threshold shift and scale [arxiv:1711.11294 §3.2, eq. 9, eq. 11]. A convolution then becomes M×N bitwise convolutions with scale factors [arxiv:1711.11294 eq. 12]. Training keeps real-valued weights and passes gradients through the sign function with the straight-through estimator [arxiv:1711.11294 §3.1.1, eq. 7]. With M = 1 the weight part reduces to the single-scale binary-weight scheme the authors cite as BWN [arxiv:1711.11294 §3.1.2]. The paper reports, for ResNet-18 on ImageNet, 65.0% Top-1 at M = N = 5, against 69.3% full precision and 51.2% for XNOR-Net [arxiv:1711.11294 tab. 3]. With binary weights and full-precision activations, M = 5 gives 68.3% [arxiv:1711.11294 tab. 1]. Smaller grids lose more: M = N = 1 gives 42.7% [arxiv:1711.11294 tab. 2]. The BNN row in that comparison is the authors' own run, not the published figure [arxiv:1711.11294 §4.1.3]. The paper reports no timings or energy measurements, so the speed gain from the M×N convolutions is not measured [arxiv:1711.11294 §5.2]. Card: [Towards Accurate Binary Convolutional Neural Network](/papers/1711.11294). ### Int8 output embeddings (TESSERA) A pixel-wise self-supervised model quantizes its 128-d output embedding to 8-bit integers, using quantization-aware training. The paper reports about a fourfold storage reduction with negligible loss [arxiv:2506.20380 §3.2, §5]. Its one ablation of this step, on Austrian Crop validation F1, gives 77.3 with quantization and 77.9 without [arxiv:2506.20380 tab. 3]. The released int8 path in the authors' repository uses one symmetric scale per vector, and that scale must be stored to dequantise. The pretraining code is not in the repository, so the quantization used during training cannot be checked against code. Card: [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](/papers/2506.20380). ### Fully quantized training on microcontrollers (no float copy) The on-device paper stores weights, feature maps and errors as 8-bit tensors, with one scale and zero point per tensor, and uses the inference scheme for the forward pass, the backward pass and the update [arxiv:2407.10734 §3.1]. The text describes the stored weights as 8-bit and does not describe a separate float copy. The update is applied to the dequantized values. The scale and zero point are then re-fit from the new minimum and maximum over 255 steps, so the range follows the weights [arxiv:2407.10734 §3.1, eq. 5–7]. The paper accumulates gradients over the minibatch rather than storing every sample, and uses no momentum or adaptive optimizer, to save memory [arxiv:2407.10734 §3.1]. The same paper also prunes the backward pass per sample. It keeps only the top-k error channels by L1 norm, with the keep rate set by the loss [arxiv:2407.10734 §3.2, eq. 9]. This part is separate from quantization, and its rate formula does not match the text; see Contradictions. On seven transfer-learning datasets, the uint8 configuration matches or exceeds the GPU baseline on the simpler ones and falls below it on flowers, cifar100 and speech. The mixed configuration, with a float32 classification head, matches float [arxiv:2407.10734 §4.1, fig. 4]. Those three datasets have 102, 100 and 36 classes [arxiv:2407.10734 tab. 1]. In full training on four MNIST-family datasets, float32 is highest and the quantized configurations are close behind, with a larger gap than in transfer learning [arxiv:2407.10734 §4.4, fig. 7]. Against MCUNet-5FPS on CIFAR-10, on the IMXRT2062 (Cortex-M7) board, the paper reports 34.8% less memory and 49.0% lower latency per sample. It does not say whether the memory figure is ROM or RAM [arxiv:2407.10734 §4.5, fig. 9]. The 6.64x average backprop speed-up is for the mixed configuration at lambda_min 0.1. That setting was unstable for uint8 [arxiv:2407.10734 §4.3, fig. 6]. All figures are author-reported. Card: [On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollers](/papers/2407.10734). ### Survey-tabulated QAT results (secondhand) The survey copies QAT rows from the works it cites. Most rows have no float baseline [arxiv:2112.06126 tab. 4]. The rows relevant here are: | Method | Model | w/a | Top-1 (%) | |---|---|---|---| | Jacob et al. | ResNet50 | 8/8 | 74.9 | | Zhuang et al. | ResNet50 | 4/4 | 75.7 | | Zhuang et al. | AlexNet | 4/4 | 58 | | BWN | ResNet18 | 1/float32 | 60.8 | | TWN | ResNet18 | 2/float32 | 61.8 | | TTQ | ResNet18 | 2/float32 | 66.6 | [arxiv:2112.06126 tab. 4]. In prose, the survey says the ternary ResNet18 row loses "only" about 4 points against float [arxiv:2112.06126 §6.2], and that 4-bit AlexNet and ResNet50 marginally beat float [arxiv:2112.06126 §5]. The survey does not show those baselines in its table. These numbers are copied from the original papers and should be checked there before use. Card: [Neural Network Quantization for Efficient Inference: A Survey](/papers/2112.06126). ## When to use which Only what the sources' measurements support. The two whitepapers measure accuracy on ImageNet CNNs and related tasks, and the 2018 paper also measures latency on a Pixel 2. The LSQ+ numbers are author-reported on ImageNet models. The spiking tutorial has no numbers of its own. None says anything about other hardware or domains beyond what the cards record. | Situation | Use | Why | |---|---|---| | 8-bit, target only supports per-tensor grids | Per-layer asymmetric QAT | Post-training per-layer collapses MobileNets to 0.001, while QAT reaches 0.70 vs 0.709 float [arxiv:1806.08342 tab. 4] | | 8-bit, target supports per-channel weights | Post-training first, then check asymmetric per-channel | Asymmetric per-channel post-training is within about 1 point of float for most listed networks, and about 2 points below on MobileNet-v2 1.0 [arxiv:1806.08342 tab. 2, tab. 3]. Symmetric per-channel can lose much more on MobileNet-v1 (0.591 vs 0.703) [arxiv:1806.08342 tab. 3] | | Depth-wise separable network (MobileNet) with per-tensor weights | Run CLE before QAT | Without it, MobileNetV2 at W4A8 stays at 0.10 after QAT; with it, 70.13 [arxiv:2106.08295 §4.3, tab. 9] | | 4-bit weights are required | Symmetric per-channel QAT, ideally with the 2021 pipeline | Post-training 4-bit fails on MobileNets (0.001), and QAT gives 0.62 to 0.65 [arxiv:1806.08342 tab. 5]. The 2021 pipeline gives 70.01 and 70.48 at W4A8 [arxiv:2106.08295 tab. 10]. The two disagree; see Contradictions | | 4-bit activations are required | Measure before committing | The 2018 paper gives 0.4 to 0.74 at W8A4 across its networks [arxiv:1806.08342 tab. 6]. The 2021 pipeline gives 66.43 to 77.74 at W4A4 across ImageNet CNNs [arxiv:2106.08295 tab. 10]. The setups differ, so the numbers are not directly comparable | | Batch norm sits after the convolutions, per-tensor weights | Correction plus freezing, or static folding | Correction plus freezing was the only steady scheme in the 2018 comparison [arxiv:1806.08342 §4, fig. 14]. Static folding was on par or better than the double-forward method in the 2021 runs [arxiv:2106.08295 §4.2, tab. 7] | | Batch norm, per-channel weights | Keep BN unfolded, merge its scale afterwards | Highest in three of four columns of the 2021 comparison [arxiv:2106.08295 §4.2, tab. 7] | | No training data or no training budget | Post-training quantization | Calibration takes about 100 mini-batches [arxiv:1806.08342 §3.1.2]. The 2018 paper does not state its QAT step counts. The 2021 paper used 20 epochs for ImageNet models [arxiv:2106.08295 §4.5] | | Small networks such as MobileNets | Expect the largest gaps | Small models lose the most accuracy under quantization [arxiv:1806.08342 §3.1.3, tab. 5] | | Skewed activations (e.g. Swish) at 2 to 4 bits, and a learned quantizer is acceptable | Asymmetric learned offset (LSQ+) | On EfficientNet-B0 it beats symmetric unsigned LSQ at W4A4 (73.8% vs 71.9%) and W2A2 (48.7% vs 43.5%) [arxiv:2004.09576 tab. 2]. These are the paper's own runs with its grid search | | Low-bit QAT where run-to-run variance matters | Initialise scales from weight and activation statistics | At W4A4 the spread falls from ±2.2 (min-max) to ±0.9 points (proposed) over five runs [arxiv:2004.09576 tab. 5] | | Spiking network, quantizing weights and membrane states | Follow the 8-bit weight and 4-bit state listings | The tutorial shows the listings but gives no accuracy numbers [arxiv:2109.12894 §4.3.2] | | Embedding table dominates model size, and the target runs mixed precision | 8-bit embeddings, 16-bit other weights, FP32 shadow copies | The paper reports no separate ablation for the quantization step, so its effect on accuracy is not measured [arxiv:1911.03688 §2.1, tab. 4] | | Low-bit activations on a ReLU network, and a full training run from scratch is affordable | Learned per-layer clip level (PACT), with DoReFa weights | Same weights, PACT activations score above DoReFa activations in every cell of the paper's table, e.g. ResNet-50 at 4 bits, 0.765 against 0.714 [arxiv:1805.06085 tab. 1]. The paper trains from scratch, and its results are not a fine-tuning benchmark | | Analog in-memory accelerator, weights programmed once and drift over time | Hardware-aware training with the chip model in the forward pass | Five of eleven networks reach iso-accuracy at one hour, against one of eleven without retraining [arxiv:2302.08469 §2.2, §2.3, tab. 2, tab. 3]. The chip model is the paper's own, so check it against your device | | Binary weights and binary activations, and the accuracy loss of plain XNOR-style binarization is too high | Several weight bases and several activation bits (ABC-Net) | M = N = 5 gives 65.0% on ResNet-18 against 51.2% for XNOR-Net [arxiv:1711.11294 tab. 3]. Each layer then costs M×N binary convolutions, and the paper does not measure the net speed-up [arxiv:1711.11294 eq. 12, §5.2] | | Binary weights, full-precision activations | ABC-Net with M = 5 | 68.3% against 69.3% full precision on ResNet-18 [arxiv:1711.11294 tab. 1] | | Frozen embeddings stored for many downstream users, where storage matters | Int8 output with QAT, plus a stored per-vector scale | About a fourfold storage cut with negligible loss, per the paper [arxiv:2506.20380 §3.2, §5]. The only ablation shows 77.3 with quantization and 77.9 without, on one dataset [arxiv:2506.20380 tab. 3] | | Training on a microcontroller, with data collected on the device and weights that must change in place | Fully quantized training with 8-bit weights and errors, and the classification head kept in float32 (the paper's mixed configuration) | On transfer tasks the mixed configuration matches float, while uint8 alone falls below the GPU baseline on flowers, cifar100 and speech, which have 102, 100 and 36 classes [arxiv:2407.10734 §4.1, fig. 4, tab. 1]. Tested on three boards, with one model family and one MCUNet comparison | | Backward pass dominates the time per sample, as in full training on the device | Partial error-channel updates with lambda_min 0.5 | The 6.64x average speed-up was measured at lambda_min 0.1 on the mixed configuration. That setting was unstable for uint8, while 0.5 stayed close to full updates [arxiv:2407.10734 §4.3, fig. 6]. The rate formula is unclear; see Contradictions | ## Implementation sketch Pseudocode in our own words. The fake quantizer composes eqs. 1 to 3, not eq. 13 as printed; see Contradictions. ```pseudocode # Fake quantizer: quantize then dequantize, 256 levels at 8 bits [eq. 1-3] def fake_quant(x, delta, z, n_levels=256): x_int = round(x / delta) + z x_q = clamp(0, n_levels - 1, x_int) return (x_q - z) * delta ``` ```pseudocode # Straight-through backward: pass the gradient inside the range [eq. 14-15] def fake_quant_backward(grad_out, x, x_min, x_max): inside = (x >= x_min) & (x <= x_max) return grad_out * inside ``` ```pseudocode # One QAT step; the float copy holds the update [§3.2, eq. 16-17] def qat_step(batch, w_float, lr): w_q = fake_quant(w_float, delta_w, z_w) # forward uses rounded weights y = forward(batch.x, w_q, quantize_acts=True) loss = criterion(y, batch.labels) grad = backward_with_ste(loss) # STE at every fake quant return w_float - lr * grad # update the float copy only ``` ```pseudocode # Conv + batch norm with correction and freezing [eq. 22-31] def bn_conv_qat(x, w, gamma, beta, mu_ema, sigma_ema, step, freeze_at): mu_b, sigma_b = batch_stats(conv(x, w)) c = sigma_b / sigma_ema # correction factor [eq. 22] w_fold = c * gamma * w / sigma_b # folded, long-term scale [eq. 23] y = conv(fake_quant(w_fold), x) bias = beta - gamma * mu_b / sigma_b if step < freeze_at: y = y / c # same output as batch norm [eq. 25] corr = 0 else: # moving statistics frozen, as at inference [eq. 29-31] corr = gamma * (mu_b / sigma_b - mu_ema / sigma_ema) return y + bias + corr ``` Notes that apply across variants: - The 2018 paper's learning rate for fine-tuning is 1e-5 with SGD. The schedule is not stated [arxiv:1806.08342 §3.2.3]. - The example snippet uses `quant_delay=2000000`. The value used in experiments is not stated [arxiv:1806.08342 §3.2]. - The 2018 paper does not state how the scale and zero-point are computed from the observed range, the rounding mode for activations, or the moving-average decay for activation ranges [arxiv:1806.08342 §2.1, §2.5]. - For SIMD kernels, signed symmetric weights are clamped to ±127 at 8 bits, dropping the most negative level [arxiv:1806.08342 §2.2, eq. 10]. Use the same range in training and at deployment. - How the two batch-norm bias terms combine in phase 2 is implicit in the 2018 paper. The sum above reproduces the inference bias [arxiv:1806.08342 §3.2.2, eq. 21, fig. 9]; this is our reading. - Before QAT, CLE rescales channel pairs so the float output is unchanged, which spreads the weight ranges evenly [arxiv:2106.08295 §3.2, eq. 22]. Run it on the float model, and only inside residual blocks, since skip connections break the equivariance across blocks [arxiv:2106.08295 §3.2]. - The 2021 paper's QAT runs use Adam [arxiv:2106.08295 §4.5]. With SGD-type optimizers, the rate for scales and zero-points should be lower than for weights; the factor is not stated [arxiv:2106.08295 §4.4]. - With a learned offset (LSQ+), the zero-gradient rule for β means it stops moving once it is below the lowest value, since every value is then inside the range [arxiv:2004.09576 §3.1 eq. 4]. Check the learned offsets after training. The paper's learned values stayed above the Swish minimum [arxiv:2004.09576 §5.1]. - ABC-Net's straight-through mask in eq. 10 is centred on R = v + 0.5, while its forward threshold sits at R = 0.5 − v. The card reads this as an inconsistency in the paper, so decide the mask deliberately and check the gradient [arxiv:1711.11294 eq. 8–10]. Its shift formula is undefined at M = 1, where the supplement uses zero shift [arxiv:1711.11294 supp. Parameter settings]. - PACT's clip level alpha has an L2 penalty, but the paper names the penalty without writing its exact form, and it does not state the gradient with respect to the input [arxiv:1805.06085 §4, app. A.2]. An implementation has to choose both. Its initial alpha is given only for one network, 10 for ResNet-20 on CIFAR-10 [arxiv:1805.06085 §4, fig. 2]. - The fully quantized variant on microcontrollers stores 8-bit weights, takes the update in floating point, then re-fits the scale and zero point from the new minimum and maximum [arxiv:2407.10734 §3.1, eq. 5–7]. The paper does not say how a zero range is handled when the new minimum equals the maximum [arxiv:2407.10734 eq. 6], or whether the batch gradient is summed or averaged before the update [arxiv:2407.10734 §3.1]. ## Contradictions - **MobileNetV2 at 4-bit weights after QAT.** The 2018 paper gives 0.62 against a float of 0.719, a gap of about 10 points, with symmetric per-channel weights and 8-bit activations [arxiv:1806.08342 tab. 5]. The 2021 paper gives 70.48 at W4A8 per-channel against a float of 71.72, a gap of about 1.2 points, after its standard pipeline [arxiv:2106.08295 tab. 10]. The 2018 paper fine-tunes TF-Slim checkpoints with BN freezing [arxiv:1806.08342 §3.2.3]. The 2021 paper's QAT uses Adam and 20 epochs on ImageNet models [arxiv:2106.08295 §4.5]. Neither card states the other paper's setup, so the gap is not explained here. - **Post-training 4-bit weights on MobileNetV2.** The 2018 paper gives 0.001 with symmetric per-channel weights [arxiv:1806.08342 tab. 5]. Its weights use the actual min and max [arxiv:1806.08342 §2.5]. The 2021 paper gives 69.79 at W4A8 per-channel for its PTQ pipeline [arxiv:2106.08295 tab. 6]. That pipeline adds CLE, MSE ranges, and AdaRound or bias correction [arxiv:2106.08295 §3.5]. The methods differ, so the two results are not in conflict on the same method. - **Batch-norm schemes.** The 2018 paper finds correction with freezing steadier than naive folding, batch renormalization, or moving averages without freezing, on MobileNet-v1 [arxiv:1806.08342 §4, fig. 14]. The 2021 paper finds static folding on par with or better than the double-forward method, and better than it in all four columns of its table, on MobileNetV2 and ResNet-18 [arxiv:2106.08295 §4.2, tab. 7]. Neither paper tests the other's scheme, so both results stand as stated. - **Freeze step for MobileNet-v2.** About 300000 steps in [arxiv:1806.08342 §3.2.2], and 400000 in [arxiv:1806.08342 §4] and the fig. 15 caption [arxiv:1806.08342 fig. 15]. - **Eq. 13 sign.** The 2018 paper writes the fake-quant with a minus sign on the zero-point inside the clamp. Its eqs. 1 to 3 use a plus sign, and the two agree only when $z=0$ [arxiv:1806.08342 §2.4, eq. 1-3, eq. 13]. The sketch uses the composed form. - **Per-layer weights, weight-only post-training.** The 2018 text says asymmetric per-layer is best. Its table 2 shows asymmetric per-channel as best, with per-layer collapsing on MobileNets [arxiv:1806.08342 §3.1.3, tab. 2]. This bears on the Variants above, which use the table values. - **Within the 2021 paper, text and table disagree.** Its text says W8A8 PTQ is within 0.7 % of float [arxiv:2106.08295 §3.6]. Its table shows MobileNetV2 per-tensor at 70.99, a 0.73 drop, and EfficientDet-D1 at 38.29, a 1.79 drop [arxiv:2106.08295 tab. 6]. Its text says W4A4 QAT stays within 1 % on networks without depth-wise convolutions [arxiv:2106.08295 §4.5], but its table shows ResNet-18 per-tensor at 68.32, 1.36 below float [arxiv:2106.08295 tab. 10]. - **Within the 2021 paper, table labels.** The MobileNetV2 values 12.99 (PTQ) and 70.13 (QAT) appear as "W4A8 with MSE weight init" in tab. 8 and as "W4A8 with CLE" in tab. 9. The source of tab. 9 has a TODO on its PTQ numbers. Which label is right is not determinable from the paper [arxiv:2106.08295 tab. 8, tab. 9]. - **Within the 2021 paper, learned scales.** Its §4.4 says that updating the scale every epoch, rather than learning it, gives better results at low bit-width. Read against the learned-scale setting above, this wording is ambiguous, so no claim is made here [arxiv:2106.08295 §4.4]. - **Within ConveRT, model size.** With 16M embedding and 13M network parameters, the stated 8-bit and 16-bit widths give about 42 MB, but tab. 2 lists 59 MB for the quantized ConveRT [arxiv:1911.03688 tab. 2]. The same convention reproduces the 118 MB figure for PolyAI-dual in that table, so the gap is not explained by the paper [arxiv:1911.03688 tab. 2]. - **Within ABC-Net, the full-precision gap.** The text gives a Top-1 gap of 0.9 points at M = 5 with full-precision activations [arxiv:1711.11294 §4.1.1]. Table 1 gives 69.3 − 68.3 = 1.0 [arxiv:1711.11294 tab. 1]. - **Within the HWA paper, network name.** §2.3 says WideResNet-32 on CIFAR-100 reaches iso-accuracy, but tab. 3 lists WideResNet-16 at 100.0 [arxiv:2302.08469 §2.3, tab. 3]. The variant above uses the table. - **Within the survey, the zero-point sign.** Its eq. 5 subtracts the zero point and its eq. 7 adds it [arxiv:2112.06126 §3.1, §3.2]. An implementation that follows one equation and not the other is off by 2Z on every value. The survey does not say which convention a deployment framework uses. - **Within 2407.10734, the CIFAR-10 accuracy.** Section 4.5 gives 89.5% for MCUNet-5FPS and 85.9% for MbedNet [arxiv:2407.10734 §4.5]. Table 4 gives 89.5 for the authors' uint8 method on the MCUNet backbone [arxiv:2407.10734 tab. 4]. The same number appears under two labels, and the paper does not say which is right. - **Within 2407.10734, the sparse-update rate.** The text says the rate falls toward lambda_min as the loss falls [arxiv:2407.10734 §3.2]. As written, eq. 9 uses the absolute gap between the loss and the largest loss seen so far. That gives the lowest rate when the loss equals that maximum, which is typical early in training, and the highest rate near zero loss [arxiv:2407.10734 eq. 9]. The paper does not resolve this. ## Papers - [Quantizing deep convolutional networks for efficient inference: A whitepaper](/papers/1806.08342): defines the uniform affine and symmetric quantizers, post-training and QAT procedures, batch-norm correction with freezing, and measured accuracy and latency on CNNs. - [A White Paper on Neural Network Quantization](/papers/2106.08295): defines PTQ and QAT pipelines for uniform affine quantization, with CLE, bias correction, AdaRound, learned scales, and its own batch-norm comparison and results on ImageNet, DeepLabV3, EfficientDet and BERT. - [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](/papers/2004.09576): adds a learned offset to learned-scale quantization for activations, with weight and activation initialisation schemes and results at W2A2 to W4A4 on ImageNet. - [Training Spiking Neural Networks Using Lessons From Deep Learning](/papers/2109.12894): a tutorial on training spiking networks with surrogate gradients. Its quantization part is brief, with listings and no accuracy numbers. - [ConveRT: Efficient and Accurate Conversational Representations from Transformers](/papers/1911.03688): a dual encoder for reply selection, with mixed-precision QAT that keeps embeddings at 8 bits and other weights at 16 bits. - [Analog in-memory hardware-aware training study](/papers/2302.08469): retrains eleven networks with a simulated analog crossbar in the forward pass, reporting accuracy at drift times up to one year. - [PACT: Parameterized Clipping Activation for Quantized Neural Networks](/papers/1805.06085): replaces the ReLU with a clipped activation whose clip level is learned per layer, with a straight-through gradient, and reports 2 to 5 bit results on CIFAR10, SVHN, AlexNet, ResNet18 and ResNet50 trained from scratch. - [Towards Accurate Binary Convolutional Neural Network](/papers/1711.11294): approximates weights with several binary bases and activations with several shifted binary thresholds, reporting ImageNet results for ResNet-18, ResNet-34 and ResNet-50. - [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](/papers/2506.20380): a self-supervised remote-sensing embedding model whose 128-d output is quantized to int8 with quantization-aware training. - [Neural Network Quantization for Efficient Inference: A Survey](/papers/2112.06126): a survey of number formats, QAT and PTQ, and binary, ternary and mixed-precision schemes, with comparison tables copied from cited works and no new method. - [On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollers](/papers/2407.10734): fully quantized training on three Cortex-M boards, with 8-bit weights and errors, a scale re-fit after each update, and partial error-channel updates.