--- kind: concept slug: asymmetric-quantization title: Asymmetric quantization summary: >- Low-bit grids whose lowest level can sit away from zero, set by an offset next to the scale. On EfficientNet-B0 with 4-bit activations, a learned offset lifts accuracy over a symmetric grid, and most of the gain comes at 2 bits. For LLM weights, a per-row min-max grid collapses at 3 bits without GPTQ's rounding. Two cards so far: CNN activations and LLM weights. tags: [quantization, low-bit-inference, activation-quantization, weight-quantization, efficient-inference] papers: ["2004.09576", "2210.17323"] related: [quantization-aware-training, weight-quantization, post-training-quantization] status: draft updated_at: 2026-10-19 --- ## Overview Asymmetric quantization maps each real value to a low-bit integer code on a grid that does not have to be centred on zero. A symmetric grid is fixed around zero, so a range that is mostly positive, or that dips only slightly below zero, wastes levels. An asymmetric grid adds an offset β next to the scale s, so its lowest level can sit at the most negative value that occurs. The shared invariant is that a quantized value is an integer code times s, plus β [arxiv:2004.09576 §3.1] [eq. 2]. GPTQ uses the same kind of grid with a per-row minimum and maximum, and it does not learn the offset [arxiv:2210.17323 §5 Setup]. Two papers are on this page so far, and they apply the idea to different targets: - [LSQ+](/papers/2004.09576) quantizes activations of ImageNet CNNs during quantization-aware training. It learns s and β by gradient descent with a straight-through rounding estimator [arxiv:2004.09576 §3.1] [eq. 3] [eq. 4]. The motivating case is Swish-like activations, whose negative part is bounded near −0.278 while the positive part is unbounded [arxiv:2004.09576 §3.1]. In that paper the weights stay symmetric and signed [arxiv:2004.09576 §3.2.1]. - [GPTQ](/papers/2210.17323) quantizes the weights of GPT-style language models after training, with no retraining, using a calibration set of 128 text segments [arxiv:2210.17323 §5 Setup]. Its grid is uniform and asymmetric per row, set by min-max [arxiv:2210.17323 §5 Setup]. ## Variants Each variant is a choice of grid, of how its parameters start, or of what the grid is applied to. Activation results come from [LSQ+](/papers/2004.09576) on EfficientNet-B0, MixNet-S and ResNet18 at W2A2, W3A3 and W4A4 [arxiv:2004.09576 tab. 2, tab. 3, tab. 4]. Weight results come from [GPTQ](/papers/2210.17323) on OPT and BLOOM language models [arxiv:2210.17323 tab. 3, tab. 4, tab. 5]. ### Symmetric signed grid (baseline) No offset, with a sign bit: the grid is centred on zero and the range is −2^(b−1) to 2^(b−1) − 1 [arxiv:2004.09576 §3.1] [tab. 1]. This is Config 2 in the paper. It is the weakest of the four configurations on EfficientNet-B0, at 68.8% for W4A4 and 23.7% for W2A2, against 76.1% full precision [arxiv:2004.09576 tab. 2]. ### Unsigned symmetric grid (LSQ) No offset and no sign bit, so the grid covers only zero and positive values [arxiv:2004.09576 §3.1] [tab. 1]. Negative activations are clamped to zero. This is Config 1, the LSQ reference, at 71.9% for W4A4 and 43.5% for W2A2 on EfficientNet-B0 [arxiv:2004.09576 tab. 2]. Its loss against full precision at W4A4 is 4.2 points [arxiv:2004.09576 tab. 2]; see Contradictions. ### Signed asymmetric grid A learned offset with a sign bit. This is Config 3 in the paper. On EfficientNet-B0 it reaches 73.5% at W4A4, 69.9% at W3A3 and 49.1% at W2A2 [arxiv:2004.09576 tab. 2]. On MixNet-S it reaches 71.6% at W4A4 and 42.5% at W2A2 [arxiv:2004.09576 tab. 3]. ### Unsigned asymmetric grid (LSQ+) A learned offset and no sign bit. This is Config 4, and it is the configuration the paper uses for its analyses [arxiv:2004.09576 §4.3] [tab. 5] [tab. 6]. On EfficientNet-B0 it reaches 73.8% at W4A4, 69.3% at W3A3 and 48.7% at W2A2 [arxiv:2004.09576 tab. 2]. Configs 3 and 4 are nearly equal in tables 2 and 3, within about 0.3 points at W4A4 on EfficientNet-B0 and MixNet-S [arxiv:2004.09576 tab. 2, tab. 3]. The weights in the same experiments are still symmetric and signed [arxiv:2004.09576 §4.3]. ### Fixed offset at the observed minimum The offset is not learned. It is set to the lowest observed value, x_min, and the scale is learned. On EfficientNet-B0 at W4A4 with Config 4 this gives 72.5%, between the symmetric LSQ baseline at 71.9% and the learned offset at 73.8% [arxiv:2004.09576 tab. 6]. This is the only fixed-offset row the card records, and it is for one network and one bit-width. ### Initialisation of s and β Min-max initialisation sends the observed minimum to the lowest level and the observed maximum to the highest [arxiv:2004.09576 §3.2.2] [eq. 6] [eq. 7]. The paper's proposed MSE initialisation chooses s and β to minimise the squared error between quantized and original activations, found by gradient steps over a few batches [arxiv:2004.09576 §3.2.2] [eq. 8]. On EfficientNet-B0 with Config 4, the mean over 5 runs is 73.0% ± 0.9 at W4A4 and 46.8% ± 1.9 at W2A2 for the proposed initialisation. Min-max gives 71.3% ± 2.2 and 43.8% ± 4.7, and LSQ-style initialisation gives 72.0% ± 1.6 and 44.4% ± 2.9 [arxiv:2004.09576 tab. 5]. The spread is the main reported effect, and it is largest for min-max. ### Per-row min-max grid for weights (GPTQ) Each row of a weight matrix gets its own grid, fitted to that row's minimum and maximum, and the grid is fixed before rounding starts [arxiv:2210.17323 §5 Setup]. Weights are the only quantized tensor, and activations are not quantized [arxiv:2210.17323 §6]. The grid is asymmetric, so it can cover a row whose values are not centred on zero. GPTQ's main contribution is not this grid. Round-to-nearest on the same grid is the baseline in the paper [arxiv:2210.17323 §5 Baselines], so the gap between the two comes from how each weight is rounded, not from the grid. Main results, OPT-175B, WikiText2 perplexity (lower is better), FP16 8.34 [arxiv:2210.17323 tab. 5]: - 4-bit, round-to-nearest 10.54 against GPTQ 8.37 [arxiv:2210.17323 tab. 5]. - 3-bit, round-to-nearest 7.3e3 against GPTQ 8.68 [arxiv:2210.17323 tab. 5]. Round-to-nearest collapses at 3 bits on this grid. - 3-bit with group size 128 gives 8.45, and with group size 1024 gives 8.45 [arxiv:2210.17323 tab. 5]. Grouping adds about 0.15 bits per weight at g128 and about 0.02 at g1024 [arxiv:2210.17323 §5 175 Billion Parameter Models]. - 2-bit with group size 32 gives 8.94, and a ternary grid with group size 8 gives 9.20 [arxiv:2210.17323 tab. 7] [arxiv:2210.17323 §5 Extreme Quantization]. Quantizing OPT-175B takes about four hours on one A100 80 GB [arxiv:2210.17323 tab. 2]. ## When to use which The two papers answer different questions, and the constraints below are limited to the settings they tested. [LSQ+](/papers/2004.09576) tests learned offsets during fine-tuning from a pre-trained float model. Every result comes from that setting, so the claims below are limited to it [arxiv:2004.09576 §4]. Post-training use with no fine-tuning is not among the recorded results. [GPTQ](/papers/2210.17323) tests post-training weight quantization with no retraining, on language models with 125M to 176B parameters [arxiv:2210.17323 §5 Setup]. | Situation | Use | Why | |---|---|---| | Skewed activations, for example Swish-like, at 2 to 4 bits, with fine-tuning available | Asymmetric (signed or unsigned) with a learned offset | At W2A2 on EfficientNet-B0 the asymmetric grids reach 48.7% to 49.1%, against 43.5% for the symmetric LSQ baseline and 23.7% for signed symmetric [arxiv:2004.09576 tab. 2] | | Activations that are only non-negative in practice | Unsigned asymmetric, not signed | Config 3 and Config 4 are close. Config 4 is higher at W4A4 (73.8% against 73.5%), and Config 3 is higher at W2A2 and W3A3 on EfficientNet-B0 [arxiv:2004.09576 tab. 2]. The recorded results do not isolate a non-negative activation function, so this row rests on the close numbers only | | Need the largest gain at the lowest bit-width | Learned offset | The gain over LSQ is about 5 points at W2A2 on EfficientNet-B0, and smaller at W4A4 [arxiv:2004.09576 tab. 2]. Its gap to full precision is still large | | Only a small gain is needed at 4 bits, or no training is possible | Fixed offset at x_min, or LSQ | The fixed offset gives 72.5% against 73.8% learned and 71.9% for LSQ, on one network [arxiv:2004.09576 tab. 6]. The paper does not test it without training | | ResNet18 at W4A4 | Expect little gain over symmetric LSQ | Config 1 gives 70.7%, and Config 4 gives 70.8% against 70.1% full precision [arxiv:2004.09576 tab. 4]. The paper reports no gap between Configs 1, 3 and 4 on this network [arxiv:2004.09576 §4.2]. The gain is network-dependent | | Run-to-run variance matters, e.g. for a repeatable recipe | Use the MSE initialisation | Its spread at W2A2 is ±1.9 points over 5 runs, against ±4.7 for min-max [arxiv:2004.09576 tab. 5] | | Depthwise-separable networks such as MobileNets | Pay attention to initialisation | The paper says poor starting values give high variance, especially with depthwise separable convolutions [arxiv:2004.09576 §3.2]. The recorded tables cover EfficientNet-B0, MixNet-S and ResNet18, not MobileNets | | Layer whose sigmoid output has no negative part, e.g. squeeze-and-excite | Use no offset | The paper leaves these layers without an offset [arxiv:2004.09576 §5.1] | | Target runtime cannot add a bias to the layer | Check before choosing an asymmetric grid | The paper's zero-overhead claim depends on folding the offset into a bias [arxiv:2004.09576 §3.1] [eq. 5] [§6] | | Weights of a trained LLM, no retraining, 4 bits | Per-row asymmetric min-max grid with GPTQ-style rounding | At 4 bits on OPT-175B, GPTQ gives 8.37 WikiText2 perplexity against 8.34 FP16 [arxiv:2210.17323 tab. 5] | | Weights of a trained LLM, no retraining, 3 bits | GPTQ-style rounding on the per-row grid, not plain rounding | Plain round-to-nearest on the same grid gives 7.3e3 at 3 bits on OPT-175B, while GPTQ gives 8.68 [arxiv:2210.17323 tab. 5] | | Weights below 3 bits | GPTQ with grouping, for the largest models only | The paper's 2-bit results are grouped and cover OPT-175B and BLOOM-176B [arxiv:2210.17323 tab. 7]. Group size 32 gives 8.94 against 8.34 FP16 on OPT-175B [arxiv:2210.17323 tab. 7] | | Both weights and activations at low bits | No result on this page | GPTQ does not quantize activations [arxiv:2210.17323 §6], and LSQ+ keeps weights symmetric [arxiv:2004.09576 §4.3]. The two papers do not test the combination | ## Implementation sketch Pseudocode in our own words. The quantizer follows eq. 2, and the initialisers follow §3.2.2. The offset fold is the part where variants most often break, so check eq. 5 before implementing it. ```pseudocode # Asymmetric activation quantizer [§3.1] [eq. 2] # Input: x (activations); scale s > 0; offset beta; integer range n, p # Output: x_hat, same shape as x def quantize_activation(x, s, beta, n, p): # Shift by the offset, then divide by the scale [eq. 2] scaled = (x - beta) / s # Clamp to the integer range before rounding [eq. 2] codes = round(clamp(scaled, n, p)) # Map codes back to real values [eq. 2] return codes * s + beta ``` ```pseudocode # Min-max initialisation of s and beta [§3.2.2] [eq. 7] # Input: observed x_min, x_max; integer range n, p # Output: s_init, beta_init def init_minmax(x_min, x_max, n, p): # Stretch the observed range over the integer levels [eq. 7] s_init = (x_max - x_min) / (p - n) # Place x_min at the lowest level n [eq. 7] beta_init = x_min - n * s_init return s_init, beta_init ``` ```pseudocode # Offset fold for a linear layer with symmetric weights [§3.1] [eq. 5] # Input: weight codes, weight scale s_w, activation offset beta, bias # Output: bias to use with integer-only matmul on the codes def fold_offset(w_codes, s_w, beta, bias): # Expanding the product gives y = s * s_w * (w_codes . x_codes) + beta * s_w * sum(w_codes) # The second term is per output channel and depends only on the weights [eq. 5] # Adding it to the bias removes it from the runtime [eq. 5] correction = beta * s_w * sum(w_codes, axis=1) return bias + correction ``` ## Contradictions No two papers on this page report conflicting results. [LSQ+](/papers/2004.09576) measures activations of CNNs during fine-tuning on ImageNet, and [GPTQ](/papers/2210.17323) measures weights of language models after training on perplexity, so their numbers are not comparable. Inside the LSQ+ paper, several reported numbers do not match each other. Each is set out in the card's Pitfalls, and the page does not resolve them: - Gain over LSQ at W4A4. The text gives 1.6 to 1.8 points [arxiv:2004.09576 §4.1]. Table 2 gives 1.6 for Config 3 and 1.9 for Config 4 [arxiv:2004.09576 tab. 2]. - LSQ loss at W4A4 on EfficientNet-B0. Section 1 gives 4.1 points [arxiv:2004.09576 §1]. Table 2 gives 4.2 [arxiv:2004.09576 tab. 2]. - Comparability of numbers. Tables 2 to 4 report the best accuracy after a grid search, while table 5 reports a mean over 5 runs [arxiv:2004.09576 tab. 5 caption]. The 73.8% in table 2 and the 73.0% in table 5 are not comparable. - Fixed β = 0 baseline. The row in table 6 is 71.9%, identical to table 2's Config 1. The paper does not say whether it was a separate run [arxiv:2004.09576 tab. 6]. Between the GPTQ paper and its code, the grid differs. The paper says min-max [arxiv:2210.17323 §5 Setup], and the official code also forces zero into the range and rounds the zero point. The paper does not state either step. A grid written only from the paper will differ from the released code. The details are in [the GPTQ card](/papers/2210.17323), under Paper vs code. ## Papers - [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](/papers/2004.09576): learned offset next to the learned scale for activations, and MSE-based initialisation that narrows run-to-run spread. - [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](/papers/2210.17323): per-row asymmetric min-max grid for weights, with error compensation that makes 3-bit and 4-bit weights usable on large language models without retraining.