A tensor can be easy to represent in floating point and awkward to map into a small integer range. The problem becomes acute when most activation values occupy a narrow interval while a small number have much larger magnitude. A shared quantization scale must cover those extremes, so the ordinary values receive fewer representable levels.
That is the practical effect of activation outliers in low-bit inference. The issue is not merely that an outlier is numerically large. Its location, persistence, and relationship to the axis over which a scale is shared determine whether it materially degrades the quantized representation.
A shared scale couples unrelated magnitudes
Consider symmetric integer quantization with an integer range [-Q, Q]. A simple scale can be written as:
s = max(|x|) / Q
q = clamp(round(x / s), -Q, Q)
x_hat = s * qIf one value makes max(|x|) much larger than the typical magnitude, s increases for every value that shares that scale. The quantization step therefore becomes coarser across the whole group.
For example, suppose most values are between -1 and 1, but one value is 20. With the same integer bit width, a scale selected to retain 20 allocates much of the available numeric range to magnitudes that most elements never use. Values near zero can collapse onto the same integer code even though their floating-point values differ.
This is a scale-allocation problem. It does not imply that every large activation is an error, nor that clipping it is automatically acceptable. A large value may carry information that subsequent operations use.
The quantization granularity changes the failure surface
Per-tensor quantization gives one scale to an entire tensor. Per-channel or per-group schemes divide values into smaller sets with separate scales. Smaller groups can isolate an extreme value so it does not set the resolution for unrelated elements.
The relevant grouping depends on the operation and implementation. For a matrix multiplication, activation and weight layouts, kernel support, and the axis used for scaling all matter. Two quantizers with the same nominal bit width can therefore behave differently on the same model because their scale granularity differs.
This also explains a common measurement trap. Reporting only a tensor-wide maximum says little about how much of the tensor is affected. A useful inspection includes the distribution of magnitudes and their positions along the axis that shares a quantization scale. A few large values concentrated in recurring channels present a different problem from large values scattered unpredictably across tokens and channels.
Clipping exchanges range error for resolution
Clipping reduces the range used to choose a scale. If a threshold T replaces the absolute maximum, the scale can become:
s = T / Q
x_clipped = clamp(x, -T, T)The smaller range gives finer steps to values inside [-T, T], but values outside the interval are altered before integer rounding. Quantization error falls for some values and rises sharply for the clipped tail.
The appropriate threshold is therefore tied to model sensitivity, not just the activation histogram. A percentile rule can describe the data distribution, but it does not establish that the removed magnitude is unimportant to the model output. Calibration data is useful only to the extent that it represents the activation patterns expected at inference time.
Static calibration also has a boundary: a threshold selected from one sample of prompts can encounter a different range later. Dynamic activation quantization can adapt scales at runtime, but it adds scale computation and still has to operate at a supported granularity.
Persistent outlier channels permit a different intervention
When unusually large activations repeatedly occur in particular channels, scaling can redistribute numeric difficulty between activations and weights without changing the corresponding floating-point linear operation.
For a linear transform:
y = xWintroduce a positive diagonal scaling matrix D:
y = (x D^-1)(D W)In exact arithmetic, the product is unchanged. The activation channels can be reduced by D^-1 while the corresponding weight rows are enlarged by D. Quantization then acts on tensors with different ranges.
The identity does not guarantee equal behavior after quantization. It deliberately moves range between two operands, so the resulting error depends on their quantizers, bit widths, grouping, calibration, and kernel implementation. Excessive rescaling can make the weight side harder to quantize even as the activation side becomes easier.
This mechanism is most applicable when the troublesome activation structure is stable enough for an offline transformation to capture. If extreme channels vary substantially with input or layer state, a fixed redistribution may not isolate the problem as neatly.
Calibration data defines the range that gets observed
Post-training activation quantization commonly relies on representative inputs to estimate ranges or other scale statistics. Those inputs are not merely a validation convenience. They determine which activation regimes the quantizer sees before deployment.
A calibration set dominated by short, uniform prompts can miss magnitude patterns associated with different sequence lengths, languages, formatting, modalities, or task structures. The relevant dimensions depend on the model and serving workload. More calibration examples do not automatically fix a distribution mismatch if they repeat the same narrow regime.
Range diagnostics are more informative when recorded per layer and at the same granularity used by the quantizer. Useful signals include the ratio between extreme and typical magnitudes, the recurrence of high-magnitude channels, and the fraction of values that a proposed clipping threshold would alter. These measurements identify where a low-bit representation is under pressure without assuming that every outlier should be suppressed.
Kernel support constrains the numerical design
A mathematically attractive quantization scheme still has to map onto an inference kernel. Very fine-grained scales can reduce interference from outliers, but they require scale metadata and arithmetic at matching boundaries. Hardware and libraries differ in which integer formats, group sizes, zero points, and accumulation types they support.
As a result, quantization quality cannot be separated completely from serving implementation. A model converted with one grouping scheme may need a different representation on a backend that lacks the required kernel. Falling back to a slower path can also defeat the deployment objective even if numerical error is low.
Activation outliers are therefore best treated as a boundary between model statistics and integer representation. The key question is not whether large values exist, but which values share their scale, how stable the extremes are across inference inputs, and what transformations the serving kernel can execute without changing the intended computation.