A decoder can receive logits large enough that direct exponentiation is numerically unsafe even though the intended probability distribution is ordinary. Softmax does not require exponentiating the original values. Subtracting the largest logit from every logit produces the same distribution in exact real arithmetic while moving the exponentials into a safer numeric range.

This shift is a property of softmax itself, not a model-specific heuristic.

A shared shift cancels during normalization

For logits (z_1,\ldots,z_n), softmax assigns

[ p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}. ]

Subtract any shared constant (c):

[ \frac{e^{z_i-c}}{\sum_j e^{z_j-c}}

\frac{e^{-c}e^{z_i}}{e^{-c}\sum_j e^{z_j}}

p_i. ]

Choosing (c=\max_j z_j) is useful because every shifted logit is then non-positive and at least one is zero. The corresponding exponentials fall in the interval ((0,1]) in real arithmetic. A very large positive logit therefore no longer requires evaluating an equally large exponential.

The transformation changes the numeric representation of the computation, not the mathematical softmax result.

Overflow and underflow have different consequences

Directly computing (e^{z_i}) can overflow when a logit exceeds the usable exponential range of the active floating-point format and implementation. Once an intermediate becomes infinity, normalization can produce invalid operations such as infinity divided by infinity.

After max subtraction, positive-exponential overflow from finite shifted logits is avoided because no shifted value exceeds zero. Very negative differences can still underflow so their exponentials become zero in finite precision. That behavior is not mathematically exact, but it usually represents probability mass already far below the dominant terms at the available precision.

The distinction matters: max subtraction controls one severe failure mode, but it does not make finite-precision softmax identical to real arithmetic.

Logit differences carry the distribution

Softmax probabilities depend on differences between logits rather than their absolute offset. If every logit rises by 1,000, the distribution is unchanged mathematically. If one logit rises relative to the others, the distribution changes.

This also means a diagnostic that only inspects absolute logit magnitude can miss the relevant geometry. Large logits clustered near one another can produce a broad distribution after shifting. Moderate logits separated by large gaps can produce a sharply concentrated distribution.

For generation systems, temperature changes those differences before normalization. With temperature (T>0), a common form is

[ p_i = \frac{\exp(z_i/T)} {\sum_j \exp(z_j/T)}. ]

A stable implementation can subtract the maximum of the scaled logits, or equivalently apply a correctly derived shared shift. The ordering of mathematically equivalent operations can still affect finite-precision rounding, so runtime details remain relevant when exact reproducibility is required.

Masked values need explicit handling

Attention and token-selection code often represents excluded positions with a large negative value or negative infinity before softmax. The max-shift argument assumes the row contains at least one finite value that can anchor the subtraction.

If every entry is negative infinity, then the maximum is also negative infinity, and subtracting it yields undefined expressions. Libraries and kernels may handle fully masked rows differently. A caller cannot infer valid zero probabilities solely from the algebra used for an ordinary finite row.

The same boundary applies to NaNs. Subtracting a maximum is not a repair mechanism for non-finite inputs produced earlier in the pipeline.

Log-softmax uses the same shift

Log probabilities are often computed without first materializing normalized probabilities. The stable identity is

[ \log p_i

z_i - m

\log\left(\sum_j e^{z_j-m}\right), \quad m=\max_j z_j. ]

This form avoids taking a logarithm of a softmax value that may already have rounded to zero. It also exposes the close relation between stable softmax and the log-sum-exp operation.

The implementation boundary remains finite precision. Reduction order, accumulator precision, fused kernels, quantized inputs, and device-specific math can produce small numeric differences even when every implementation follows the same algebraic identity. Max subtraction supplies a stable coordinate shift; it does not promise bitwise agreement across runtimes.