A language model can assign high probability to tokens that are fluent but generic. Contrastive decoding changes token selection by comparing a stronger expert model with a weaker amateur model at the same generation position. A token becomes attractive when the expert favors it more strongly than the amateur does.
The comparison is not an unrestricted subtraction across the vocabulary. The original method also keeps candidate tokens inside a plausibility set defined by the expert. That constraint matters because a large expert-amateur score gap can otherwise promote a token that both models consider implausible.
The score compares two conditional distributions
At generation position t, both models receive the same prefix. Let p_e(x) be the expert probability for candidate token x, and let p_a(x) be the amateur probability. A basic contrastive score can be written as:
score(x) = log p_e(x) - log p_a(x)This is also the logarithm of a probability ratio:
score(x) = log(p_e(x) / p_a(x))A token scores highly when its probability under the expert is large relative to its probability under the amateur. That is different from selecting the expert’s highest-probability token. A token can rank second or third under the expert yet win the contrast if the amateur assigns it much less probability.
Some implementations use a weighted form such as:
score(x) = (1 + beta) * log p_e(x) - beta * log p_a(x)with beta >= 0. At beta = 0, this reduces to ordinary expert log-probability scoring. Increasing beta gives the amateur distribution more influence over token ranking.
The exact parameterization is implementation-specific. A configuration value called a contrast weight has no portable meaning unless its scoring equation is known.
Plausibility filtering keeps the expert in control
Contrastive scoring alone has a structural failure mode. Suppose the expert assigns a token probability of 0.0001 and the amateur assigns 0.0000001. Their ratio is large even though the expert itself considers the token very unlikely.
A plausibility constraint prevents that ratio from dominating selection. One common form keeps tokens whose expert probability is at least a fraction alpha of the expert’s maximum token probability:
V_valid = {
x : p_e(x) >= alpha * max_y p_e(y)
}Selection then applies the contrastive score only inside V_valid.
With alpha = 0.1, a candidate must have at least one tenth of the probability assigned to the expert’s most probable token. This is a relative threshold, not a fixed probability cutoff. Its effective strictness changes with the shape of the expert distribution at each position.
A peaked expert distribution can therefore admit few candidates, while a flatter distribution can admit more. The contrast operation acts only after that local gate has been established.
Model compatibility is part of the mechanism
The expert and amateur need scores for corresponding candidate tokens. The simplest setup uses models with the same tokenizer and vocabulary, so token IDs have matching meanings.
If the vocabularies differ, subtracting logits at the same numeric index is invalid. A token ID is local to a tokenizer. Building a correct cross-tokenizer mapping is also not generally equivalent to direct token-level scoring because one token from one vocabulary can correspond to multiple tokens in another.
The models must also evaluate equivalent prefixes. Differences in chat templates, special tokens, beginning-of-sequence handling, or normalization can cause the two distributions to represent different conditional contexts. In that case, the numerical contrast still exists, but its interpretation as expert versus amateur judgment on the same next-token choice is weakened.
Using related model families often simplifies these constraints, but family membership alone is not a guarantee. Tokenizer assets and prompt serialization should be checked explicitly.
Raw logits require careful normalization
A softmax log probability has the form:
log p_i = z_i - logsumexp(z)where z_i is a logit. For a single model, subtracting the same logsumexp term from every candidate does not change token ranking. Across two models, however, the normalization constants are different.
For the unweighted difference
log p_e(x) - log p_a(x)the two normalization constants contribute a candidate-independent offset, so they do not change the ranking among candidates at that position. This makes a difference of corresponding raw logits rank-equivalent for that specific unweighted objective.
That equivalence should not be generalized carelessly. Plausibility filtering is defined from expert probabilities, weighted variants change coefficients, and other transformations can depend on normalized values. An implementation should derive any logit shortcut from its actual equation rather than assume logits and log probabilities are interchangeable in every contrastive decoder.
The amateur is a reference distribution, not a draft model
Contrastive decoding and speculative decoding can both pair a large model with a smaller model, but the smaller model has a different role.
In speculative decoding, a draft model proposes tokens and the target model verifies them under a procedure designed to preserve the target distribution. The central goal is reducing target-model decoding work when draft tokens are accepted.
In contrastive decoding, the amateur contributes directly to the token score. Its distribution changes which token is selected. The method is therefore a decoding objective, not merely a scheduling optimization around an unchanged expert distribution.
This distinction affects serving design. Both expert and amateur scores are needed at positions where contrastive selection is performed. Running the amateur adds computation and model state even when it is much smaller than the expert. Whether that cost is acceptable depends on hardware placement, batching, model sizes, and the value of the changed decoding behavior.
Amateur strength changes the available contrast
The method relies on disagreement. If expert and amateur distributions become nearly identical, their log-probability difference approaches a candidate-independent value and contributes little useful ranking signal.
The opposite extreme also needs care. A very weak or poorly matched amateur can disagree for reasons unrelated to the behavior the decoder is intended to suppress. The expert plausibility gate limits some damage, but it cannot guarantee that every remaining contrast reflects a desirable distinction.
The amateur should therefore be treated as part of the decoding specification. Changing its checkpoint can change outputs even when the expert, prompt, threshold, and contrast weight stay fixed.
This also makes regression testing more specific. Recording only the expert checkpoint is insufficient for reproducing a contrastive decoder. The amateur checkpoint, tokenizer compatibility, prompt serialization, plausibility rule, score equation, and selection policy all participate in the generated result.
Contrastive decoding changes the target of generation
Ordinary greedy decoding asks for the most probable next token under one model. Contrastive decoding asks a different question: among tokens the expert considers plausible, which token is favored by the expert relative to the amateur?
That objective can expose distinctions hidden by the expert distribution alone, but it is not a generic quality switch. Its behavior depends on the relation between two models and on the gate that limits candidate tokens. For developers, those relationships are the core configuration surface: the decoder is defined as much by the reference model and plausibility rule as by the expert that produces the final text.