https://nalar.dev/tags/text-generation/ Recent content in Text Generation on Nalar Hugo en-us Sat, 05 Sep 2026 00:00:00 +0700 https://nalar.dev/understand-contrastive-decoding-with-expert-and-amateur-models/ Sat, 05 Sep 2026 00:00:00 +0700 https://nalar.dev/understand-contrastive-decoding-with-expert-and-amateur-models/ <p>A language model can assign high probability to text that is fluent but unhelpfully generic, repetitive, or too close to an easy pattern. Changing temperature or top-p changes how tokens are sampled from one model&rsquo;s distribution, but it does not ask a different question: which candidate tokens are especially characteristic of a stronger model rather than a weaker one?</p> <p><strong>Contrastive decoding</strong> asks exactly that. It uses two language models at inference time: a stronger <strong>expert</strong> and a weaker <strong>amateur</strong>. A candidate is favored when the expert scores it well relative to the amateur, while a plausibility constraint prevents the decoder from choosing bizarre tokens merely because the amateur dislikes them even more.</p>