https://nalar.dev/tags/inference/ Recent content in Inference on Nalar Hugo en-us Wed, 02 Sep 2026 00:00:00 +0700 https://nalar.dev/control-llm-randomness-with-temperature-and-top-p/ Wed, 02 Sep 2026 00:00:00 +0700 https://nalar.dev/control-llm-randomness-with-temperature-and-top-p/ <p>Large language models usually generate text one token at a time. At each step, the model assigns scores to possible next tokens, those scores become probabilities, and a decoding strategy chooses what comes next.</p> <p>Two common controls in that process are <strong>temperature</strong> and <strong>top-p</strong>. They are often described as creativity settings, but that description is incomplete. They change how the model samples from its probability distribution, which affects repeatability, diversity, and the chance of selecting lower-probability tokens.</p>