https://nalar.dev/tags/model-routing/ Recent content in Model Routing on Nalar Hugo en-us Sat, 12 Sep 2026 00:00:00 +0700 https://nalar.dev/control-expert-routing-in-mixture-of-experts-models/ Sat, 12 Sep 2026 00:00:00 +0700 https://nalar.dev/control-expert-routing-in-mixture-of-experts-models/ <p>A mixture-of-experts layer can contain far more parameters than it evaluates for each token. A router scores a set of experts, selects a small subset, and sends each token only to those selected computation paths. The parameter count can grow without making every token execute every expert.</p> <p>That sparse structure creates a separate systems problem: the router decides where computation lands. Two models with the same experts and the same nominal top-k routing can have very different behavior if one spreads tokens across experts and the other concentrates them on a few paths.</p>