Expert Capacity Turns Uneven MoE Routing into Token Overflow

Thu, 24 Sep 2026 00:00:00 +0000

A sparse Mixture-of-Experts layer can route many tokens toward the same expert even when every expert has identical nominal capacity. The router makes token-dependent choices, while distributed execution commonly allocates bounded token slots per expert. When those two mechanisms disagree, an expert can receive more assignments than its execution buffer admits.

That boundary is not an inherent property of every MoE architecture. It is a property of capacity-constrained routing designs, including the routing formulation described for Switch Transformers. In such systems, expert capacity converts an uneven routing distribution into an operational event: some assignments fit, while assignments beyond the capacity limit require an explicit overflow policy.

Routing on Nalar

Expert Capacity Turns Uneven MoE Routing into Token Overflow