https://nalar.dev/tags/grouped-query-attention/ Recent content in Grouped-Query Attention on Nalar Hugo en-us Thu, 24 Sep 2026 00:00:00 +0000 https://nalar.dev/grouped-query-attention-shrinks-kv-cache-by-sharing-key-value-heads/ Thu, 24 Sep 2026 00:00:00 +0000 https://nalar.dev/grouped-query-attention-shrinks-kv-cache-by-sharing-key-value-heads/ <p>Grouped-query attention changes the number of key and value heads that must be stored during autoregressive decoding. Instead of giving every query head its own key-value pair, several query heads address the same key-value head. The query side can retain many heads while the persistent KV state uses fewer independent projections.</p> <p>This is an architectural change to attention, not a cache compression codec. The smaller cache follows from producing fewer distinct key and value heads per token.</p>