Artificial Intelligence
24 Sep 2026
6 min read
Grouped-Query Attention Shrinks the KV Cache by Sharing Key-Value Heads
Grouped-query attention changes the number of key and value heads that must be stored during autoregressive decoding. Instead of giving every query head its own key-value pair, several query heads address the same key-value head. The query side can retain many heads while the persistent KV state uses fewer independent projections. This is an architectural change to attention, not a cache compression codec. The smaller cache follows from producing fewer distinct key and value heads per token.