Grouped-Query Attention
An attention variant in which multiple query heads share fewer key/value heads to reduce memory and inference cost.
Related Concepts
- Multi-Head Attention
- KV Cache
- Transformer
- Memory Bandwidth
An attention variant in which multiple query heads share fewer key/value heads to reduce memory and inference cost.