Multi-Head Attention
An attention mechanism that runs several learned attention projections in parallel to capture different representation subspaces.
Related Concepts
- Self-Attention
- KV Cache
- Transformer
- Grouped-Query Attention
An attention mechanism that runs several learned attention projections in parallel to capture different representation subspaces.