Rotary Position Embedding
A positional encoding technique that rotates query and key vectors so relative token positions are represented directly in attention geometry.
Related Concepts
- Self-Attention
- Context Window
- KV Cache
- Transformer
A positional encoding technique that rotates query and key vectors so relative token positions are represented directly in attention geometry.