Self-Attention
An attention operation in which tokens attend to other tokens within the same sequence to build context-dependent representations.
Related Concepts
- Attention
- Transformer
- Context Window
- KV Cache
An attention operation in which tokens attend to other tokens within the same sequence to build context-dependent representations.