Transformer
A sequence-model architecture built primarily around attention, feed-forward blocks, residual connections, and normalization rather than recurrence.
Related Concepts
- Self-Attention
- Multi-Head Attention
- Tokenization
- Context Window
Source
- https://arxiv.org/abs/1706.03762