Byte Pair Encoding

Turkish equivalent: Bayt çifti kodlamaDomain: Large Language Models

A subword tokenization method that repeatedly merges frequent adjacent symbol pairs to build a compact vocabulary from raw text units.

  • Tokenization
  • Vocabulary
  • Subword
  • Context Window