ORCIDGitHubLinkedIn
RSS Curriculum Vitae TR

Prefix Caching

Turkish equivalent: Önek önbelleklemeDomain: Machine Learning Systems

Reuse of previously computed model state for a shared prompt prefix so repeated requests avoid recomputing identical early tokens.

  • KV Cache
  • Context Window
  • Paged Attention
  • Cache Stampede

Updated:

Related concepts

Machine Learning Systems