ORCIDGitHubLinkedIn
RSS Curriculum Vitae TR

Speculative Decoding

Turkish equivalent: Spekülatif çözümlemeDomain: Large Language Models

An inference technique in which a cheaper draft model proposes multiple tokens that a larger target model verifies in batches to increase decoding throughput.

  • KV Cache
  • Inference
  • Latency
  • Autoregressive Model

Source

  • https://arxiv.org/abs/2211.17192

Updated:

Related concepts

Large Language Models