Pipeline Parallelism

Turkish equivalent: Pipeline paralelliğiDomain: Machine Learning Systems

A distributed-training or inference strategy that partitions model layers into stages and overlaps processing of multiple microbatches across devices.

  • Tensor Parallelism
  • Data Parallelism
  • Throughput
  • Load Balancing