Pipeline Parallelism
A distributed-training or inference strategy that partitions model layers into stages and overlaps processing of multiple microbatches across devices.
Related Concepts
- Tensor Parallelism
- Data Parallelism
- Throughput
- Load Balancing