Load Balancing | Capacity, Backpressure and Saturation

The distribution of incoming work across multiple processors, processes, servers, or replicas so that avoidable saturation is not concentrated on one resource.

Load balancing is more than selecting round-robin. Equal request counts do not imply equal load when requests have different costs; long-lived connections, cache locality, session affinity, and heterogeneous node capacity all change the distribution. Least-connections, weighted routing, and latency-aware selection therefore suit different workloads.

A balancer does not create capacity. If aggregate arrival rate exceeds downstream service capacity, the system still needs backpressure, admission control, or load shedding. Health checks should also represent usable service capacity rather than merely whether a process is alive.