Queueing Delay
The time a unit of work spends waiting for a processor, worker, connection, or other service resource before execution actually begins.
End-to-end latency does not tell whether work itself is slow or merely waiting. As utilization approaches capacity, service time may remain nearly unchanged while requests accumulate behind workers, GPUs, database connections, or storage queues.
Queueing delay should therefore be measured separately from service time. In real-time systems it consumes the same deadline budget as computation, and near saturation it can grow much faster than workload size alone suggests. Queue stability in ASR systems and capacity engineering for real-time speech recognition provide concrete examples.