INT8
An 8-bit integer numerical representation commonly used in quantized inference to reduce model memory traffic and increase arithmetic throughput on supported hardware.
Quantized Execution
INT8 inference represents activations and/or weights with 8-bit integers plus scale/zero-point or related calibration parameters. The goal is usually lower memory bandwidth and more efficient matrix operations.
The benefit depends on hardware support and runtime kernels; changing the stored datatype alone does not guarantee faster inference.
Accuracy Boundary
Quantization changes numerical representation. Calibration data and the sensitivity of specific layers can affect accuracy, especially when value distributions contain outliers.
Benchmarking should therefore include both task metrics and end-to-end latency/throughput.