Quantization
The representation of model weights, activations, or intermediate values with reduced numerical precision to decrease memory use and often improve inference speed.
Boundaries
Quantization reduces numerical precision and resource cost but is distinct from pruning, distillation, or architectural compression.
Related Concepts
- BF16
- INT8
- Inference
- Product Quantization