Inference Engine
A runtime optimized to execute trained models efficiently by managing graph execution, kernels, memory, batching, device placement, and numerical precision.
Related Concepts
- ONNX
- Operator Fusion
- Quantization
- Tensor
A runtime optimized to execute trained models efficiently by managing graph execution, kernels, memory, batching, device placement, and numerical precision.