Knowledge Distillation
A model-compression technique in which a smaller student model is trained to reproduce informative outputs or representations from a larger teacher model.
Related Concepts
- Model Compression
- Teacher Model
- Student Model
- Quantization