Knowledge Distillation
Also called: Model Distillation, Teacher-Student Learning
A model optimization technique in which a smaller AI model, called the student model, is trained to mimic the behavior or outputs of a larger, more capable teacher model, reducing model size and inference cost while preserving much of the original performance.
Explore more about Optimization
Related terms
The process of reducing the precision of a model's weights and activations to lower memory footprint and speed up inference.
Fine-tuningThe process of further training a pretrained AI model on a task-specific or domain-specific dataset to improve its performance, behavior, or specialization for particular use cases.
Inference OptimizationThe process of improving the speed, latency, throughput, memory usage, or cost-efficiency of AI model inference through techniques such as quantization, pruning, batching, caching, hardware acceleration, and optimized model serving.