Model Optimization

Understand Knowledge Distillation for Smaller AI Models

A large model may produce useful predictions but still be too expensive or slow for the environment where it must run. A mobile application, an edge device, or a high-volume service can have tighter limits on memory, latency, and compute. Knowledge distillation is one way to address that gap. Instead of training a smaller model only from the original labels, we also train it to imitate information produced by a stronger teacher model. The smaller model is called the student.