Knowledge distillation is a machine-learning training method where a smaller “student” model learns from the outputs of a larger “teacher” model. Instead of copying the teacher’s internal weights, the student is trained to imitate its predictions, rankings, or generated text. This can preserve useful behavior while reducing compute cost, memory use, and latency.
In cyber security, distillation matters because model behavior can be transferred without stealing the original system. An attacker may query a high-end AI service many times, collect responses, and train a local model to mimic its capabilities. That can create a cheaper copy of coding, analysis, or decision-making behavior. Defenders also use distillation to deploy smaller models for detection, filtering, and on-device security tools. Because the technique is normal and legitimate, security teams focus on abuse signals such as large-scale querying, repeated prompt patterns, and output harvesting.