Knowledge distillation

Made to be adapted.

Temperature changes both teacher and student probabilities. The outline bars show the teacher; filled bars show the student. A higher temperature exposes relative probability away from the top outcome.

A six-outcome mathematical model with exact updates. No language model is being trained.