Distillation that keeps confidence honest

Traditional on-policy distillation (OPD) causes smaller student models to exhibit overconfidence due to the larger teacher model's access to privileged context. New research formalizes this mismatch and proposes CaOPD to rectify this certainty illusion without sacrificing accuracy gains.