SEAD: Competence-aware on-policy distillation via entropy-guided supervision
A framework which uses entropy as a unified probe of this competence-dependent degradation at three scales.
Topics:
On-policy distillation (OPD) has a property absent in offline distillation and RL: teacher supervision quality depends on student competence. Incoherent rollouts yield noisy gradients; already-mastered tokens yield redundant ones. This creates waste at three scales—tokens, training phases, and prompts—yet existing methods supervise uniformly. We introduce SEAD, which uses entropy as a unified probe of this competence-dependent degradation at three scales: (1) joint teacher–student entropy partitions tokens into zones receiving tailored divergences or zero gradient (∼50% skipped); (2) a cosine schedule anneals from forward to reverse KL as competence grows; (3) a competence-gated curriculum introduces prompts easy-to-hard. These components are symbiotically necessary: token selection requires coherent rollouts (curriculum), annealing requires monotonic improvement (also curriculum). On OLMo-3 (7B→32B), SEAD achieves +4.8 avg accuracy over vanilla OPD across six math benchmarks, with ablations confirming super-additive interactions.
Latest publications
Ask, don’t judge: Binary questions for interpretable LLM evaluation and self-improvement
A framework that decomposes evaluation criteria into atomic binary questions and aggregates the resulting verdicts.
ICMLStress tests REVEAL fragile temporal and visual grounding in video-language models
A diagnostic benchmark that probes fundamental weaknesses of contemporary VidLMs through five controlled stress tests.
ICMLCritique-guided distillation for robust reasoning via refinement
A training framework that decouples critique consumption from critique generation.
ICML