Skill-Conditioned Gated Self-Distillation for LLM Reasoning
Jiazhen Huang, Xiao Chen, Xiao Luo, Yong Dai, Senkang Hu, Yuzhi Zhao · 2026
Read Skill-Conditioned Gated Self-Distillation for LLM Reasoning by Jiazhen Huang, Xiao Chen, Xiao Luo, Yong Dai, Senkang Hu, Yuzhi Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28791.txt.
Description
Jiazhen Huang and five colleagues propose Skill-Conditioned Gated Self-Distillation, a method letting LLM reasoning improve from a skill bank rather than reference answers. Retrieved skills are compact but may mislead, so the method treats distillation as teacher validation, not blind imitation. Skill-mistake pairs feed a multi-teacher pool where skill teachers score the same student rollout. The verifier checks each teacher's polarity. A robust gated objective distills useful teacher-student disagreements, suppressing uncertain or extreme signals. It beats GRPO and rivals answer-conditioned methods on math benchmarks.
Catalog record
| CREATOR | Jiazhen Huang, Xiao Chen, Xiao Luo, Yong Dai, Senkang Hu, Yuzhi Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55965 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28791