Uni-OPD : Unifying On-Policy Distillation with a Dual-Perspective Recipe
Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, [2pt] Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, [2pt] Han Hu, Yi Yang, Fei Wu, Hehe Fan, [2pt] Zhejiang University · 2026
Read Uni-OPD : Unifying On-Policy Distillation with a Dual-Perspective Recipe by Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, [2pt] Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, [2pt] Han Hu, Yi Yang, Fei Wu, Hehe Fan, [2pt] Zhejiang University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03677.txt.
Description
Wenjin Hou and fifteen co-authors propose Uni-OPD, a unified framework for on-policy distillation, a post-training method that merges specialized expert models into one student model. Distillation on the student's own generated states works well in practice, but the conditions for reliable improvement were poorly understood. The authors identify two bottlenecks: too little exploration of useful states, and unreliable teacher supervision.
Their dual-perspective recipe balances training data so the student explores informative states it generates itself. On the teacher side, an outcome-guided margin calibration keeps token-level guidance consistent with final rewards, separating correct from incorrect paths. The framework covers both language and multimodal models, tested across five domains and sixteen benchmarks.
Catalog record
| CREATOR | Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, [2pt] Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, [2pt] Han Hu, Yi Yang, Fei Wu, Hehe Fan, [2pt] Zhejiang University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03677