BicKD: Bilateral Contrastive Knowledge Distillation
Jiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu · 2026
Read BicKD: Bilateral Contrastive Knowledge Distillation by Jiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01265.txt.
Description
Jiangnan Zhu and co-authors build BicKD, a two-sided contrast teach plan for model distillation. Plain distillation lines up teacher and student odds case by case but skips class-wise contrasts and sets no shape rule on the odds space. BicKD adds a two-sided contrast loss that pushes unlike class zones apart while holding like-class zones tight. That twin form lets teacher and student compare both case-wise and class-wise odds at once. Tests show BicKD beating top distillation plans across net forms and test sets.
Catalog record
| CREATOR | Jiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34459 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01265