BicKD: Bilateral Contrastive Knowledge Distillation

Jiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu · 2026

Read BicKD: Bilateral Contrastive Knowledge Distillation by Jiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01265.txt.

Description

Jiangnan Zhu and co-authors build BicKD, a two-sided contrast teach plan for model distillation. Plain distillation lines up teacher and student odds case by case but skips class-wise contrasts and sets no shape rule on the odds space. BicKD adds a two-sided contrast loss that pushes unlike class zones apart while holding like-class zones tight. That twin form lets teacher and student compare both case-wise and class-wise odds at once. Tests show BicKD beating top distillation plans across net forms and test sets.

Catalog record

CREATORJiangnan Zhu, Yukai Xu, Li Xiong, Yixuan Liu, Junxu Liu, Hong kyu Lee, Yujie Gu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34459
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01265

Browse the full library · About the Tower of Babel