Distillation versus Contrastive Learning: How to Train Your Rerankers
Zhichao Xu Zhiqi Huang Shengyao Zhuang Vivek Srikumar · 2025
Read Distillation versus Contrastive Learning: How to Train Your Rerankers by Zhichao Xu Zhiqi Huang Shengyao Zhuang Vivek Srikumar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08336.txt.
Description
Zhichao Xu, Zhiqi Huang, Shengyao Zhuang, and Vivek Srikumar compare two ways to train text rerankers, in a research paper from 2025. Rerankers sort search results and matter for retrieval quality. One way learns from true labels, and the other learns from a larger teacher model, but a fair head-to-head test under real conditions was missing.
The team trained rerankers of four sizes and two designs both ways, on the same data. Learning from a stronger teacher won on in-domain and new-domain tests in most cases. A teacher of equal strength gave no such gain. The guidance is plain: use a larger teacher if one is on hand.
Catalog record
| CREATOR | Zhichao Xu Zhiqi Huang Shengyao Zhuang Vivek Srikumar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-08336