Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation
Junru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis · 2025
Read Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation by Junru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04820.txt.
Description
Asking a language model to compare every pair of documents ranks them superbly, at a cost that grows with the square of the list. Junru Wu and seven co-authors distill that pairwise wisdom into a fast single-pass ranker in 2025.
The teacher labels come from pairwise prompting, and a pointwise student learns to reproduce the order. The surprise is thrift: training on just two percent of all pairs matched the performance of using every pair. The result keeps the expensive method's ranking quality while serving at ordinary cost. For search builders, the work turns a laboratory trick into something deployable, and it questions how much comparison data distillation really needs.
Catalog record
| CREATOR | Junru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04820