Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation

Junru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis · 2025

Read Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation by Junru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04820.txt.

Description

Asking a language model to compare every pair of documents ranks them superbly, at a cost that grows with the square of the list. Junru Wu and seven co-authors distill that pairwise wisdom into a fast single-pass ranker in 2025.

The teacher labels come from pairwise prompting, and a pointwise student learns to reproduce the order. The surprise is thrift: training on just two percent of all pairs matched the performance of using every pair. The result keeps the expensive method's ranking quality while serving at ordinary cost. For search builders, the work turns a laboratory trick into something deployable, and it questions how much comparison data distillation really needs.

Catalog record

CREATORJunru Wu, Le Yan, Zhen Qin, Honglei Zhuang, Paul Suganthan G. C, Tianqi Liu, Zhe Dong, Xuanhui Wang, Harrie Oosterhuis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-04820

Browse the full library · About the Tower of Babel