Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders
Benjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu · 2026
Read Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders by Benjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05588.txt.
Description
Benjamin Rozonoyer and 7 co-authors develop, in a 2026 paper, autoregressive ranking as a middle path between two classic search architectures. Dual encoders score documents fast but coarsely, while cross encoders are accurate yet expensive. Autoregressive ranking instead has a language model generate document identifiers token by token, enabling ranked retrieval through beam search.
The authors first prove this approach is strictly more expressive than dual encoders, needing only constant hidden size where dual encoders need dimension growing with corpus size. They then propose a rank-aware training loss that spreads probability over valid identifier tokens according to true relevance. Experiments on WordNet and ESCI data showed fewer invalid identifiers and clearly better ranking metrics.
Catalog record
| CREATOR | Benjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05588