Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders

Benjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu · 2026

Read Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders by Benjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05588.txt.

Description

Benjamin Rozonoyer and 7 co-authors develop, in a 2026 paper, autoregressive ranking as a middle path between two classic search architectures. Dual encoders score documents fast but coarsely, while cross encoders are accurate yet expensive. Autoregressive ranking instead has a language model generate document identifiers token by token, enabling ranked retrieval through beam search.

The authors first prove this approach is strictly more expressive than dual encoders, needing only constant hidden size where dual encoders need dimension growing with corpus size. They then propose a rank-aware training loss that spreads probability over valid identifier tokens according to true relevance. Experiments on WordNet and ESCI data showed fewer invalid identifiers and clearly better ranking metrics.

Catalog record

CREATORBenjamin Rozonoyer, Chong You, Michael Boratko, Himanshu Jain, Nilesh Gupta, Srinadh Bhojanapalli, Andrew McCallum, Felix Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05588

Browse the full library · About the Tower of Babel