Do Neural Retrievers Prefer Certain Documents? Evidence of Learned Relevance Priors

Francisco Valentini, Edgar Altszyler, Martin Fajcik · 2026

Read Do Neural Retrievers Prefer Certain Documents? Evidence of Learned Relevance Priors by Francisco Valentini, Edgar Altszyler, Martin Fajcik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02814.txt.

Description

Valentini and co-authors show that neural retrievers learn to prefer some documents before any query arrives. Retrievers train on annotated query-document pairs. Labeling covers only a subset of documents. That selection can favor certain document types. The team tests whether a relevance prior forms that ignores the query. Simple classifiers train on frozen document embeddings to estimate it. Three top retrievers are evaluated across several benchmarks.

The priors carry over to unseen documents. They are consistent across models. A findability gap results. Low-prior documents are always harder to retrieve, even when truly relevant. The effect is weaker and less consistent in BM25. Controlled matched-document comparisons confirm it persists. Language-model explanations characterize the favored documents. They tend to be full, self-contained summaries of mainstream topics. Niche, fragmentary, or technical content is often left unjudged. Retrievers internalize that bias from their training data.

Catalog record

CREATORFrancisco Valentini, Edgar Altszyler, Martin Fajcik
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.02814

Browse the full library · About the Tower of Babel