Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers
Wei Huang, Keping Bi, Yinqiong Cai, Wei Chen, Jiafeng Guo, Xueqi Cheng · 2026
Read Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers by Wei Huang, Keping Bi, Yinqiong Cai, Wei Chen, Jiafeng Guo, Xueqi Cheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06163.txt.
Introduction
Huang and five colleagues trace retriever bias toward machine-written texts to training data. Neural retrievers often rank LLM-generated texts above human-written ones, and the bias runs sharpest in supervised systems. Annotated positives and negatives differ in traits like fluency and word specificity, the same split seen between machine and human text, so retrievers learn artifact shortcuts for relevance. Theory under contrastive learning confirms such artifacts can enter scores. They counter with balanced negative sampling in training and removal of the artifact-aligned part at inference.
Catalog record
| CREATOR | Wei Huang, Keping Bi, Yinqiong Cai, Wei Chen, Jiafeng Guo, Xueqi Cheng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77425 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06163
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers Wei Huang, Keping Bi, Yinqiong Cai, Wei Chen, Jiafeng Guo, Xueqi Cheng Abstract Recent studies show that neural retrievers often display source bias, favoring passages generated by LLMs over human-written ones, even when both are semantically similar. This bias has been considered an inherent flaw of retrievers, raising concerns about the fairness and reliability of modern information access systems. Our work challenges this view by showing that source bias stems from supervision in retrieval datasets rather than the models themselves. We found that non-semantic differences, like fluency and term specificity, exist between positive and negative documents, mirroring differences between LLM and human texts. In the embedding space, the bias direction from negatives to positives aligns with the direction from human-written to LLM-generated texts. We theoretically show that retrievers inevitably absorb the artifact imbalances in the training data during contrastive learning, which leads to their preferences over LLM texts. To mitigate the effect, we propose two approaches: 1) reducing artifact differences in training data and 2) adjusting LLM text vectors by removing their projection on the bias vector. Both methods substantially reduce source bias. We hope our study alleviates some concerns regarding LLM-generated texts in information access systems. 1 Introduction The rapid rise of large language models (LLMs) has reshaped the information landscape, creating corpora where human-written and LLM-generated texts coexist. Within this hybrid ecosystem, an emerging phenomenon has been observed: neural retrievers often prefer LLM-generated passages over semantically similar human-written ones, a phenomenon known as source bias ( 8 ; 10 ) . This bias raises concerns at multiple levels. For users, it risks diminishing search quality by ranking fluent but less relevant or even misleading LLM outputs above more relevant human-authored content. For human creators, it undermines fairness by systematically downranking their work and reducing its visibility. At the ecosystem level, it may amplify LLM-generated text through self-reinforcing feedback loops, further marginalizing human contributions ( 3 ; 41 ) . Given these significant concerns, understanding the root cause of source bias is crucial. Prior work offers different explanations: 8 attribute the bias to architectural similarities between retrievers built on pretrained language models (PLMs) and LLMs, while 33 argue that retrievers prefer low-perplexity texts, a property often exhibited by LLM outputs. However, it remains unclear why such preferences emerge, and no explanation has been widely accepted. Consequently, recent efforts have shifted toward mitigating source bias, for example, through causal debiasing to reduce the impact of perplexity ( 33 ) or by aligning LLM outputs to be less biased for retrievers ( 9 ) . In this paper, we aim to uncover the root cause of source bias in neural retrievers. Specifically, we address three research questions (RQs): • RQ1: Is source bias a general property of neural retrievers? Beyond the commonly studied retrievers trained on MS MARCO ( 27 ) , we examine two additional families: (1) general-purpose embedding models trained for diverse tasks such as clustering, classification, semantic similarity, and retrieval, and (2) unsupervised retrievers trained without relevance annotations, such as Contriever ( 20 ) and SimCSE ( 15 ) . We find that these models exhibit only mild source bias, whereas fine-tuning the unsupervised retrievers on MS MARCO induces severe bias. This suggests that source bias is not inherent to neural retrievers but is largely introduced through relevance supervision. • RQ2: Why does relevance supervision induce source bias? Our analysis of 14 retrieval datasets uncovers systematic non-semantic differences between positive and negative documents, including variations in fluency, as measured by perplexity, and lexical specificity. These differences closely mirror the distinctions between LLM-generated and human-authored texts. In the embedding space, we further observe that the bias direction from negatives to positives aligns strongly with the direction from human-written to LLM-generated texts. Theoretical analysis confirms that retrievers trained with contrastive losses inevitably absorb these imbalances. • RQ3: How can source bias be mitigated? We propose two mitigation strategies: (1) reducing artifact differences in training data to prevent retrievers from encoding non-semantic factors, and (2) debiasing embeddings by subtracting the projection of LLM-generated vectors on the bias direction. Both approaches substantially reduce source bias, confirming that it originates from systematic imbalances in relevance annotations. In summary, we challenge the prevailing view that neural retrievers are inherently biased toward LLM-generated texts. Instead, we show that source bias arises from artifact imbalances in retrieval datasets rather than model architecture. Our findings highlight two complementary pathways for mitigation: curating training data to minimize non-semantic artifacts and explicitly decoupling artifact effects in retrievers. With a deeper understanding of source bias, LLM-generated texts need not be regarded as inherently problematic. We hope this study alleviates concerns about their use and fosters a more objective perspective on integrating LLM-generated data into retrieval systems. 2 Related Work Source Bias in Information Retrieval. 10 revealed that neural retrievers exhibit a clear preference for LLM-generated passages even when their semantic content is similar to human-written ones, a phenomenon termed source bias . Cocktail ( 7 ) further established a benchmark to evaluate this phenomenon across diverse retrieval datasets systematically. Similar effects have also been noted in related IR scenarios, including multimodal retrieval ( 39 ) , recommender systems ( 41 ) , and retrieval-augmented generation ( 3 ) , underscoring the view that source bias is a broad challenge in the LLM era. Mechanisms and Mitigation. Prior work has examined both explanations and mitigations for source bias. Early studies linked it to architectural similarity between PLMs and LLMs ( 10 ) . 33 showed that PLM-based retrievers overrate low-perplexity documents, and 8 framed the issue more broadly as a distribution mismatch. Mitigation approaches include retriever-side methods such as causal debiasing ( 33 ) and LLM-side methods like LLM-SBM ( 9 ) . Following these perspectives, prior work has often assumed that source bias is a universal property of neural retrievers. By contrast, we evaluate a broader spectrum of retrievers and show that source bias is not inherent to neural retrievers. We further develop a retriever-centric theory and conduct a set of experiments indicating that the bias largely arises from supervision, and we provide practical mitigations. 3 RQ1: Is Source Bias a General Property of Neural Retrievers The previously discussed phenomenon of source bias ( 8 ; 10 ) has been mainly observed in retrieval-supervised models, which are trained on relevance-labeled datasets such as MS MARCO ( 27 ) . This observation prompts us to examine whether source bias is a general property of neural retrievers or a phenomenon largely induced by relevance supervision. We therefore design two controlled experiments to disentangle the role of supervision from model architecture: (1) we examine whether source bias persists in models beyond those primarily finetuned on retrieval datasets, considering both general-purpose embedding models and unsupervised retrievers; and (2) we assess the impact of retrieval supervision by fine-tuning several unsupervised retrievers on MS MARCO while holding architecture fixed. Next, we present the model families, datasets, and metrics used in these experiments. 3.1 Experimental Setup Model Families. We evaluate three distinct families of models: (A) Relevance-Supervised Retrievers , trained with direct or distilled supervision signals derived from large-scale human relevance annotations (e.g., MS MARCO), including