Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG
Agam Goyal, Koyel Mukherjee, Apoorv Saxena, Anirudh Phukan, Eshwar Chandrasekharan, Hari Sundaram · 2026
Read Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG by Agam Goyal, Koyel Mukherjee, Apoorv Saxena, Anirudh Phukan, Eshwar Chandrasekharan, Hari Sundaram (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06097.txt.
Introduction
Goyal and colleagues' 2026 paper presents the first systematic study of query rewriting effects on dense retriever biases in RAG. Dense retrievers show systematic biases. The team tests five methods across six retrievers. Simple LLM-based rewriting cuts bias most, by 54 percent, yet fails when biases combine adversarially. Simple rewriting cuts bias by raising score variance, while pseudo-document generation cuts it through decorrelation from bias cues. No technique fixes every bias, and effects differ across retrievers. A taxonomy splits query-document interaction biases from document encoding biases.
Catalog record
| CREATOR | Agam Goyal, Koyel Mukherjee, Apoorv Saxena, Anirudh Phukan, Eshwar Chandrasekharan, Hari Sundaram |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43784 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06097
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG Agam Goyal, Koyel Mukherjee, Apoorv Saxena, Anirudh Phukan, Eshwar Chandrasekharan, Hari Sundaram Abstract Dense retrievers in retrieval-augmented generation (RAG) systems exhibit systematic biases—including brevity, position, literal matching, and repetition biases—that can compromise retrieval quality. Query rewriting techniques are now standard in RAG pipelines, yet their impact on these biases remains unexplored. We present the first systematic study of how query enhancement techniques affect dense retrieval biases, evaluating five methods across six retrievers. Our findings reveal that simple LLM-based rewriting achieves the strongest aggregate bias reduction (54%), yet fails under adversarial conditions where multiple biases combine. Mechanistic analysis uncovers two distinct mechanisms: simple rewriting reduces bias through increased score variance, while pseudo-document generation methods achieve reduction through genuine decorrelation from bias-inducing features. However, no technique uniformly addresses all biases, and effects vary substantially across retrievers. Our results provide practical guidance for selecting query enhancement strategies based on specific bias vulnerabilities. More broadly, we establish a taxonomy distinguishing query-document interaction biases from document encoding biases, clarifying the limits of query-side interventions for debiasing RAG systems. $\ast$ $\ast$ footnotetext: Work done during internship at Adobe Research $\ddagger$ $\ddagger$ footnotetext: Corresponding Authors 1 Introduction Retrieval-Augmented Generation (RAG) systems have become foundational for grounding large language models in external knowledge, enabling applications from question-answering to document analysis across diverse domains 18 ; 21 ; 15 ; 7 ; 1 . These systems typically employ dense retrievers 16 ; 34 to identify relevant passages from document collections, which are then provided as context for response generation 24 . Figure 1: Paper Overview. Dense retrievers are susceptible to various biases, leading to incorrect retrieval and erroneous LLM outputs. Can query rewriting help? We systematically evaluate its effects on retrieval bias. Despite their effectiveness, recent work has revealed that dense retrievers exhibit systematic biases that can compromise retrieval quality 26 ; 4 . 5 demonstrated that retrievers consistently favor documents with surface-level characteristics—brevity, early answer positioning, lexical overlap, and entity repetition—over documents containing actual factual evidence. The ideal solution would be dense retrieval models inherently robust to such biases and capable of prioritizing semantic relevance. However, achieving this through architectural modifications or training objective changes remains challenging. A more tractable first step is understanding whether these biases persist under query enhancement techniques 22 , which are standard in modern RAG pipelines. Production RAG systems routinely employ query transformations including LLM-based rewriting for better semantic matching and pseudo-document generation approaches like HyDE 6 and Query2Doc 36 . While these techniques improve retrieval effectiveness, their interaction with systematic biases remains unexplored: do they amplify existing biases by generating surface-level features that retrievers favor, or do specific strategies mitigate particular bias types? Understanding this interaction is critical for deploying robust and fair RAG systems, particularly in high-stakes domains such as medical, sociotechnical, educational, and enterprise settings 28 ; 35 ; 8 ; 31 ; 40 ; 9 ; 25 . In this work, we conduct the first systematic investigation into how query rewriting techniques affect retrieval biases . We evaluate five enhancement methods across six retrievers using the controlled framework of 5 . This framework’s treatment-control methodology, which uses paired documents that differ only in specific surface characteristics while maintaining factual equivalence, enables us to isolate and measure the effect of query rewriting on individual bias types rather than confounding bias reduction with overall retrieval improvements. We find that query enhancement effects are highly differential: simple LLM rewriting achieves the strongest aggregate bias reduction (54%) but fails under adversarial conditions, while pseudo-document methods provide more modest but mechanistically robust improvements. We establish a taxonomy distinguishing interaction biases amenable to query-side interventions from encoding biases requiring retriever-level modifications, providing practical guidance for bias-aware RAG deployment. 2 Related Works Bias in Dense Retrieval. Dense retrieval has emerged as a powerful alternative to traditional sparse methods like BM25 for information retrieval tasks 16 ; 34 , showing strong performance on retrieval benchmarks. However, despite these advances, recent work has begun to uncover systematic biases in how dense retrievers encode and rank documents, including position biases 4 and lexical biases 26 . Similarly, 2 show that Dense Passage Retrieval (DPR) models fail to retrieve statements requiring reasoning beyond surface-level similarity. In terms of downstream consequences of these biases, 11 and 29 show how these biases can affect the coverage, presentation, and fairness of the outputs from generative search systems. Most relevant to our work, 5 introduced ColDeR, a controlled benchmark for systematically measuring various types of retrieval biases that biases persist across multiple dense retriever architectures and can compound adversarially. Our work extends this line of research by investigating whether and how query-side interventions can mitigate these documented biases. Query Enhancement in RAG. RAG systems 18 ; 7 have become foundational for knowledge-intensive NLP tasks, with query rewriting now a standard pipeline component 22 . Several query enhancement techniques have been proposed to bridge the semantic gap between queries and documents. HyDE 6 generates pseudo-documents and retrieves based on their embeddings rather than the original query; Query2Doc 36 concatenates LLM-generated pseudo-documents to expand queries; and various other simple LLM-based query rewriting techniques have been proposed in prior work 14 ; 23 ; 17 . While these methods improve retrieval effectiveness, their impact on systematic retrieval biases remains unexplored. As RAG systems are increasingly deployed in high-stakes domains, understanding how retrieval biases propagate through these pipelines becomes critical. Our work fills this gap by analyzing how query enhancement techniques interact with known retrieval biases. 3 Preliminaries, Data, and Methods 3.1 Problem Formulation We investigate whether and how query rewriting techniques affect systematic biases in dense retrievers. We adopt the treatment-control ColDeR benchmark contributed by 5 , which uses carefully constructed document pairs using the Re-DocRED dataset 32 that differ only in specific surface characteristics while maintaining factual equivalence. Given a query q and a pair of documents (D_{1},D_{2}) where D_{1} contains a bias-inducing characteristic and D_{2} serves as the control, we measure retrieval preferences using the similarity scores assigned by a dense retriever \mathcal{S} . This controlled design enables us to isolate the effect of query rewriting on individual bias types, separating bias mitigation from general retrieval performance improvements. Then, for each document pair, we compute the difference in retrieval scores: \displaystyle\delta=\mathcal{S}(q,D_{1})-\mathcal{S}(q,D_{2}) where \mathcal{S}(q,D) represents the retriever’s similarity score between the generated embeddings for the query q and document D . Positive \delta values indicate bias toward surface-level characteristics, while values near zero suggest factual grounding. We apply different query enhancement techniques to transform the original query q into q^{\prime} , and measure how this transformation affects the distribution of \delta across query-document pairs. For each bias type, we collect N=200 paired documents \{(D_{1},D_{2})\}_{i}^{N} along with associated queries. We now describe each bias type in detail. Document D_{1} (Bias-amplified) Document D_{2} (Control) Brevity Bias Q: What series is Lost Verizon part of? D: “ Lost Verizon ” is the second episode of The Simpsons ’ twentieth season. Q: What series is Lost Verizon part of? D: “ Lost Verizon ” is the second episode of The Simpsons ’ twentieth season. It first aired on the Fox network in the United States on