On the impact of retrieved content representations in RAG pipelines
Jonathan J Ross, Bevan Koopman, Anton van der Vegt, Guido Zuccon · 2026
Read On the impact of retrieved content representations in RAG pipelines by Jonathan J Ross, Bevan Koopman, Anton van der Vegt, Guido Zuccon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30790.txt.
Introduction
Ross and colleagues study how retrieved content should be represented for a language model rather than a human reader. Holding retrieval fixed, they vary only document representation across thirteen transformations. Across fourteen representations, they measure accuracy for four generators, plus answer retention: whether an answer-bearing document still supports its answer after transformation. They find answer retention drives generator accuracy; when retention is high, wording, structure, length, and query-dependence have limited effect. Gains attributed to specific mechanisms may reflect how well they preserve answer-bearing content.
Catalog record
| CREATOR | Jonathan J Ross, Bevan Koopman, Anton van der Vegt, Guido Zuccon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78461 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30790
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the impact of retrieved content representations in RAG pipelines Thanks: Code available at https://github.com/ielab/rag-representation Jonathan J Ross, Bevan Koopman, Anton van der Vegt, Guido Zuccon Abstract Retrieval-Augmented Generation (RAG) supplements a language model’s input with retrieved documents, yet most RAG pipelines inherit retrieval components designed for human readers. How retrieved content should be represented when the consumer is a large language model (LLM) rather than a human is less well understood. Recent work has proposed transformations of retrieved content and identified properties that affect generation, but each examines a single transformation or property in isolation, leaving open which features of a document’s representation matter most. We address this with a controlled comparison: holding retrieval fixed, we vary only the representation of retrieved documents, comparing an original baseline against thirteen transformations spanning selection, summarisation, and reformulation, in query-dependent and query-independent variants. Across these fourteen representations we measure question-answering accuracy for four generators, and for each representation we also measure answer retention: whether a known answer-bearing document still supports its answer after transformation. We find that answer retention is the primary determinant of generator accuracy; notably, when retention is high, a representation’s wording, structure, length, and query-dependence have limited effect. This suggests that accuracy gains attributed to specific mechanisms in prior work may be partly explained by how well those mechanisms preserve answer-bearing content, an attribution that cannot be settled without controlling for retention. 1 Introduction Retrieval-Augmented Generation (RAG) supplements the input of a large language model (LLM) with external documents sourced by a retrieval system 16 ; 8 . RAG has enabled strong performance on knowledge-intensive tasks 10 ; 29 , improved answer attribution 2 , and allowed models to incorporate new knowledge without retraining. Most RAG systems adopt retrieval components built to serve human users, and many of their design choices reflect that origin: ranked result lists 11 , query-biased snippets 28 , and query-term highlighting 9 all help a human reader locate and consume relevant information. These design choices are grounded in decades of study of how human readers process search results, and RAG pipelines have largely inherited them. What works best when the consumer is an LLM is less well understood, but there are consequential differences. For example, LLMs are sensitive to changes in input text that would not be meaningful to a human reader: 24 found that prompts differing in minor formatting choices such as separators or capitalisation produced accuracy differences of up to 76 points. Unrelated input affects answer generation in unintuitive ways: 7 found that randomly sampled passages can actually improve RAG accuracy. And beyond accuracy, the cost of processing retrieved content scales superlinearly with its length, giving practical weight to representation choices that a human consumer would not require. If LLMs process retrieved content differently from humans, the representation in which that content is presented to them — how it is worded and structured, and not just what information it contains — becomes a design choice worth examining directly. Recent work has begun to study how the representation of retrieved content affects downstream generation. Recursive clustering and summarisation of passages produces a tree spanning multiple levels of abstraction ( 23 ) ; post-retrieval compression reduces input length while preserving accuracy ( 30 ) ; and document rewriting optimises retrieved content for generative utility rather than retrieval relevance ( 13 ) . A separate line of work has identified specific properties of retrieved content that affect generator behaviour, including query similarity and semantic completeness ( 25 ) , and relevance combined with logical interconnectivity ( 3 ) . These studies demonstrate that representation matters for downstream performance, but each examines a single technique or property in isolation. What is missing is a controlled comparison: varying the representation of retrieved content, holding other factors fixed, to identify which properties matter most when the consumer is an LLM rather than a human. We address this gap by treating the representation as a design choice and studying it directly. Holding retrieval fixed, we vary only the representation of retrieved content via a set of document representation transformations spanning selection, summarisation, and reformulation, with both query-dependent and query-independent variants. We then examine the effect on answer accuracy. For each representation, we also measure answer retention: whether a known answer-bearing document still contains a valid answer after transformation. We find that answer retention is the primary determinant of generator accuracy. Other aspects of the representation — its wording, structure, length, and whether it incorporates the query — have limited effect when retention is high, suggesting that recent gains attributed to specific representation mechanisms may be partly explained by how well those mechanisms preserve answer-bearing content. Our contributions are as follows: • We conduct a controlled comparison of fourteen document representations—an original baseline and thirteen transformations spanning selection, summarisation, and reformulation—holding retrieval fixed to isolate the effect of representation on generator accuracy. • We find that answer retention is the primary determinant of generator accuracy across representations. • We show that query-dependent representations do not systematically outperform query-independent ones. • We find that generators do not prefer LLM-produced transformations over non-LLM ones; apparent preferences track answer retention rather than transformation source. • We argue that accuracy gains cannot be attributed to specific representation mechanisms without controlling for answer retention, which prior work has not generally done. Our findings provide empirical grounding for the design of retrieval systems that serve LLMs rather than humans. 2 Related Work A growing body of work asks how the representation of retrieved content affects generator accuracy in RAG. Two directions have emerged. The first, transformation-based, proposes specific transformations of retrieved content and measures their effect on accuracy. The second, property-based, characterises which properties of retrieved content generators are sensitive to. Transformation-based studies intervene at one of two points in the pipeline: at indexing, by changing what gets stored and retrieved; or post-retrieval, by transforming retrieved content before it reaches the generator. At indexing, 4 use atomic propositions rather than passages, arguing that propositions are a more effective unit of retrieval granularity, and RAPTOR 23 builds hierarchical summaries to make different levels of abstraction retrievable. Post-retrieval, HtmlRAG 26 preserves HTML structure rather than flattening retrieved content to plain text, arguing that structural information aids the generator. Compression methods 30 ; 19 ; 18 reduce retrieved content to lower inference cost, finding that generators tolerate substantial reductions with comparable or only slightly degraded accuracy. Finally, rewriting methods learn to transform retrieved content for downstream generator utility rather than for retrieval relevance 13 ; 17 . Each of these reports an accuracy effect — a gain, or preservation under compression — and attributes it to the proposed mechanism, but does not isolate which properties of the resulting representation drive that effect. Several properties of retrieved content shape generator behaviour. Utility — whether a passage actually improves a generator’s answer — is a property distinct from retriever-side relevance 27 , and one that varies by generator: 31 show that passages maximising utility for one generator are consistently suboptimal for another. Logical connectivity across retrieved passages also matters: 3 find that LLMs prefer external knowledge that is both relevant and logically connected. Irrelevance also matters: 7 find that distracting irrelevance