How Retrieved Context Shapes Internal Representations in RAG

Samuel Yeh, Sharon Li · 2026

Read How Retrieved Context Shapes Internal Representations in RAG by Samuel Yeh, Sharon Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20091.txt.

Description

Retrieval-augmented generation feeds language models external documents. Retrieved sets mix useful and irrelevant material, and how that mix changes the model's inner workings is poorly understood. A 2026 arXiv paper by Samuel Yeh and Sharon Li studies retrieval-augmented generation through latent states rather than outputs alone.

The authors methodically analyze how different kinds of retrieved documents shift the hidden states of language models and how those inner shifts relate to downstream output behaviour. They worked across four question-answering datasets and three language models, under controlled single-document and multi-document settings. They trace how context fit and layer-by-layer processing shape internal states. The results explain aspects of model outputs in terms of representation shifts and offer guidance for retrieval-based generation system design.

Catalog record

CREATORSamuel Yeh, Sharon Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-20091

Browse the full library · About the Tower of Babel