“Lost - in - the - Later” : Framework for Quantifying Contextual Grounding in Large Language Models
Yufei Tao, Adam Hiatt, Rahul Seetharaman, Ameeta Agrawal, {yutao, ahiatt, ameeta}@pdx.edu, [email protected] · 2025
Read “Lost - in - the - Later” : Framework for Quantifying Contextual Grounding in Large Language Models by Yufei Tao, Adam Hiatt, Rahul Seetharaman, Ameeta Agrawal, {yutao, ahiatt, ameeta}@pdx.edu, [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05424.txt.
Introduction
Yufei Tao, Adam Hiatt, Rahul Seetharaman, and Ameeta Agrawal measure how language models weigh a provided passage against what they already know, in a paper published in two thousand twenty-five. Their framework, CoPE, scores contextual knowledge and built-in parametric knowledge separately across models and languages.
They built a dataset from Wikipedia facts in English, Spanish, and Danish, then watched how models answer open questions. A clear pattern emerged, which they name lost-in-the-later: models tend to overlook information placed late in a passage. That positional bias weakens how well answers are grounded in the given text. Reasoning models leaned on context even less. So did ordinary models told to think step by step. The findings warn that longer context windows do not guarantee that late facts get used. Anyone building retrieval systems should test where in the passage the answer sits.
Catalog record
| CREATOR | Yufei Tao, Adam Hiatt, Rahul Seetharaman, Ameeta Agrawal, {yutao, ahiatt, ameeta}@pdx.edu, [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05424
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
“Lost - in - the - Later” : Framework for Quantifying Contextual Grounding in Large Language Models Yufei Tao, Adam Hiatt, Rahul Seetharaman, Ameeta Agrawal, {yutao, ahiatt, ameeta}@pdx.edu, [email protected] Abstract Large language models are capable of leveraging both contextual and parametric knowledge but how they prioritize and integrate these sources remains underexplored. We introduce CoPE 1 1 1 https://github.com/PortNLP/CoPE , a novel evaluation framework that systematically measures contextual knowledge (CK) and parametric knowledge (PK) across models and languages. Using our MultiWikiAtomic dataset in English, Spanish, and Danish, we analyze how large language models (LLMs) integrate context, prioritize information, and incorporate PK in open-ended question answering. Our analysis uncovers a phenomenon we call lost-in-the-later , where LLMs tend to overlook or deprioritize information that appears later in a given context, revealing a strong positional bias that affects contextual grounding. We further find that reasoning models, as well as non-reasoning models prompted with chain-of-thought (CoT), use context even less than non-reasoning models without CoT and fail to mitigate the lost-in-the-later effect. CoT prompting, in particular, results in lower recall and shorter responses, leading to degraded contextual grounding. Based on these insights, we design prompt-based methods to effectively leverage input context. A case study applying CoPE to summarization demonstrates that CK-informed prompting improves factual grounding and reduces hallucination. 1 Introduction Figure 1: Overview of the MultiWikiAtomic dataset creation and CoPE framework. Large language models (LLMs) have demonstrated impressive capabilities in processing contextual information to generate task-relevant and coherent responses across a variety of applications. By conditioning outputs on provided input, LLMs can adapt to different domains, ranging from technical to legal Bayarri-Planas et al. 2024 ; Hu et al. 2025 ; Chen et al. 2025 . However, recent studies highlight that LLMs do not fully utilize all available context, even when sufficient contextual cues are provided Ghosh et al. 2024 ; Zhang et al. 2024a . Following prior work, we adopt a behavioral definition: Contextual Knowledge (CK) refers to content that is entailed by the input context, while Parametric Knowledge (PK) includes any output that is not entailed by the context Tao et al. 2024 ; Kortukov et al. 2024 ; Xu et al. 2024 ; Bi et al. 2025 . PK may include memorised facts, spontaneous generalisations, or new inferences; thus, our classification is behavioral only. This framing allows us to study whether outputs are grounded in the provided input, without assuming anything about how that knowledge is stored internally. Context grounding is also distinct from factual correctness: a model can faithfully follow context that is itself inaccurate, or it can produce correct answers that are not grounded in the context at all. Understanding how LLMs balance CK and PK is critical, especially in high-stakes domains such as medicine or law, where relying too much on PK can potentially increase the risk of factual inaccuracies and hallucination. Yet existing frameworks to precisely quantify CK and PK are lacking. Furthermore, since models predominantly trained on English may exhibit different CK-PK balances when processing other languages, multilingual evaluation remains an important yet largely unexplored area. In parallel, reasoning-oriented prompting strategies such as Chain-of-Thought (CoT) have gained popularity Wei et al. 2022 ; Sprague et al. 2024 . CoT is often assumed to help models stay aligned with input context by encouraging more deliberate reasoning. Yet it remains unclear whether CoT also improves contextual grounding or mitigates positional biases. Our findings challenge this assumption. We show that CoT prompting not only fails to reduce reliance on parametric knowledge but also results in degraded recall from later context segments, further exacerbating the “lost-in-the-later” effect. Moreover, reasoning models, which produce multi-step outputs by design, demonstrate the lowest contextual knowledge (CK) scores across all settings, highlighting a potential trade-off between reasoning and grounding that persists even without explicit CoT prompting. To systematically quantify the balance between CK and PK, we introduce the Context and Parametric Evaluation (CoPE) framework. Figure 1 presents a schematic overview of our study, from dataset construction to CoPE-based evaluation and final insights. The process begins with the creation of the MultiWikiAtomic dataset, where Wikipedia articles are segmented into atomic sentences to serve as context. We then formulate questions based on these contexts and collect model responses, which are also atomized. Both context and response atomic units are analyzed through CoPE, which attributes the information in the response to contextual or parametric knowledge sources. Some key findings include: • LLMs do not fully utilize the context, with CK usage peaking at approximately 70%, across all models and languages studied. • Reasoning models use context even less. Chain-of-Thought (CoT) prompting does not mitigate this and yields the lowest overall CK usage. • Models prioritize earlier context, incorporating progressively less information from the later portions of the context, even when relevant—a phenomenon we term “lost-in-the-later” . • PK is more likely to appear at the end of responses. • In contradiction rich setting, CK drops as expected, but not completely. • Higher percentage of CK is correlated with a reduced likelihood of hallucination. Informed by this analysis, we then developed a simple and adaptable prompting strategy to increase CK reliance. Our contributions are as follows: 1. A framework – CoPE – for systematically evaluating LLMs’ ability to leverage provided context versus relying on PK. 2. A multilingual dataset of 15,000 atomic sentences in English, Spanish, and Danish to study LLMs’ use of CK and PK. 3. An empirical analysis using six LLMs and a simple yet effective prompting strategy for increasing context grounding. 4. A case study showing that CK prompting, developed based on insights from CoPE, improves factual grounding in multidocument summarization while maintaining the quality. 2 Context and Parametric Evaluation (CoPE) Framework CoPE is model- and task-agnostic, requiring only model outputs for evaluation. Unlike prior work limited to specific tasks, contexts, or model accessibility, CoPE can be applied to both open- and closed-source models, and across different generation tasks. This flexibility makes it useful for studying knowledge attribution patterns in different settings. It provides a structured methodology for assessing how LLMs utilize CK and PK. Figure 2 illustrates an example CoPE usage process. Given an input context and a corresponding model-generated response, CoPE computes two scores: contextual knowledge score and context recall distribution. Contextual Knowledge score This metric quantifies the proportion of a response that is directly entailed by the provided context versus information not directly entailed from the context. Formally, given a model-generated response R consisting of n atomic sentences, the CK score is computed as: \text{CK}=\frac{\sum_{i=1}^{n}\mathbb{I}(S_{i}\in C)}{n}\times 100 where C represents the set of entailed context sentences and \mathbb{I} is an indicator function that assigns 1 if a response atomic sentence S_{i} is entailed by the context and 0 otherwise. A higher CK score indicates that the model is effectively grounding its response in context. Since CK and PK are complementary, PK=100-CK . Context Recall distribution This metric examines how models recall information from different segments of the input context (early, middle, or late). To quantify this, the input context is divided into k segments \{Q_{1},Q_{2},...,Q_{k}\} , and the context recall for each segment is computed as: \text{CR}_{q}=\frac{\sum_{i=1}^{n}\mathbb{I}(S_{i}\text{ is entailed by }Q_{q})}{|Q_{q}|} where CR_{q} represents the recall percentage from the q^{th} segment of