Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

Sekh Mainul Islam Pepa Atanasova Isabelle Augenstein · 2025

Read Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement by Sekh Mainul Islam Pepa Atanasova Isabelle Augenstein (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01706.txt.

Introduction

Sekh Mainul Islam, Pepa Atanasova, and Isabelle Augenstein dissect how large language models write explanations. A natural language explanation can lean on two knowledge sources: the context handed to the model, and the parametric knowledge stored in its weights during training. How those sources mix, step by step, decides whether an explanation is grounded or hallucinated.

Earlier analyses forced the mixture into a one-dimensional, either-or subspace and looked at single steps. The authors build a rank-2 projection subspace that separates the two contributions more faithfully, then track them across long explanations on four question-answering datasets and three open-weight models. Supportive interactions align with the parametric direction, conflicting ones with the context direction, and hallucinated passages lean strongly parametric, while faithful ones stay balanced.

Catalog record

CREATORSekh Mainul Islam Pepa Atanasova Isabelle Augenstein
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01706

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement Sekh Mainul Islam Pepa Atanasova Isabelle Augenstein Abstract Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions, drawing on both external Context Knowledge (CK) and Parametric Knowledge (PK) stored in model weights. Understanding their interaction is key to assessing the grounding of NLEs, yet it remains underexplored. Prior work has largely examined only single-step generation -- typically the final answer, and has modelled PK–CK interaction only as a binary choice in a rank-1 subspace. This overlooks richer forms of interaction, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments on four QA datasets and three open-weight instruction-tuned LLMs show that diverse knowledge interactions are poorly represented in a rank-1 subspace but are effectively captured in our rank-2 formulation. Our multi-step analysis reveals that hallucinated NLEs align strongly with the PK direction, context-faithful ones balance PK and CK, and Chain-of-Thought prompting for NLEs shifts generated NLEs toward CK by reducing PK reliance. This work provides the first framework for systematic studies of multi-step knowledge interactions in LLMs through a richer rank-2 subspace disentanglement. 1 1 1 Code and data: https://github.com/copenlu/pk-ck-knowledge-disentanglement Figure 1: Llama-3.1-8B-Instruct model combines parametric (green) and contextual (red) knowledge to generate NLEs. Projection onto a learned low-rank subspace \mathbf{P} disentangles their contributions – rank-1 discards richer interactions, while rank-2 separates complementary and conflicting components. 1 Introduction Large Language Models (LLMs) are employed to generate Natural Language Explanations (NLEs) in a human-readable format, illustrating the underlying decision-making process for predictions in complex reasoning tasks such as Claim Verification (CV) and Question Answering (QA). These NLEs are valuable because they can reveal the utilization of external context and the knowledge stored in model parameters. Consider the Fig. 1 example from a QA task ( Cheng et al. 2024 ) , where Llama-3.1-8B-Instruct Meta-Team 2024 generates the NLE by utilizing both Context Knowledge (CK) and Parametric Knowledge (PK), to explain the underlying decision-making process for the final answer prediction. For some tasks, such as CV, we assume the decision-making process and, in turn, the NLE, will rely more on evidence (CK) Wang and Shu 2023 ; Tan et al. 2025 , and in QA tasks with misleading external context, we assume they will rely on the PK. On the other hand, Chain-of-Thought (CoT) prompting ( Wei et al. 2022 ) , a widely adopted reasoning methodology, explicitly elicits NLEs of the intermediate reasoning steps and is hypothesized to influence how LLMs integrate PK and CK ( Cheng et al. 2024 ; Su et al. 2024 ; Tao et al. 2025 ) , potentially improving contextual grounding. However, it remains unclear what the learned PK-CK interaction dynamics in generating the NLEs are, necessitating a multi-step analysis of PK-CK interaction across longer NLE sequences. Prior work ( Longpre et al. 2021 ; Xu et al. 2024 ; Minder et al. 2025 ) has primarily focused on uncovering the single-step generation mechanism – typically the final answer, and modelled only conflicting PK–CK interaction as a binary choice in a rank-1 subspace, thereby overlooking richer forms such as complementary or supportive knowledge ( Cheng et al. 2024 ) . We thus hypothesize that this rank-1 subspace is not sufficient to disentangle the individual contributions of PK and CK in all types of knowledge interaction scenarios . Moreover, they did not characterize the step-by-step PK-CK interaction dynamics over longer sequences, such as NLE generation for different knowledge interactions. To accurately understand the knowledge interaction dynamics during NLE generation, we investigate the following research questions: RQ1. Is a rank-1 projection subspace enough for disentangling PK and CK contributions in all types of knowledge interaction scenarios? RQ2. How do individual PK and CK contributions change over the NLE generation steps for different knowledge interactions? RQ3. Can we find reasons for hallucinations based on PK-CK interactions? RQ4. How is the CoT mechanism aligned with the knowledge interaction subspace? We perform experiments on four publicly available QA datasets for three open-weight instruction-tuned LMs. In RQ1 (§ 4.2 ), we find that the rank-1 subspace ( Minder et al. 2025 ) fails to disentangle individual knowledge contributions for different knowledge interactions. For RQ2 (§ 4.3 ), using a more accurate rank-2 subspace disentanglement, we find that, during NLE generation, the model utilizes both knowledge sources with slight prioritization of PK; towards the final answer generation, the model aligns closely with the CK direction for conflicting examples; for supportive examples, it aligns more with the PK direction. This learned rank-2 subspace also illustrates PK dominance for the sequences with hallucinated spans (RQ3, § 4.4 ). Finally, for the RQ4 (§ 4.5 ), we observe that the CoT mechanism is also encoded in the LLM as a low-rank space, and the CoT encoding subspace closely aligns with the context direction of the rank-2 knowledge interaction subspace. Overall, this work advances understanding of how LLMs integrate internal and external knowledge by introducing the first systematic framework for multi-step analysis of knowledge interactions via rank-2 subspace disentanglement. 2 Related Work Parametric vs. contextual knowledge in LMs. Prior work has extensively examined what LMs store as parametric knowledge (PK) and how they integrate context knowledge (CK) during inference ( Petroni et al. 2019 ; Jiang et al. 2020 ; Roberts et al. 2020 ; Hagström et al. 2025 ; Marjanovic et al. 2024 ; Yu et al. 2024 ) . Early studies showed that LMs act as factual knowledge bases, with knowledge retrievable via prompts ( Petroni et al. 2019 ) , while probing analyses disentangled what is encoded in the parameters versus what is acquired from context ( Brown et al. 2020 ; Tenney et al. 2019 ; Bi et al. 2025 ) . Recent research has explored the interplay and conflict between PK and CK. Cheng et al. 2024 revealed PK suppression under various knowledge interactions, and Xu et al. 2024 categorized conflict types (context–memory, inter-context, intra-memory) to study resulting behavioral shifts. Enhancing PK–CK balance has been pursued through improved pretraining or fine-tuning ( Zhang et al. 2024 ) , context-aware representation interventions ( Yuan et al. 2025b ) , contrastive decoding ( Zhao et al. 2024 ) , and lightweight steering methods that increase context sensitivity without weight updates ( Wang et al. 2025 ) . Our work extends this literature by providing a geometric framework that accurately models PK–CK interactions in a rank-2 subspace and enables multi-step analysis of how both knowledge sources jointly shape Natural Language Explanations (NLEs). NLEs and explanation-guided reasoning. Natural Language Explanations (NLEs) Camburu et al. 2018 ; Atanasova et al. 2020 have been used to expose, supervise, and steer model reasoning in complex tasks. Human-written NLEs improve commonsense QA via explanation-based fine-tuning ( Rajani et al. 2019 ) , while Chain-of-Thought (CoT) prompting elicits stepwise reasoning that boosts performance on multi-hop and arithmetic tasks ( Wei et al. 2022 ) . Explanations have also been used as few-shot exemplars to enhance in-context learning ( Lampinen et al. 2022 ) . Despite their utility, studies reveal that NLEs often fail

Read the complete text