DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention Journal: Journal of Machine Learning Research

Kabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, Ravi Narayanan · 2025

Read DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention Journal: Journal of Machine Learning Research by Kabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, Ravi Narayanan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07185.txt.

Introduction

Kabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, and Ravi Narayanan wrote a 2025 paper on a framework for updating the knowledge of large language models in real time. A trained model's knowledge is frozen and quickly goes stale. Retraining is prohibitively expensive, and existing editing techniques can be slow or cause side effects.

Their framework, DySK-Attn, connects the model to a knowledge graph, a structured store of facts that can be updated instantly. A dynamic sparse knowledge attention mechanism lets the model attend to just the relevant facts from the graph while answering. Because attention is sparse, the lookup stays fast. The design aims to keep deployed models current without rebuilding them.

Catalog record

CREATORKabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, Ravi Narayanan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07185

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention Journal: Journal of Machine Learning Research Kabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, Ravi Narayanan Abstract Large Language Models (LLMs) suffer from a critical limitation: their knowledge is static and quickly becomes outdated. Retraining these massive models is computationally prohibitive, while existing knowledge editing techniques can be slow and may introduce unforeseen side effects. To address this, we propose DySK-Attn, a novel framework that enables LLMs to efficiently integrate real-time knowledge from a dynamic external source. Our approach synergizes an LLM with a dynamic Knowledge Graph (KG) that can be updated instantaneously. The core of our framework is a sparse knowledge attention mechanism, which allows the LLM to perform a coarse-to-fine grained search, efficiently identifying and focusing on a small, highly relevant subset of facts from the vast KG. This mechanism avoids the high computational cost of dense attention over the entire knowledge base and mitigates noise from irrelevant information. We demonstrate through extensive experiments on time-sensitive question-answering tasks that DySK-Attn significantly outperforms strong baselines, including standard Retrieval-Augmented Generation (RAG) and model editing techniques, in both factual accuracy for updated knowledge and computational efficiency. Our framework offers a scalable and effective solution for building LLMs that can stay current with the ever-changing world. 1 Introduction Large Language Models (LLMs) have marked a paradigm shift in artificial intelligence, demonstrating remarkable capabilities in natural language understanding, generation, and reasoning across a multitude of domains lewis2020retrieval . Their success in knowledge-intensive tasks has positioned them as foundational tools for applications ranging from sophisticated question answering systems to complex problem-solving agents capable of using external tools qin2023toolllm , wang2024target . The underlyi ng principle of these models is their ability to internalize vast amounts of factual knowledge from extensive training corpora, effectively creating an implicit knowledge base petroni2019language , cui2025learning . However, this very strength introduces a critical vulnerability: the knowledge encoded within their parameters is inherently static, bound to a "knowledge cut-off" date. This static nature renders them unable to reason about events or information that emerged post-training, leading to factual inaccuracies or "hallucinations" when confronted with novel topics ji2023llms . The challenge of keeping these models current is non-trivial, as complete retraining is computationally prohibitive and unsustainable for real-world, dynamic environments, such as those encountered in continuous sensing applications where new data patterns constantly emerge huang2023phaseanti . This need for adaptability is a shared challenge in many AI fields, including the development of robust federated learning systems. To surmount this limitation, Retrieval-Augmented Generation (RAG) has emerged as a dominant paradigm lewis2020retrieval , guu2020realm . RAG-based methods enhance LLMs by dynamically retrieving information from external corpora at inference time. The field has seen rapid advancements, with sophisticated frameworks like Self-RAG, which learns to critique its own retrieved documents zhang2025wiopen , and CRAG, which introduces a corrective mechanism to improve retrieval robustness ahmed2024crag . Other approaches focus on optimizing the retrieval process itself, such as determining the optimal granularity of information to retrieve sachan2024dense , developing active retrieval strategies zhong2023flare , gu2023attention , or combining retrieval with chain-of-thought reasoning ram2023chainofthought . While powerful, many RAG systems rely on unstructured text, which can be noisy. This has motivated research into leveraging structured knowledge sources, most notably Knowledge Graphs (KGs), which offer a more organized representation of facts pan2024knowledge . Integrating KGs allows for more precise reasoning, as demonstrated in tasks like visual dialog where context-aware graph inference is crucial guo2021context , and in unified question-answering systems that operate over heterogeneous data formats hui2022unikqa . Yet, a significant challenge remains in how to efficiently fuse the symbolic knowledge of KGs with the sub-symbolic representations of LLMs without incurring substantial overhead zhang2022greaselm . Furthermore, as LLMs are tasked with more complex reasoning, often modeled as a "graph of thoughts" besta2023graph , the efficiency of their core architecture becomes paramount. The quadratic complexity of the standard Transformer attention mechanism poses a bottleneck for processing long contexts beltagy2020longformer . This has spurred the development of more efficient architectures, including sparse attention models like Longformer beltagy2020longformer and Big Bird , Mixture-of-Experts (MoE) models like Switch Transformers fedus2021switch , fan2024multi and Mixtral jiang2024mixtral , and even entirely new paradigms like state-space models (Mamba) gu2023mamba and other efficient long-context mechanisms han2024hyperattention . These architectural innovations, alongside hardware-aware optimizations like FlashAttention dao2022flashattention , dao2023flashattention2 , are critical for building scalable systems. The challenge of efficiency is not unique to LLMs; it is a central theme in many areas of deep learning, from model compression through quantization to designing lightweight models for gesture recognition on commodity devices . In this paper, we propose a novel framework that directly addresses the dual challenges of knowledge staleness and computational inefficiency. Our approach, named DySK-Attn (Dynamic Sparse Knowledge Attention) , synergizes LLMs with an external, dynamic Knowledge Graph that can be updated in real-time. The core of our contribution is a sparse knowledge attention mechanism, which enables the LLM to efficiently query the dynamic KG and selectively focus on a small subset of the most relevant facts. This mechanism avoids the dense computations associated with full attention over a large knowledge base. By doing so, our model can seamlessly integrate new information without costly retraining, effectively mitigating knowledge decay and enhancing factual accuracy. This mirrors challenges in other domains where systems must adapt to new information, such as generalizing facial expression recognition from heterogeneous data or developing robust security against side-channel attacks . The main contributions of this work are threefold: • We design a novel architecture that facilitates a deep, yet efficient, fusion of a large language model with a dynamic, continuously updated knowledge graph. • We introduce a sparse knowledge attention mechanism that allows the model to intelligently retrieve and reason over a vast external knowledge base with minimal computational overhead. • We empirically demonstrate through extensive experiments that our proposed model significantly outperforms existing baselines in tasks requiring up-to-date knowledge, while also being more computationally efficient and robust against factual hallucinations. The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 details our proposed methodology. Section 4 describes the experimental setup, followed by results and analysis in Section 5. Finally, Section 6 concludes the paper. 2 Related Work Our research is situated at the intersection of three primary areas: knowledge augmentation for LLMs, the development of efficient model architectures, and the fusion of LLMs with structured knowledge bases like KGs. In this section, we review seminal and recent works in these domains to contextualize our contributions. 2.1 Knowledge-Augmented Language Models The inherent limitation of static knowledge in LLMs has led to the development of knowledge-augmented models, with RAG being the most prominent approach lewis2020retrieval , guu2020realm . The core idea of RAG is to supplement the model’s internal knowledge with information retrieved from an external source at inference time. Recent advancements have focused on enhancing the robustness and intelligence of the retrieval process. For instance, Self-RAG huang2024phyfinatt introduces a self-reflection mechanism, enabling the model to decide when retrieval is necessary and to evaluate the quality of retrieved passages. Similarly, Corrective RAG (CRAG) asai2023selfrag employs a retrieval evaluator to assess document

Read the complete text