Dynamic Context Tuning for Retrieval-Augmented Generation: Enhancing Multi-Turn Planning and Tool Adaptation
Jubin Abhishek Soni, Rajesh Kumar Pandey, Amit Anand, Aniket Abhishek Soni · 2025
Read Dynamic Context Tuning for Retrieval-Augmented Generation: Enhancing Multi-Turn Planning and Tool Adaptation by Jubin Abhishek Soni, Rajesh Kumar Pandey, Amit Anand, Aniket Abhishek Soni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11092.txt.
Introduction
Jubin Abhishek Soni and co-authors give chat models a way to keep up with a chat that shifts as it goes, in a 2025 paper. Tools that fetch facts for a model tend to freeze the scene: one turn, one set of tools, no drift.
Real tasks in health or smart homes drift all the time, as aims change and tools come and go. The new frame keeps a cache of past cues that still count, picks tools on the fly, and trims the feed so it fits the model's span. It needs no fresh training. On made and real test sets it raised plan scores and cut made-up claims by a wide mark. It matched a top paid model at far less cost, and it took new tools in stride.
Catalog record
| CREATOR | Jubin Abhishek Soni, Rajesh Kumar Pandey, Amit Anand, Aniket Abhishek Soni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11092
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Context Tuning for Retrieval-Augmented Generation: Enhancing Multi-Turn Planning and Tool Adaptation Thanks: This research was conducted independently. No external funding was received. Jubin Abhishek Soni, Rajesh Kumar Pandey, Amit Anand, Aniket Abhishek Soni Abstract Retrieval-Augmented Generation (RAG) has advanced large language models (LLMs) by grounding their outputs in external tools and knowledge sources. However, existing RAG systems are typically limited to static, single-turn interactions with fixed toolsets, making them ill-suited for dynamic domains such as healthcare and smart homes, where user intent, available tools, and context evolve over time. We present Dynamic Context Tuning (DCT) , a lightweight framework that extends RAG to multi-turn dialogue and evolving tool environments without requiring retraining. DCT integrates an attention-based context cache to track relevant past information, LoRA-based retrieval to dynamically select domain-specific tools, and efficient context compression to keep inputs within LLM context limits. Through experiments on both synthetic and real-world benchmarks, DCT achieves a 14% improvement in plan accuracy and a 37% reduction in hallucinations, while matching GPT-4 performance at significantly lower cost. Moreover, DCT generalizes to previously unseen tools, enabling scalable and adaptable AI assistants across a wide range of dynamic environments. I Introduction Large Language Models (LLMs) have shown strong performance in executing complex tasks by interfacing with external tools, such as APIs and databases [ 1 ] . Retrieval-Augmented Generation (RAG) enhances LLMs by grounding outputs in retrieved knowledge, improving factuality and reducing hallucinations [ 2 ] . However, most RAG systems assume static, single-turn settings and struggle with multi-turn dialogues involving contextual ambiguity and tool variability [ 3 ] . For example, the query “Remind John about his meeting tomorrow” requires resolving ambiguous references (e.g., “his meeting”), temporal reasoning, and tool interaction (e.g., calendar APIs). Traditional methods [ 4 ] often require extensive retraining to accommodate new tools or domains, limiting scalability and responsiveness in practical deployments. These challenges are particularly pronounced in real-world applications like healthcare, smart assistants, and education, where toolsets evolve, interactions are long-term, and compute resources are constrained [ 5 , 6 ] . Additionally, static context windows often lead to context overflow or degraded model performance over extended dialogues [ 7 ] . To address these issues, we propose Dynamic Context Tuning (DCT) , a flexible RAG-based framework designed for dynamic, multi-turn tool use. DCT introduces three key innovations: • Multi-Turn Context Cache : Tracks dialogue history to improve disambiguation and continuity across turns. • Domain-Adaptive Tool Retrieval : Enables fast adaptation to new tools using LoRA [ 8 ] . • Lightweight Context Compression : Condenses dialogue history to efficiently manage LLM context limits [ 9 ] . We evaluate DCT on tasks in medical decision support and IoT orchestration. It improves multi-turn task accuracy by 14%, reduces hallucinations by 37%, and achieves GPT-4 Chain-of-Thought performance with 58% lower inference costs. DCT also generalizes well to new tools, ensuring robust performance in dynamic, resource-constrained settings. DCT advances the deployment of scalable, context-aware, and tool-adaptive LLMs for practical AI assistants. II Related Work II-A Tool-Augmented Language Models Integrating LLMs with external tools has emerged as a promising strategy to enhance task completion and reduce hallucinations. Early systems like REaLM [ 10 ] introduced document-based retrieval to ground model outputs, while recent frameworks such as Chameleon [ 11 ] and Toolformer [ 12 ] explored self-supervised or modular API interactions for tool use. These systems, however, largely operate under static toolsets and assume clean, single-turn prompts. Contextual tool usage remains underexplored. While InfoXLM [ 13 ] and multilingual models [ 14 ] showcase cross-task and cross-lingual generalization, they fall short on procedural or temporal reasoning necessary for dynamic tools. Even advanced models like GPT-4 [ 7 ] show limitations in reference resolution, particularly in dialogic contexts, which can also be helpful in cybersecurity techniques [ 15 ] . Our work addresses these issues by maintaining a memory-augmented cache that improves tool invocation over evolving dialogue contexts. II-B Conversational Retrieval-Augmented Generation Conversational RAG systems extend retrieval to incorporate dialogue history for more coherent responses. Self-RAG [ 16 ] enables on-the-fly evidence gathering using introspective token cues, while FLARE [ 17 ] anticipates future queries to preemptively fetch relevant documents. Despite these innovations, these approaches generally assume static corpora and do not support active tool invocation. More recent work, such as Instruct-RAG [ 3 ] , explores few-shot prompting with retrieved examples for multi-hop QA. However, these models are not designed for dynamic tools or procedural actions. FLARE and related methods struggle with ambiguous entities in conversations—such as resolving “his calendar” across agents. Our method integrates context caching with runtime retrieval to overcome these limitations in procedural multi-turn settings. II-C Efficient Retrieval and Tool Adaptation Scalable retrieval remains a bottleneck, particularly in real-time or resource-constrained environments. ColBERT [ 4 ] introduces token-wise late interaction, enabling accurate passage search with fine-grained relevance estimation. Concurrently, PRP [ 18 ] uses pairwise ranking via LLMs, trading efficiency for precision. Foundational approaches like BPR [ 19 ] remain useful in low-resource setups due to their simplicity and implicit feedback alignment. On the tool adaptation front, LoRA [ 8 ] allows for efficient fine-tuning of LLMs by introducing low-rank trainable matrices within transformer blocks. While LoRA has shown promise in generation and summarization, its potential for real-time retrieval and tool routing remains underutilized. We extend LoRA to enable adaptive tool retrievers, allowing rapid integration of new APIs without retraining the base model. II-D Summary of Gaps Across the literature, three gaps are apparent: (1) lack of robust support for multi-turn, context-sensitive tool planning; (2) rigidity in tool-use architectures that inhibit rapid adaptation; and (3) inefficiency in current retrieval methods for low-latency deployment. Our proposed framework, Dynamic Context Tuning (DCT) , addresses these challenges by combining dialogue-aware memory, LoRA-based retriever tuning, and compact history compression, as discussed in Section III . III Methodology We propose Dynamic Context Tuning (DCT) , a retrieval-augmented generation (RAG) framework that mitigates core limitations in production-level assistants—particularly in the areas of ambiguous turn resolution, restricted context windows, and brittle tool generalization. The architecture, illustrated in Figure 1 , is composed of three primary components: 1. Multi-Turn Context Cache: A memory module that tracks intents, tool usage, and references across dialogue history to resolve ambiguities and enable coherent multi-turn understanding. 2. Lightweight Context Compression: A span-based summarization layer that reduces token usage while preserving essential details like parameters, tool entities, and temporal relations. 3. Domain-Adaptive Tool Retriever: A trainable retriever that supports evolving and domain-specific toolsets using parameter-efficient fine-tuning techniques such as LoRA. Fig. 1: Overview of the Dynamic Context Tuning (DCT) architecture, structured around three key modules: Multi-Turn Context Cache, Lightweight Context Compression, and Domain-Adaptive Tool Retrieval. III-A Multi-Turn Context Cache In order to track tool usage, user goals, and ambiguous references across turns, we implement a multi-turn cache storing intent and context representations in a structured fashion. This cache takes the form of a key-value store, where keys correspond to intent embeddings \mathbf{e}_{i} obtained from a frozen GTR-T5-XL encoder [ 20 ] , and values contain structured tool invocations (name, parameters), entity references, and associated timestamps. When a new user input u_{t} arrives, we encode it as a query vector \mathbf{q}_{t} using the same encoder.