Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions

Payal Fofadiya, Sunil Tiwari · 2026

Read Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions by Payal Fofadiya, Sunil Tiwari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29193.txt.

Introduction

Fofadiya and Tiwari present an adaptive method for compressing context in long language-model chats. Long conversations cause memory saturation and slower replies. Their framework scores which past content matters most. A coherence-sensitive filter drops material that adds little. A dynamic budget step controls how much context survives. Tests on LOCOMO, LOCCO, and LongBench measured answer quality and retrieval. The authors report better stability and retrieval with fewer tokens. Latency also fell against rival memory methods.

Catalog record

CREATORPayal Fofadiya, Sunil Tiwari
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31353
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.29193

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions Payal Fofadiya, Sunil Tiwari Abstract Large Language Models (LLMs) often experience performance degradation during long-running interactions due to increasing context length, memory saturation, and computational overhead. This paper presents an adaptive context compression framework that integrates importance-aware memory selection, coherence-sensitive filtering, and dynamic budget allocation to retain essential conversational information while controlling context growth. The approach is evaluated on LOCOMO, LOCCO, and LongBench benchmarks to assess answer quality, retrieval accuracy, coherence preservation, and efficiency. Experimental results demonstrate that the proposed method achieves consistent improvements in conversational stability and retrieval performance while reducing token usage and inference latency compared with existing memory and compression-based approaches. These findings indicate that adaptive context compression provides an effective balance between long-term memory preservation and computational efficiency in persistent LLM interactions. I Introduction Large Language Models (LLMs) have shown strong capabilities in reasoning, dialogue generation, and task completion across diverse domains [ 10 ] . Their performance depends heavily on the availability of contextual information within the input window, which allows the model to maintain continuity and produce coherent responses [ 11 ] . In long-running interactions, however, conversation history grows continuously and creates challenges related to token limits, memory saturation, and loss of earlier information. As interaction length increases, models often fail to preserve important details, resulting in context decay and reduced response consistency [ 5 ] . This limitation has become more visible with the adoption of LLM-based agents in persistent conversational settings where sessions extend across many turns or multiple sessions. Recent research has explored extended context windows, memory augmentation, and retrieval-based architectures to address long-context processing [ 7 ] . Benchmarks such as LOCOMO, LOCCO, and LongBench show that long interactions require both memory retention and efficient context management. While larger context windows reduce information loss, they increase computational cost and latency [ 3 ] . Compression and memory selection methods aim to control context growth, but preserving coherence during reduction remains challenging. These limitations motivate adaptive approaches that selectively retain essential information while removing low-impact history. The research problem addressed in this work is how to maintain coherent and consistent long-running interactions in LLM-based systems while operating under limited context budgets [ 17 ] . Existing models struggle to balance three competing requirements, memory fidelity, computational efficiency, and response quality. Static truncation strategies remove useful historical information, while fixed summarization approaches can introduce information drift [ 19 ] . As a result, long-term conversational stability declines as interaction length increases. The central question is how context can be compressed adaptively without harming retrieval accuracy or conversational coherence. Current approaches attempt to solve this problem through memory indexing, retrieval augmentation, context summarization, or inference-time cache compression [ 1 ] . Memory-based architectures improve retrieval relevance but often retain large historical states that increase computation. Compression methods reduce token usage but may ignore dialogue structure or consistency signals. Benchmark-focused works provide evaluation frameworks but do not introduce adaptive compression mechanisms [ 4 ] . These methods demonstrate progress but still show limitations in recall performance, dynamic adaptation, and robustness across diverse interaction scenarios. The framework presents an adaptive context compression framework for long-running interactions. The approach integrates importance-aware memory selection, coherence-sensitive filtering, and dynamic budget allocation to retain essential information while reducing token overhead. Unlike fixed compression strategies, it adjusts context representation based on interaction complexity and memory relevance, improving stability and inference efficiency without compromising response quality. By jointly modeling relevance and coherence, the framework mitigates context decay and enhances long-term conversational consistency. The aim of this study is to develop and evaluate an adaptive context compression technique that preserves memory consistency and dialogue coherence while improving efficiency in long-running LLM interactions. Based on the stated aim, the study is guided by the following research questions: 1. How effectively can adaptive context compression preserve answer quality and memory retention during long-running LLM interactions compared with existing context management approaches.? 2. To what extent does coherence-aware context selection improve conversational consistency as interaction length increases.? 3. How much efficiency gain in token usage and inference time can be achieved without reducing retrieval performance or response coherence.? This study addresses the challenge of maintaining coherent and efficient long-running interactions in LLM-based systems, where increasing context length leads to memory decay and higher computational cost. The proposed adaptive context compression framework balances memory retention, coherence preservation, and efficiency, offering a practical approach for improving persistent conversational agents beyond simply extending context windows. The results also provide empirical insights into how adaptive memory selection supports long-context reasoning and interaction stability, contributing to the development of scalable LLM systems for sustained dialogue applications. The rest of this paper is organized as follows. Section II reviews related work, Section III presents the proposed framework, Section IV describes the experimental setup, Section V reports results and comparative analysis, and Section VI concludes the paper with future directions. II Literature Review Wu, Ming, and Chen [ 13 ] presented an indexed long-term and short-term memory architecture for chat scenarios using global memory tables and structured layouts. The evaluation used the MemoryBank dataset and the SCM dataset. The model reported answer accuracy of 0.884, retrieve accuracy of 0.938, recall accuracy of 0.663, and coherence of 0.948. Compared with the MemoryBank baseline, answer accuracy increased from 0.624 to 0.884, which corresponded to about a 41.67% relative gain. Execution time decreased from 5.17s without CPM to 4.06s with CPM, which represented a 21.45% reduction. Amogha Rao K [ 16 ] proposed an edge-side context optimization framework combining summarization, retrieval, and token-budget management. Token reduction reached 1% for 10 messages, 50% for 25 messages, 69% for 50 messages, and 83% for 100 messages. A reported example showed a saving of 8560 tokens for a 50-message interaction. These results showed that memory indexing and compression improved efficiency in long interactions. Yang, Sun, and Wang [ 22 ] introduced an LLM-interactive honeypot framework with relevance-based interaction history pruning. The dataset included ShellBox prompt injection data and multi-turn attack flow files. Dynamic error simulation achieved 81.63% accuracy. Interaction history pruning improved turn-level coherence score by 34.5% compared with baseline behavior. The system focused on cybersecurity scenarios but demonstrated measurable gains in interaction consistency through pruning. Jung and Kim [ 8 ] discussed an LLM-based interaction framework for practical settings. The extracted sections did not provide benchmark tables or numerical evaluation values. The dataset field remained unspecified in the available content. As a result, percentage improvements were not reported. The work contributed methodological discussion rather than quantitative benchmarking. Jia et al. [ 6 ] introduced LOCCO and LOCCO-L through the LoCoGen pipeline for evaluating long-term conversational memory. Manual assessment reported consistency of 4.40 \pm 0.52, coherence of 4.45 \pm 0.78, participation of 4.58 \pm 0.86, and overall score of 4.47. The consistency model achieved 98% validation accuracy. A total of 2,981 dialogue QA pairs were retained for evaluation. Maharana et al. [ 12 ] presented LOCOMO as a benchmark for long-term conversational memory across temporal and adversarial conditions. GPT-4-turbo achieved overall QA F1 of 51.6. Retrieval using dialog units produced overall F1 of 41.0 with Recall@k of 76.7. The baseline without retrieval achieved F1

Read the complete text