Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning

Xiaozhe Li, Tianyi Lyu, Yizhao Yang, Liang Shan, Siyi Yang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu, Yang Li · 2026

Read Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning by Xiaozhe Li, Tianyi Lyu, Yizhao Yang, Liang Shan, Siyi Yang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu, Yang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11462.txt.

Introduction

Li and eight colleagues ease the context bottleneck for long-horizon language-model agents with active context curation. Their framework pairs a lightweight reinforcement-learned ContextCurator policy with a frozen TaskExecutor foundation model, decoupling context management from task execution. The curator prunes environmental noise while preserving reasoning anchors, sparse data points critical for future deductions. On WebArena it lifts Gemini-3.0-flash success from 36.4 to 41.2 percent while cutting tokens 8.8 percent, and on DeepSearch it reaches 57.1 percent against 53.9 percent with eightfold token savings. A 7B curator matches GPT-4o at context management.

Catalog record

CREATORXiaozhe Li, Tianyi Lyu, Yizhao Yang, Liang Shan, Siyi Yang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu, Yang Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60048
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11462

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning Xiaozhe Li, Tianyi Lyu, Yizhao Yang, Liang Shan, Siyi Yang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu, Yang Li Abstract Large Language Models (LLMs) struggle with long-horizon tasks due to the ”Context Bottleneck” and the ”Lost-in-the-Middle” phenomenon, where accumulated noise from verbose environments degrades reasoning over multi-turn interactions. To address this, we introduce a symbiotic framework that decouples context management from task execution. Our architecture pairs a lightweight, specialized policy model (ContextCurator) that actively curates the working memory for a powerful, frozen foundation model (TaskExecutor). Trained via reinforcement learning, the ContextCurator performs active information entropy reduction. It aggressively prunes environmental noise while meticulously preserving ”reasoning anchors”—sparse data points critical for future deductions. On WebArena, our framework boosts Gemini-3.0-flash’s success rate from 36.4% to 41.2% while reducing token consumption by 8.8% ( 47.4 ​ K → 43.3 ​ K 47.4K\rightarrow 43.3K ). On DeepSearch, it achieves 57.1% success (vs. 53.9%) with an 8 × 8\times token reduction. Remarkably, a 7B ContextCurator matches the context management performance of GPT-4o, providing a scalable, computationally efficient paradigm for autonomous long-horizon agents. 1 Introduction The trajectory of artificial intelligence has fundamentally shifted from the development of static conversational models to the deployment of autonomous, goal-oriented Large Language Model (LLM) agents capable of sequential decision-making in complex environments ( 20 ; 13 ; 19 ; 15 ) . This advancement enables agents to tackle complex real-world challenges, from web browsing ( 5 ; 27 ) to multi-hop information retrieval ( 26 ; 23 ) . However, a critical bottleneck emerges in long-horizon tasks: while frontier models exhibit profound zero-shot reasoning capabilities, their performance degrades dramatically due to the inherent limitations of their cognitive span ( 7 ; 4 ) . The fundamental challenge to reliable long-horizon execution is the ability to maintain task-relevant working memory while systematically filtering environmental noise over extended trajectories. Real-world observation spaces are characterized by extreme informational entropy. In web tasks, DOM trees contain over 90% structural noise (e.g., advertisements, styling scripts, redundant navigational elements), while search paradigms flood the context with semantic noise and distractor paragraphs. When agents process these raw streams, the signal-to-noise ratio (SNR) deteriorates rapidly. Feeding this high-entropy data directly into a transformer inevitably triggers the ”Lost-in-the-Middle” phenomenon and causes cascading logical failures , where early noise compounds into hallucinated reasoning in subsequent steps. Our experiments reveal that even frontier models like Gemini-3.0-flash achieve only 36.4% success on WebArena and 53.9% on DeepSearch (Tables 1 and 2 ). These results expose a critical limitation: brute-force expansion of context windows does not guarantee effective reasoning. The problem is not capacity, but signal-to-noise ratio. Current mitigation strategies fall short of addressing the dynamic nature of these environments. Passive memory systems ( 2 ; 17 ; 32 ; 16 ) treat context management as a static retrieval problem, relying on semantic similarity to recall information. These methods suffer from retrieval bias; they frequently fail to retrieve implicit reasoning anchors —causally essential information that may be textually dissimilar to the current query. Conversely, monolithic learned architectures attempt to internalize memory management within the same task model ( 34 ; 28 ) . This creates a severe capacity split: small models lack the reasoning depth required for complex logical execution, while large proprietary models are too computationally expensive and opaque for direct on-policy fine-tuning. To bridge this gap, we introduce ActiveContext , a framework that reframes memory management from a static storage utility into an active, sequential decision-making process . We adopt a complementary cognitive architecture that decouples the system into two specialized components: a powerful frozen foundation model ( TaskExecutor ) handles pure reasoning and task execution, while a lightweight specialized policy model ( ContextCurator ) actively curates its working memory. Trained via reinforcement learning, the ContextCurator learns to act as a ”Cognitive Architect.” Driven by a distal reward signal based on the TaskExecutor’s success, it develops a generalizable policy to perform active information entropy reduction . It aggressively prunes structural and semantic noise while meticulously preserving reasoning anchors —sparse data points critical for future deductions. This symbiotic design ensures that the TaskExecutor’s attention heads are focused solely on high-fidelity, high-signal data. Extensive evaluation demonstrates that our decoupled framework establishes a new Pareto frontier in agentic reasoning. On WebArena, ActiveContext boosts Gemini-3.0-flash from a 36.4% to a 41.2% success rate while reducing token consumption by 8.8% (47.4K \rightarrow 43.3K). On DeepSearch, the success rate improves from 53.9% to 57.1% with nearly an 8\times token reduction (46.7K \rightarrow 6.6K). Remarkably, our 7B ContextCurator matches the context management performance of GPT-4o. This result democratizes agentic pipelines, proving that active memory curation is a distinct cognitive skill that can be efficiently offloaded to smaller models. Our work makes three key contributions: • We propose a novel symbiotic framework that decouples context management from task execution. By pairing a specialized ContextCurator with a closed-source SOTA TaskExecutor, we transform memory from a passive storage utility into an active, decision-dependent control variable. • We formulate an on-policy RL pipeline that trains lightweight models to actively manage context through dynamic environment interaction. Guided by distal rewards, the model learns a generalizable policy for information entropy reduction —aggressively pruning noise while meticulously preserving critical reasoning anchors. • We demonstrate substantial empirical gains across complex long-horizon benchmarks. Our framework improves SOTA model performance while reducing token consumption, and proves that a specialized 7B model can match the context management capabilities of GPT-4o, providing a highly scalable and sustainable paradigm for autonomous agents. 2 Related Work Memory for Agentic Systems. Context memory enables autonomous agents to maintain state and leverage history for complex reasoning ( 29 ) . Existing frameworks typically utilize passive storage (e.g., Mem0 ( 2 ) , MemGPT ( 17 ) , LangMem ( 12 ) ) via semantic retrieval, or procedural workflows (e.g., HiAgent ( 6 ) , AWM ( 21 ) ) to induce structured trajectories. However, passive systems suffer from retrieval bias, frequently missing causally essential ”reasoning anchors” that lack textual similarity ( 14 ; 33 ) , while procedural workflows struggle to generalize across high-entropy domains. Advanced cognitive graphs (e.g., Synapse ( 31 ) , MAGMA ( 9 ) ) mitigate this but introduce formidable computational latency. In contrast, our framework adopts a symbiotic architecture that decouples context management from task execution . Rather than relying on static storage or heavy graphs, our ContextCurator actively curates context to maximize real-time information density. Reinforcement Learning for LLM Agents. RL is a pivotal paradigm for aligning agentic reasoning across diverse domains (e.g., web navigation, embodied environments) using algorithms like PPO and GRPO ( 18 ; 19 ; 27 ; 22 ; 11 ) . Recognizing the limits of static memory, recent works (e.g., Mem1 ( 34 ) , MemAgent ( 28 ) , MemoryR1 ( 24 ) ) apply RL to optimize active memory operations as sequential decisions. However, these approaches train a monolithic model for both memory curation and task execution. This creates a severe capacity split: small open-source models lack the deep reasoning required for complex deductions, while frontier models are opaque and computationally prohibitive for on-policy fine-tuning. Monolithic RL also risks inducing

Read the complete text