Enhancing Multi-Agent Communication through Attention Steering with Context Relevance
Hongxiang Zhang, Yuan Tian, Tianyi Zhang · 2026
Read Enhancing Multi-Agent Communication through Attention Steering with Context Relevance by Hongxiang Zhang, Yuan Tian, Tianyi Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30136.txt.
Introduction
Hongxiang Zhang, Yuan Tian and Tianyi Zhang present Agent-Radar, a training-free method for managing context in multi-agent systems. Long conversations dilute relevant information with irrelevant context, degrading performance as histories grow. Agent-Radar guides each agent's attention to relevant context through temporal and spatial decay. It beats state-of-the-art methods across five benchmarks, gaining up to 7.64 points, and stays robust as agents and rounds grow. Ablations confirm its core components matter for performance and generalize across settings.
Catalog record
| CREATOR | Hongxiang Zhang, Yuan Tian, Tianyi Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 110864 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30136
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Multi-Agent Communication through Attention Steering with Context Relevance Hongxiang Zhang, Yuan Tian, Tianyi Zhang Abstract LLM-based multi-agent systems have demonstrated remarkable performance on complex tasks through collaborative reasoning. However, these systems tend to rapidly accumulate extremely long conversation histories during interaction. As conversations lengthen, relevant information is increasingly diluted by irrelevant context, leading to degraded performance. In this work, we present Agent-Radar , a training-free context management method that dynamically steers each agent’s attention toward relevant context with a novel temporal and spatial decay mechanism. Our experiments demonstrate that Agent-Radar outperforms state-of-the-art methods across five different benchmarks, yielding gains of up to 7.64 absolute points. Furthermore, our analysis shows that Agent-Radar remains effective and robust as the number of agents and interaction rounds increases. Finally, the ablation study shows that core components in Agent-Radar are crucial to performance and generalizable in different settings. 1 Introduction LLM-based multi-agent systems have demonstrated remarkable performance on complex tasks ( 20 ; 43 ; 8 ; 46 ) . Despite these successes, agent communication often accumulates extremely long context windows rapidly ( 49 ) and suffers from communication redundancy ( 44 ; 49 ; 8 ; 20 ) . As a result, the original task description, key constraints, and other critical evidence are often buried within the context, making it difficult for LLMs to reliably identify and attend to them. This phenomenon, also known as the “lost-in-the-middle” effect, frequently induces hallucinations and logical errors during generation ( 3 ; 23 ; 5 ; 52 ) . This issue is further exacerbated as systems scale. Increasing numbers of agents and interaction rounds cause communication histories to grow rapidly, amplifying context dilution and undermining the reliability of large-scale agent collaborations ( 5 ; 3 ; 52 ) . Prior context management methods have sought to mitigate this challenge in two ways. One line of work applies periodic summarization to compress accumulated interaction history ( 41 ; 47 ) . Another adopts pruning strategies that limit context growth by removing redundant agents or communication edges ( 49 ; 44 ) . While effective at improving efficiency, these methods introduce information loss. Specifically, compressing or pruning messages can discard subtle but critical signals, potentially misleading downstream reasoning and propagating errors across subsequent interactions ( 31 ; 1 ) . We propose a shift from context compression or pruning to selectively steering agents’ attention over relevant context. In the original Transformer architecture, the attention mechanism retrieves information by matching queries to keys, allowing the model to assign importance weights across preceding tokens in the context ( 40 ) . Accordingly, in multi-agent systems, an agent’s current generation step naturally acts as a query seeking to retrieve relevant information from the communication history. Unlike prior methods that act as hard, binary filters over context, we instead steer attention toward a subset of relevant content while preserving the full transcript, thereby avoiding information loss. To achieve this, we develop Agent-Radar , a training-free method that dynamically guides agents’ attention toward relevant context without modifying the underlying communication history. To identify which parts of the history should guide each generation step, Agent-Radar selects context by jointly modeling semantic relevance, spatial distance, and temporal recency. First, inspired by organizational network theory ( 4 ; 2 ) , where influence decays with structural distance, Agent-Radar formulates the agent interaction topology as a graph and applies a spatial decay factor based on inter-agent distance, prioritizing messages from closely interacting agents. Second, Agent-Radar incorporates temporal decay to down-weight stale information, reflecting evidence from both long-context modeling and human memory that relevance diminishes over time ( 10 ; 33 ; 26 ) . Third, Agent-Radar performs fine-grained semantic matching by segmenting messages into sentence-level units to extract relevant evidence at the current step. Finally, Agent-Radar integrates these signals through a lightweight attention-steering mechanism, enabling agents to emphasize key instructions, critical constraints, and useful intermediate evidence, without compressing or pruning history tokens. We evaluate Agent-Radar on five benchmarks using three base LLMs. Compared with SOTA context management methods, including compression-based approach ( 41 ) and pruning methods ( 49 ; 44 ) , Agent-Radar achieves gains of up to 7.64 absolute points. As a plug-in method, Agent-Radar can be seamlessly integrated into diverse multi-agent systems, including debate-based systems ( 8 ) , GPTSwarm ( 54 ) , and AutoGen ( 46 ) . Across these frameworks, Agent-Radar consistently improves their performance by up to 12.87 absolute points and generalizes across different interaction topologies. Furthermore, Agent-Radar remains robust as systems scale to larger numbers of agents and longer interaction rounds. Finally, ablation studies confirm that each component contributes meaningfully to the overall performance gains. 2 Problem Formulation We model the multi-agent system architecture as a graph \mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{F}) . Let \mathcal{V}=\{v_{i}\}_{i=1}^{N} be the set of N nodes, where each node v_{i} represents an autonomous LLM agent. The edge set \mathcal{E}\subseteq\mathcal{V}\times\mathcal{V} defines the communication topology. An edge (v_{i},v_{j})\in\mathcal{E} denotes the communication path from the agent v_{i} to the agent v_{j} . The set of mapping functions \mathcal{F}=\{f_{i}\}_{i=1}^{N} represents the inference processes of the agents. We further define the hop distance d_{\mathcal{G}}(v_{j},v_{i}) as the length of the shortest path from v_{j} to v_{i} in \mathcal{G} . The k -hop neighborhood of v_{i} is \mathcal{N}_{k}(v_{i})=\{v_{j}\in\mathcal{V}\mid d_{\mathcal{G}}(v_{j},v_{i})\leq k\} . All agents that can reach to v_{i} in \mathcal{G} is \mathcal{N}_{N-1}(v_{i})=\bigcup_{k=1}^{N-1}\mathcal{N}_{k}(v_{i}) . The system proceeds over communication rounds t\in\{1,\dots,T\} . At time t , the active agent v_{i} generates the response o_{i}^{(t)}=f_{i}\!\left(q^{(t)},~\mathcal{Z}_{i}^{(t)}\right), (1) where q^{(t)} is the query received by the current agent at time t , and \mathcal{Z}_{i}^{(t)} is the accumulated communication histories available to v_{i} . The context available to the current agent v_{i} at time t is \mathcal{Z}_{i}^{(t)}=\{\ o_{j}^{(\tau)}\mid v_{j}\in\mathcal{N}_{N-1}(v_{i}),~1\leq\tau<t\,\}, (2) where o_{j}^{(\tau)} is the output produced by agent v_{j} at timestamp \tau . In our method, we introduce an additional parameter \hat{\mathcal{Z}}_{i}^{(t)} to f_{i} , denoting the selected context that Agent-Radar steers attention towards for each inference step. In the following section, we describe how Agent-Radar selects \hat{\mathcal{Z}}_{i}^{(t)} from the accumulated communication histories and uses it as an additional argument to f_{i} for attention steering. 3 Methodology We introduce Agent-Radar (Figure 1 ), a training-free method that augments multi-agent systems (MAS) by steering an agent’s attention toward semantically relevant context that is both spatially proximate and temporally salient. Building on the formulation in Section 2 , Agent-Radar selects fine-grained context by jointly modeling semantic relevance, spatial structure, and temporal recency. Then, it steers the agent’s attention toward the selected context during generation. Figure 1 : Overview of Agent-Radar . (Top) MAS interactions rapidly accumulate long communication histories, where useful information is buried in the middle, receiving insufficient attention. (Bottom) Agent-Radar preserves the full transcript and topology, scores sentence-level context by semantic relevance weighted with temporal and spatial decay, and steers the agent’s attention toward the selected context during inference. 3.1 Dynamic Context Selection While standard semantic retrieval is effective in many cases, applying it naively to the dynamic MAS context is insufficient. 1 1 1 We present ablation studies in Section 4.3 comparing Agent-Radar against semantic-only retrieval and variants with spatial or temporal decay individually removed. Unlike a flat document collection, multi-agent communication is generated by different agents at different