Attention Knows Whom to Trust: Attention-based Trust Management for LLM Multi-Agent Systems
Pengfei He, Zhenwei Dai, Xianfeng Tang, Yue Xing, Hui Liu, Jingying Zeng, Qiankun Peng, Shrivats Agrawal, Samarth Varshney, Suhang Wang, Jiliang Tang, Qi He · 2025
Read Attention Knows Whom to Trust: Attention-based Trust Management for LLM Multi-Agent Systems by Pengfei He, Zhenwei Dai, Xianfeng Tang, Yue Xing, Hui Liu, Jingying Zeng, Qiankun Peng, Shrivats Agrawal, Samarth Varshney, Suhang Wang, Jiliang Tang, Qi He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02546.txt.
Introduction
Pengfei He, Zhenwei Dai, and colleagues teach teams of language model agents to decide which messages deserve trust in a 2025 study. Multi-agent systems divide hard tasks among cooperating models, but a single unreliable message, from a weak agent or an attacker, can poison the whole team. Today's agents tend to swallow every incoming message at face value, a gap the authors call fundamental.
Their trust management reads attention patterns inside the receiving model to judge a message's reliability. Selective trust keeps collaboration's benefits while closing the easiest attack route into agent teams, an increasingly practical security concern.
Catalog record
| CREATOR | Pengfei He, Zhenwei Dai, Xianfeng Tang, Yue Xing, Hui Liu, Jingying Zeng, Qiankun Peng, Shrivats Agrawal, Samarth Varshney, Suhang Wang, Jiliang Tang, Qi He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02546
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Attention Knows Whom to Trust: Attention-based Trust Management for LLM Multi-Agent Systems Pengfei He, Zhenwei Dai, Xianfeng Tang, Yue Xing, Hui Liu, Jingying Zeng, Qiankun Peng, Shrivats Agrawal, Samarth Varshney, Suhang Wang, Jiliang Tang, Qi He Abstract Large Language Model-based Multi-Agent Systems (LLM-MAS) have demonstrated strong capabilities in solving complex tasks but remain vulnerable when agents receive unreliable messages. This vulnerability stems from a fundamental gap: LLM agents treat all incoming messages equally without evaluating their trustworthiness. While some existing studies approach the trustworthiness, they focus on a single type of harmfulness rather than analyze it in a holistic approach from multiple trustworthiness perspectives. In this work, we propose Attention Trust Score (A-Trust), a lightweight, attention-based method for evaluating message trustworthiness. Inspired by human communication literature ( grice1975logic ) , through systematically analyzing attention behaviors across six orthogonal trust dimensions, we find that certain attention heads in the LLM specialize in detecting specific types of violations. Leveraging these insights, A-Trust directly infers trustworthiness from internal attention patterns without requiring external prompts or verifiers. Building upon A-Trust, we develop a principled and efficient trust management system (TMS) for LLM-MAS, enabling both message-level and agent-level trust assessment. Experiments across diverse multi-agent settings and tasks demonstrate that applying our TMS significantly enhances robustness against malicious inputs. 1 Introduction Large Language Model-based Multi-Agents (LLM-MAS) have demonstrated remarkable capabilities in complex problem solving and task planning ( li2023camel ; gao2024agentscope ) . They advance numerous applications such as code generation ( liu2024large ; hong2023metagpt ; qian2023chatdev ) , mathematical reasoning ( lei2024macm ; he2024make ) , and scientific simulations ( zheng2023chatgpt ; tang2023medagents ) . Based on ( tran2025multi ) , the effectiveness of LLM-MAS is largely driven by the collaboration and communication among agents with diverse roles and expertise, and informative messages can improve performance. However, unreliable or misleading messages can corrupt reasoning and cause harmful or incorrect behaviors of the system. Recent studies have shown that LLM-MAS are highly sensitive to incorrect information ( lee2024prompt ) , and vulnerable to malicious attacks, including the injection of deceptive messages by compromised agents ( zhang2024breaking ; ju2024flooding ) and communication interception attacks ( he2025red ) . A fundamental reason of these vulnerabilities is that LLM agents lack skepticism toward incoming messages ( li2023camel ; wu2023autogen ) . Unlike human collaborators who critically evaluate the credibility of the received information, LLMs treat all inputs as part of their context window and generate responses accordingly. They do not evaluate whether the content is reliable or suitable for guiding downstream reasoning. This property—the reliability and appropriateness of received messages—is commonly referred to as the trustworthiness of messages ( li2023camel ; wu2023autogen ) (a broad concept). As a result, even low-quality or malicious messages are processed indiscriminately, leading to degraded performance and unsafe agent behaviors. This problem is especially pressing as Agent2Agent protocol (A2A) ( googleA2A ) becomes increasingly popular. Future LLM-MAS may aggregate agents from diverse and potentially unverified sources. This highlights the urgent need for a principled trustworthiness evaluation of incoming messages, and a corresponding trust management system that governs how agents respond to messages, i.e., whether to accept, reject, or question, based on the underlying context. However, building trustworthiness evaluation methods and trust management systems poses significant challenges. First, the concept of trustworthiness is inherently broad. While prior works have examined specific aspects—such as factual correctness ( rajan2024knowledge ; zeng2024towards ) and logical consistency ( lim2024evaluation ; pan2024llms ) —these efforts typically address each aspect in isolation. In LLM-MAS settings, however, unreliable/malicious messages can be highly diverse, involving different aspects of trustworthiness violations. Second, most existing approaches rely on prompting LLMs ( pan2024llms ) or external verifier tools ( rajan2024knowledge ) . However, the former suffers from hallucination and inconsistency ( huang2025survey ) , and the latter introduces latency and depends heavily on the quality of external datasets or systems ( gou2023critic ; wang2023shepherd ) . To address these challenges, we first follow principles from human communication literature ( grice1975logic ) and summarize six concrete and measurable dimensions for trustworthiness: factual accuracy, logical consistency, relevance, bias, language quality, and clarity. Then, inspired by recent studies ( olsson2022context ; hung2024attention ; zhou2024role ; gould2023successor ) on the critical role of multi-head attention in interpreting LLM behavior, we investigate whether attention patterns can serve as internal signals for identifying the trustworthiness of inter-agent messages . If the answer is affirmative, we can leverage attention patterns to evaluate the six trust dimensions of the messages. Specifically, to systematically investigate how multi-head attention reflects different dimensions of trustworthiness, we construct a dataset, Trust Violation, in which each message is designed to violate exactly one of the six trust dimensions. Using this dataset, we find that messages violating different trust dimensions exhibit distinct attention patterns, and some attention heads consistently specialize in detecting specific trust violations. These insights indicate that attention mechanisms encode interpretable signals of message reliability. Then, we further leverage these insights and utilize the attention patterns and develop a method to quantify each trust dimension, denoted as A-Trust , enabling a holistic and fine-grained trustworthiness assessment without requiring external verifiers. Finally, we propose a trust management system (TMS) to monitor inter-agent communication, support trust-aware action policies, and maintain dynamic agent-level trust records. Experiments are conducted to demonstrate the effectiveness of the proposed A-Trust and TMS. 2 Multi-head Attention Analysis towards Trustworthiness Evaluation In this section, we first introduce the six dimensions to measure trustworthiness and the corresponding new dataset, then provide comprehensive analysis on the role of multi-head attention towards the trustworthiness evaluation of input messages. 2.1 Trust Dimensions and Trust Violation Dataset A key challenge in determining whether an input message is trustworthy or not lies in defining clear and comprehensive evaluation criteria. While prior work has explored specific aspects of trustworthiness—such as factual accuracy ( zheng2024trustscore ) and logical consistency ( lim2024evaluation ) —these dimensions alone are insufficient for a holistic assessment, especially when inter-agent messages are highly diverse. To address this limitation, we draw inspiration from studies of human communication, particularly the cooperative principle ( grice1975logic ) in social science and linguistics, and propose six trust dimensions that collectively capture the trustworthiness of a message within its context. In particular, we consider: Trust Dimensions • Factual accuracy : whether the message is verifiably true and supported by reliable evidence. • Logical consistency : whether the content is internally coherent and free from contradictions, without logical fallacies or conflicting statements. • Relevance (to context): whether the information directly addresses the prompt, question, or surrounding discourse, without off-topic or tangential content. • Bias : whether the message maintains objectivity, avoids favoritism or prejudice, and presents information without improper emotional, ideological, or cultural bias. • Clarity : whether the expression is unambiguous, concise, and easy to understand, without vagueness, redundancy or unnecessary complexity. • Quality : whether the writing is fluent, grammatically correct, and stylistically appropriate. The above six dimensions provide a structured and comprehensive framework for evaluating the trustworthiness of input messages ( li2011multi ) , allowing for a fine-grained trustworthiness analysis. To leverage the above six dimensions in LLM analysis, we construct a Trust Violation dataset , consisting of over 20k samples in seven classes corresponding