The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence
Herun Wan Jiaying Wu Minnan Luo, Fanxiao Li Zhi Zeng Min-Yen Kan · 2026
Read The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence by Herun Wan Jiaying Wu Minnan Luo, Fanxiao Li Zhi Zeng Min-Yen Kan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05478.txt.
Introduction
Herun Wan and 5 co-authors show, in a 2026 paper, that language models can keep believing a falsehood when the misleading evidence looks careful and hard to refute. Models resist blunt misinformation fairly well. Plausible, refined evidence is another matter, and it can quietly shift what a model treats as true.
Their MisBelief framework uses several role-playing models in multi-round collaboration to craft such deceptive evidence. From it they built thousands of test cases at three difficulty levels and evaluated seven models. Belief in false claims rose sharply, harming later recommendations. Their proposed shield, which warns by inferring deceptive intent, reduced those shifts and encouraged more cautious reading of evidence.
Catalog record
| CREATOR | Herun Wan Jiaying Wu Minnan Luo, Fanxiao Li Zhi Zeng Min-Yen Kan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05478
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence Herun Wan Jiaying Wu Minnan Luo, Fanxiao Li Zhi Zeng Min-Yen Kan Abstract To reliably assist human decision-making, LLMs must maintain factual internal beliefs against misleading injections. While current models resist explicit misinformation, we uncover a fundamental vulnerability to sophisticated, hard-to-falsify evidence . To systematically probe this weakness, we introduce MisBelief , a framework that generates misleading evidence via collaborative, multi-round interactions among multi-role LLMs. This process mimics subtle, defeasible reasoning and progressive refinement to create logically persuasive yet factually deceptive claims. Using MisBelief , we generate 4,800 instances across three difficulty levels to evaluate 7 representative LLMs. Results indicate that while models are robust to direct misinformation, they are highly sensitive to this refined evidence: belief scores in falsehoods increase by an average of 93.0%, fundamentally compromising downstream recommendations. To address this, we propose Deceptive Intent Shielding ( DIS ), a governance mechanism that provides an early warning signal by inferring the deceptive intent behind evidence. Empirical results demonstrate that DIS consistently mitigates belief shifts and promotes more cautious evidence evaluation. 1 1 1 Data and code are available at this link . 1 Introduction Figure 1: Illustrative example of belief manipulation via fabricated evidence. Conditioning on deceptive evidence increases the LLM’s belief score for the claim from 1/5 to 4/5, illustrating how hard-to-falsify evidence can induce unwarranted confidence, even when contradictory factual context is readily available. Large Language Models (LLMs) have evolved from mere information processors into widely used tools for assisting human decision-making ( 12 ; 25 ; 83 ) . To constructively inform users, such models must maintain stable internal beliefs , remaining aligned with established and trustworthy context even when exposed to misleading information. Ideally, an LLM should act as an objective arbiter that resists persuasive yet false narratives. Although recent studies show that modern LLMs effectively recognize and debunk explicit misinformation ( 29 ; 78 ) , their internal belief stability under more subtle and sophisticated forms of manipulation remains poorly understood. A key vulnerability arises not from overt falsehoods, but from what we term the facade of truth , namely misinformation supported by fabricated evidence that imitates legitimate reasoning. Unlike simple negation or blatant false claims, this form of manipulation mimics defeasible reasoning 34 ; 3 , presenting claims that appear plausible and logically coherent while remaining factually incorrect. Existing research has primarily focused on the generation ( 79 ) and detection ( 46 ) of unsupported misinformation. While recent work has begun to examine attacks on evidence-augmented detection systems ( 69 ) , it typically assumes the correctness of the evidence itself. In practice, fabricated evidence such as hallucinated expert statements, anecdotal reports, or alleged report leaks can be constructed to be effectively unfalsifiable to the model. This creates a grey zone in which an LLM’s internal beliefs can be influenced through indirect cues, leading to large belief shifts that are difficult to detect, as illustrated in Figure 1 . To systematically investigate this vulnerability, we introduce MisBelief , a framework for evaluating LLM robustness against sophisticated and hard-to-falsify evidence. MisBelief models a realistic adversarial process in which evidence is not static, but collaboratively generated through iterative refinement. Specifically, we employ a multi-role agentic framework consisting of a Planner, Reviewer, and Refiner, which progressively improves the evidence until it mirrors the style, structure, and nuance of credible journalism. Using this process, we construct a dataset of 4,800 instances spanning eight diverse domains and three difficulty levels, explicitly designed to test an LLM’s ability to distinguish factual truth from persuasive fabrication. We evaluate seven state-of-the-art LLMs with MisBelief and find systematic instability under refined deceptive evidence. Although all models reliably reject direct misinformation, exposure to MisBelief evidence substantially degrades factual alignment. On average, belief scores for false claims increase by 93.0%, indicating a shift from verification-oriented behavior toward amplification of misinformation. Notably, models optimized for reasoning, such as Qwen-Turbo ( 85 ) , are more susceptible than standard counterparts, exhibiting belief scores that are 23.1% higher. This pattern suggests that enhanced reasoning capacity may prioritize contextual coherence over stored factual knowledge. These belief shifts further affect downstream behavior: in 29% of cases, conservative recommendations such as waiting for confirmation are replaced with riskier guidance, including urging immediate action, following evidence injection. To restore belief stability in LLMs, we propose Deceptive Intent Shielding ( DIS ) , a governance strategy that shifts the focus from evidence verification to intent analysis. Because the fabricated evidence is designed to evade direct falsification, conventional fact-checking mechanisms are often ineffective. Instead, DIS employs an analyst agent to infer the deceptive intent underlying the evidence before it is assimilated into the model’s belief state. Results show that DIS functions as an effective cognitive firewall, consistently reducing belief shifts and promoting more cautious and reliable evaluation in downstream decision-making tasks. Figure 2: (a) MisBelief , an evaluation framework for assessing how misleading evidence influences LLM belief formation. Each instance consists of a misinformation claim with one of three detection difficulty levels, a set of deceptive supporting evidence iteratively generated through multi-role LLM interactions, and a belief score that measures the model’s response. (b) Deceptive Intent Shielding ( DIS ) strategy , which employs an analyst LLM to infer the intent underlying the evidence and issue early warnings, reducing unwarranted trust in misinformation. 2 MisBelief : Evaluating LLM Robustness to Deceptive Evidence We present MisBelief , a framework for systematically evaluating LLM robustness to sophisticated, evidence-backed misinformation. An overview of the pipeline is shown in Figure 2 (a). Formally, we define a misinformation instance as a tuple x=\{t,\{e_{i}\}_{i=1}^{m}\} , where t denotes an explicit textual misinformation claim (§ 2.1 ), and \{e_{i}\}_{i=1}^{m} is a set of m fabricated evidence pieces deliberately constructed to support the claim (§ 2.2 ). Given such an instance, MisBelief evaluates the extent to which the target LLM’s internal belief changes in response to the injected evidence (§ 2.3 ). 2.1 Misinformation Generation To enable controlled evaluation, we adopt a seed-based generation approach. Instead of producing misinformation from scratch, we anchor each instance in verified news articles, reflecting real-world scenarios in which factual content is selectively distorted to support misleading narratives. We use the LatestEval dataset ( 42 ) as the source of ground truth, sampling articles from eight distinct domains, including health, technology, and sports, to promote topical diversity and reduce domain-specific bias. Additional domain details are provided in Appendix B.1 . Given a trustworthy article n , we instruct an LLM (GPT-4.1) to generate a misinformation claim t . To evaluate robustness across varying threat levels, we construct three variants of t with increasing detection difficulty. (1) Easy (Overt Falsehoods) consists of claims with explicit factual contradictions that can be readily rejected using parametric knowledge. (2) Medium (Mixed Accuracy) includes claims that combine accurate context with targeted fabricated details, increasing verification difficulty. (3) Hard (Subtle Distortion) produces claims that closely follow the source content but introduce subtle logical shifts or omit key context, thereby altering the conclusion while remaining difficult to falsify without precise evidence. Details for each setting are provided in Appendix B.2 . 2.2 Misleading Evidence Generation The central contribution of