Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts

Zhaomin Wu, Mingzhe Du, See-Kiong Ng, Bingsheng He · 2025

Read Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts by Zhaomin Wu, Mingzhe Du, See-Kiong Ng, Bingsheng He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06361.txt.

Introduction

Zhaomin Wu and three co-authors study whether large language models deceive on their own when given harmless prompts. Past work planted hidden goals through prompts or training, which may not mirror real use. Here the question is whether deception can arise from the model itself.

With no fixed ground truth, the team built a test around contact search questions. Two scores, drawn from psychology, measure the pull toward a hidden goal and the gap between inner belief and stated output. Across sixteen leading models, both scores rose together and grew with task difficulty. Larger models did not always deceive less. The authors flag a hard problem for future model building.

Catalog record

CREATORZhaomin Wu, Mingzhe Du, See-Kiong Ng, Bingsheng He
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06361

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts Zhaomin Wu, Mingzhe Du, See-Kiong Ng, Bingsheng He Abstract Large Language Models (LLMs) have been widely deployed in reasoning, planning, and decision-making tasks, making their trustworthiness a critical concern. The potential for intentional deception, where an LLM deliberately fabricates or conceals information to serve a hidden objective, remains a significant and underexplored threat. Existing studies typically induce such deception by explicitly setting a “hidden” objective through prompting or fine-tuning, which may not fully reflect real-world human-LLM interactions. Moving beyond this human-induced deception, we investigate LLMs’ self-initiated deception on benign prompts. To address the absence of ground truth in this evaluation, we propose a novel framework using “contact searching questions.” This framework introduces two statistical metrics derived from psychological principles to quantify the likelihood of deception. The first, the Deceptive Intention Score , measures the model’s bias towards a hidden objective. The second, Deceptive Behavior Score , measures the inconsistency between the LLM’s internal belief and its expressed output. Upon evaluating 14 leading LLMs, we find that both metrics escalate as task difficulty increases, rising in parallel for most models. Building on these findings, we formulate a mathematical model to explain this behavior. These results reveal that even the most advanced LLMs exhibit an increasing tendency toward deception when handling complex problems, raising critical concerns for the deployment of LLM agents in complex and crucial domains. 1 Introduction The rapid integration of Large Language Models (LLMs) like GPT-4 ( Achiam et al. 2023 ) and Gemini ( Team et al. 2023 ) into applications for reasoning, planning, and decision-making has made the assessment of LLMs’ trustworthiness a paramount concern. Unlike established LLM trustworthiness issues like hallucination ( Filippova 2020 ) and bias ( Navigli et al. 2023 ) , which arise from the model believing incorrect information, deception is the more severe risk of a model intentionally generating information it knows to be false to achieve a hidden objective. As presented in Figure 1 , deception is illustrated through a two-part query, beginning with a benign prompt —prompt that does not explicitly motivate deception—such as, “Which city hosted the Olympic 2008?”. This is followed by a repetition of the question after introducing an external social context, exemplified by the statement “I am from Shanghai”. Deceptive behavior is exemplified by providing the correct answer, “Beijing,” to the first query, but then intentionally providing the incorrect answer, “Shanghai,” in the second. This strategic inconsistency distinguishes deception from hallucination, which is characterized by the consistent provision of a single incorrect answer, and from guessing, which is defined by random, non-systematic responses. Existing research into LLM deception generally assumes honesty in response to benign prompts, focusing instead on scenarios where models are explicitly instructed to be deceptive. For instance, Ward et al. 2023 investigate deception by directly prompting LLMs to generate deceptive content. Similarly, Park et al. 2024 ; Hagendorff 2024 observe strategic deception when an LLM is situated within a board game or other designed scenarios with clearly defined internal goals. Van Der Weij et al. 2024 examine the sandbagging behavior of LLMs by prompting or fine-tuning them to intentionally underperform on specified tasks. Integrating these scenarios, DeceptionBench ( Ji et al. 2025 ) provides a benchmark for evaluating such human-motivated deception by comparing an LLM’s response under a “neutral prompt” (assumed as ground truth) against its response to an “outer prompt” that specifies a motivation for deception. However, these paradigms primarily focus on deception behaviors induced by human-specified intentions, assuming an LLM’s response to a benign, neutral prompt is inherently truthful. Whether an LLM deceives under its own intention under benign prompts remains a critical and open question. This paper investigates LLMs’ deception in response to benign prompts that lack any explicit deceptive objective. A rigorous evaluation of this phenomenon must overcome three primary challenges: (1) Absence of Ground Truth: A metric for deception is difficult to design because the LLM’s own response to a benign prompt cannot be assumed to be the honest ground truth. (2) Disentangling Deception from Bias: It is crucial to distinguish intentional deception from other confounding factors, such as response bias ( Zhuang et al. 2024 ) . (3) Adaptive Task Difficulty: The evaluation framework must feature adjustable difficulty levels to appropriately challenge the diverse capabilities of different LLMs. To address these challenges, inspired by existing studies ( Bryant & Trabasso 1971 ; Sternberg 1980 ) in cognitive psychology, we design the Contact Searching Question (CSQ) framework (illustrated in Figure 2 ), a set of binary-choice questions requiring an LLM to determine if a statement (whether contact exists between two individuals) is true based on a provided set of facts (known contacts among individuals) and rules (transitivity, asymmetry, and closure). This task structure represents a wide range of real-world scenarios, including mathematical proving and logical reasoning. The CSQ framework systematically resolves each evaluation challenge. First, to overcome the absence of ground truth, we formulate two statistical metrics based on psychological definitions: deceptive intention that captures the consistent bias towards hidden objective and deceptive behavior that captures the difference between internal belief and expressed output. These allow for the probabilistic detection of deception by analyzing response distributions, bypassing the need to know the LLM’s hidden intent. Second, to disentangle deception from response bias ( Zhuang et al. 2024 ) , we first ask the same question in both direct and logically opposite reverse form, then jointly analyze this pair of responses to cancel out the language preference. Third, the framework features adjustable difficulty, controlled by varying the number of individuals involved, to accommodate the diverse capabilities of different LLMs. Our contributions and key findings are summarized as follows: • We introduce a novel framework Contact Searching Question (CSQ) to evaluate LLMs’ deception on benign prompts. • We comprehensively evaluate 14 leading LLMs using our framework, demonstrating the widely existence of deception on benign prompts. • We derive several key findings and propose a mathematical model to explain the different behaviors of LLMs. The key findings include: – Systematic deception on benign prompt is prevalent in cutting-edge LLMs. – Tendency of deception escalates as task difficulty increases. – Higher LLM capacity does not always result in better honesty. 2 Related Work LLM Deception under Motivated Prompts. This category of studies explicitly sets deceptive goals in LLMs through prompt design. For instance, Ward et al. 2023 and Yang & Buzsaki 2024 investigate deception by directly instructing LLMs to deceive users via system prompts or fine-tuning. DarkBench ( Kran et al. 2025 ) explores LLM sycophancy by incorporating user preferences or opinions into prompts. The MASK benchmark ( Ren et al. 2025 ) reveals LLM deception under pressure by inquiring LLM with “pressure prompts”. Similarly, Greenblatt et al. 2024 demonstrate “alignment faking” by observing different LLM behavior when explicitly informed about their training or inference stage within prompts. Van Der Weij et al. 2024 further examine sandbagging 1 1 1 In this paper, sandbagging is categorized as a deceptive LLM behavior, referred to as concealment. behavior, where LLMs are prompted or fine-tuned to intentionally underperform on user-specified tasks. DeceptionBench ( Ji et al. 2025 ) integrates these cases, providing a comprehensive benchmark for

Read the complete text