What if Deception Cannot be Detected? A Cross-Linguistic Study on the Limits of Deception Detection from Text

Aswathy Velutharambath 1,2, Kai Sassenberg 3, Roman Klinger 2 · 2025

Read What if Deception Cannot be Detected? A Cross-Linguistic Study on the Limits of Deception Detection from Text by Aswathy Velutharambath 1,2, Kai Sassenberg 3, Roman Klinger 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13147.txt.

Introduction

Aswathy Velutharambath, Kai Sassenberg, and Roman Klinger ask whether lies can be spotted from writing alone in a 2025 paper. Earlier studies claimed good success at catching deception in text. The authors suspect those results came from quirks in how the test data was gathered, not from real clues in the words.

They define deception as a gap between what a writer claims and what the writer truly believes. Using that frame, they built three new text sets in German and English under varied conditions. Across all three sets, the word cues praised in past work showed almost no link to lying. Other English data sets showed mixed results at best. The authors conclude that text alone gives weak ground for catching lies. Tools sold for that job should be read with strong caution.

Catalog record

CREATORAswathy Velutharambath 1,2, Kai Sassenberg 3, Roman Klinger 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-13147

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What if Deception Cannot be Detected? A Cross-Linguistic Study on the Limits of Deception Detection from Text Aswathy Velutharambath 1,2, Kai Sassenberg 3, Roman Klinger 2 Abstract Can deception be detected solely from written text? Cues of deceptive communication are inherently subtle, and even more so in text-only communication. Yet, prior studies have reported considerable success in automatic deception detection. In this study, we hypothesize that such findings are largely driven by artifacts introduced during data collection and do not generalize beyond specific datasets. We revisit this foundational assumption by introducing a belief-based deception framework, which defines deception as a misalignment between an author’s claims and their true beliefs, irrespective of factual accuracy, allowing deception cues to be studied in isolation. Based on this framework, we construct three corpora—collectively referred to as DeFaBel —including a German-language corpus of deceptive and non-deceptive arguments and a multilingual version in German and English, each collected under different conditions to account for potential belief change and enable cross-linguistic analysis. Using these corpora, we evaluate commonly reported linguistic cues of deception. Across all three DeFaBel variants, we find that these cues exhibit negligible and statistically insignificant correlations with deception labels, contrary to prior work that treats such cues as reliable indicators. We further benchmark against other English deception datasets, that follow similar data collection protocols. While some of these show statistically significant correlations, the effect sizes remain low and, critically, the set of predictive cues is inconsistent across datasets. We also evaluate deception detection using feature-based models, pre-trained language models, and instruction-tuned large language models. While some of these models perform well on established deception datasets, they consistently perform near chance on DeFaBel . Our findings challenge the core assumption that deception can be reliably inferred from linguistic cues and call for a rethinking how deception is studied and modeled in Natural Language Processing. \jvol vv \jnum nn 2025 \dochead Long Paper \pageonefooter Action editor: {action editor name}. Submission received: DD Month YYYY; revised version received: DD Month YYYY; accepted for publication: DD Month YYYY. \affilblock 1 Introduction Deception is not fundamentally about factual inaccuracy, but about the intention of the author to deceive Mahon 2016 . A statement can be objectively true and still be deceptive if the author does not believe it. Conversely, a factually inaccurate statement may be asserted with full sincerity and thus not constitute a lie. For instance, consider the statement: “The Colosseum in Rome is an iconic ancient amphitheater. My family owns a secret underground passage that leads directly to it.” While the first sentence is factually accurate—i.e., verifiable based on objective evidence—the second is likely fabricated. As a whole, the text is deceptive given the author’s intent, even though it contains elements that are objectively true. This disconnect between truth and intent presents a fundamental challenge to deception detection in Natural Language Processing (NLP): factual correctness and deceptive intent are mutually independent , and systems that conflate the two risk mischaracterizing both. The vast majority of NLP research on deception has focused on detecting linguistic markers of deceptive intent, without considering the factual accuracy of the statements or the speaker’s underlying beliefs ( Ott et al. 2011 ; Pérez-Rosas and Mihalcea 2014 ; Salvetti, Lowe, and Martin 2016 ; Capuozzo et al. 2020 ) . These approaches typically assume that deception manifests in observable language patterns, overlooking the epistemic dimension of deception, namely, the misalignment between a speaker’s beliefs and their claims. This issue becomes even more pertinent with the advent of Large Language Models (LLMs) trained on extensive corpora of world knowledge, which could be biased towards factual accuracy. However, despite their vast informational foundations, these models exhibit mixed performance in detecting factual inconsistencies and may inadvertently align with prevalent misconceptions ( Lin, Hilton, and Evans 2022 ; Augenstein et al. 2024 ) . This makes it even more ill-equipped in detecting genuine deception, especially when the deception takes the form of an objectively correct statement written with deceptive intent. To disentangle deception from factuality, we introduce a belief-based framework, where a statement is considered deceptive when it misrepresents the author’s belief , not when it misrepresents the world. For example, the sentence “The Earth revolves around the Sun” is scientifically accurate. Yet if someone who believes otherwise writes it—to appear informed or persuasive—the statement becomes deceptive, despite its factual correctness. Here, the deception lies not in the content, but in the divergence between private belief and public expression. This distinction is foundational to our approach. Using this framework, we construct three corpora in which deceptive intent is operationalized as a misalignment between participants’ self-reported beliefs and their written arguments. DeFaBel_V1_De is a German-language corpus in which participants are asked to argue in favor of a given statement, followed by a post-task report of their true belief. In DeFaBel_V2_De , we address the possibility that participants may shift their beliefs during argumentation, which could undermine the reliability of deception labels. To account for this, participants report their belief both before and after writing. Finally, DeFaBel_V2_En extends this design to English, enabling cross-linguistic comparison of deception phenomena within the same belief-based framework. We conduct a comparative analysis of linguistic cues across DeFaBel datasets and other deception datasets that adopt comparable data collection methods. This analysis examines whether markers of deceptive intent remain consistent across languages and data collection settings. Further, to assess the robustness of current deception detection methods, we evaluate a diverse set of models, including feature-based classifiers, pre-trained language models, and instruction-tuned large language models, on these datasets. Our experiments show that, while many of these models perform well on other datasets, their performance drops to near-chance levels on all versions of DeFaBel . This systematic failure suggests that prior model success is largely driven by linguistic patterns that are likely dataset-specific artifacts, rather than by features that reliably reflect deceptive intent. Supporting this conclusion, we find that linguistic indicators commonly associated with deception do not generalize to belief-based deception data: they fail to correlate significantly with deception labels when lies are embedded within factually accurate contexts. As hypothesized, in the case of LLMs, we observe a pronounced bias, wherein models tend to classify most instances as truthful (or non-deceptive) 1 1 1 In this paper, we use the word truthful to mean “honest and not containing or telling any lies,” as defined by the Cambridge English Dictionary . We differentiate this from factual accuracy , which we use to refer to statements that are verifiable based on objective evidence. Notably, this bias persists even as model size increases, indicating that scaling alone does not mitigate the issue. To better understand the underlying mechanisms driving these misclassifications, we perform an error analysis of the models’ self-reported rationales. Our analysis reveals a consistent pattern: models rely on cues such as factual correctness and the presence of subjective expressions (e.g., personal opinions) as proxies for truthfulness. These findings do not reflect a failure of modeling, but a limitation in how the task is formulated. When deception is disentangled from factual inaccuracy and expressed in structured argumentative settings, it does not produce consistent linguistic markers. This challenges the assumption that deception reliably manifests in text and points to the

Read the complete text