The Impact of Negated Text on Hallucination with Large Language Models

Jaehyung Seo, Hyeonseok Moon, Heuiseok Lim · 2025

Read The Impact of Negated Text on Hallucination with Large Language Models by Jaehyung Seo, Hyeonseok Moon, Heuiseok Lim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20375.txt.

Introduction

Jaehyung Seo, Hyeonseok Moon, and Heuiseok Lim study how negated text affects hallucination detection in large language models, a question earlier hallucination research left largely unexplored. They pose three research questions about whether models recognize the contextual shifts that negation causes and whether they can judge hallucinations in negated text as reliably as in affirmative text. To test this, they built the NegHalu dataset by rewriting existing hallucination datasets with negated expressions.

Experiments show models struggle to detect hallucinations in negated text, often giving logically inconsistent or unfaithful judgments. The authors also trace model internal states at the token level while negated inputs are processed, revealing why mitigation is hard. Their analysis points to negation handling as a weak spot in current hallucination defenses.

Catalog record

CREATORJaehyung Seo, Hyeonseok Moon, Heuiseok Lim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-20375

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Impact of Negated Text on Hallucination with Large Language Models Jaehyung Seo, Hyeonseok Moon, Heuiseok Lim Abstract † † † Corresponding Author Recent studies on hallucination in large language models (LLMs) have been actively progressing in natural language processing. However, the impact of negated text on hallucination with LLMs remains largely unexplored. In this paper, we set three important yet unanswered research questions and aim to address them. To derive the answers, we investigate whether LLMs can recognize contextual shifts caused by negation and still reliably distinguish hallucinations comparable to affirmative cases. We also design the NegHalu dataset by reconstructing existing hallucination detection datasets with negated expressions. Our experiments demonstrate that LLMs struggle to detect hallucinations in negated text effectively, often producing logically inconsistent or unfaithful judgments. Moreover, we trace the internal state of LLMs as they process negated inputs at the token level and reveal the challenges of mitigating their unintended effects. 1 Introduction The rapid advancement of large language models (LLMs) continues to drive the release of diverse open-source models capable of performing a wide range of tasks ( Touvron et al. 2023 ; Jiang et al. 2023 ; Team et al. 2024 ) . As these models become more prevalent, the ability to distinguish whether generated outputs contain hallucinations is becoming increasingly critical ( Magesh et al. 2024 ) . Detecting hallucination involves identifying content that is either contextually unfaithful or contradictory to real-world facts and assessing the truthfulness of such outputs ( Ji et al. 2023a ; Zhang et al. 2023 ; Huang et al. 2023a ) . Figure 1: Examples illustrating how negation can flip the hallucination label. The negated response introduces or resolves contradictions with the given knowledge. Recent research on hallucination detection actively focuses on improving the reliability of LLMs by identifying their limitations and refining models based on insights into hallucinated outputs ( Manakul et al. 2023 ; Jiang et al. 2024 ; Chen et al. 2024 ) . However, there is a limited exploration of how negated text affects hallucination detection with LLMs. Negated text, which includes negation markers (e.g., "not," "never," "no," "without"), is commonly used in everyday communication ( Gubelmann and Handschuh 2022 ; Hossain et al. 2022 ) . Although these markers are typically single tokens, they exert a disproportionately large influence on the overall processing factuality of a sentence, fundamentally altering its meaning ( Vanek et al. 2024 ) . However, unlike humans, LLMs struggle to effectively handle negation and infer contextual meaning ( Truong et al. 2023 ; Ye et al. 2023 ) . Moreover, negative knowledge can introduce hallucinations into commonsense reasoning ( Chen et al. 2023 ; Seo et al. 2024 ) , as LLMs tend to misrepresent negation as a faulty logical operator, leading to severe hallucinations ( Bhar and Asher 2024 ) . In this paper, we present an initial exploration of how negated text influences hallucination detection in LLMs. We propose three open research questions and work towards answering them. • RQ1. Can LLMs distinguish between hallucinations and faithful statements in negated text as effectively as in affirmative text? • RQ2. Can the model internally recognize differences caused by negation when detecting hallucinations? • RQ3. Can targeted intervention strategies improve hallucination detection in the negated text? To address these research questions, we investigate whether LLMs can recognize contextual shifts and reliably detect hallucinations in the presence of negated text. As illustrated in Figure 1 , we conduct this analysis by reconstructing existing hallucination detection benchmarks with negated expressions and introducing NegHalu, a dataset in which hallucination labels are newly assigned to account for the effects of negation. Our experiments show that Llama-2-7B ( Touvron et al. 2023 ) , Llama-3-8B ( AI@Meta 2024 ) , and Mistral-7B-v0.3 ( Jiang et al. 2023 ) exhibit performance degradation in 17 out of 18 post-negated hallucination detection cases. Furthermore, when the same content is expressed in both affirmative and negated forms, models tend to exhibit a bias toward classifying post-negated scenarios as hallucinations rather than faithful statements. We demonstrate that the influence of negated text on hallucination detection extends across multiple tasks, including question answering (QA), dialogue, summarization, and completion, indicating its task-agnostic. Moreover, this phenomenon is observed across a diverse range of domains, including 10 general topics, science, and autobiographies, suggesting its broad applicability. For in-depth analyses, we employ lens observation to examine the internal states of LLMs during hallucination detection. Furthermore, we examine approaches to alleviate the impact of negated text without resorting to unrealistic external modules or excessive parameter modifications, assessing the effectiveness of in-context learning ( Brown et al. 2020 ) , Chain-of-Thought (CoT) reasoning ( Wei et al. 2022 ) , and knowledge editing ( Fang et al. 2024 ) as potential solutions. 2 Related Work Negated Text in LLMs Handling negated text has been a long-standing challenge in NLP. Minsky 1997 emphasized the importance of understanding negated expressions and meanings, highlighting the need to integrate negation into NLP systems. Building on this foundation, Morante et al. 2011 analyzed how negation operates within the text, providing the essential groundwork for subsequent studies. Further exploration aimed to understand the effects of negation on semantic structures and meaning ( van Son et al. 2016 ; Khandelwal and Sawant 2020 ) . Kassner and Schütze 2020 and Hossain et al. 2022 demonstrated that models frequently overestimate their confidence in predictions, leading to errors when processing negated inputs. Arnaout et al. 2022 and Chen et al. 2023 revealed that negated knowledge can introduce biases into LLMs, further complicating their performance. Truong et al. 2023 observed that even as the model size increases, the ability to effectively handle negation does not necessarily improve. Moreover, Ye et al. 2023 found that negation can cause significant performance drops, even when advanced strategies like chain-of-thought reasoning are employed. Most recently, Bhar and Asher 2024 highlighted how negation can lead to unique types of hallucinations in tasks such as natural language inference. Hallucination in LLMs The issue of hallucination in LLMs has gained increasing importance as these models are applied to various NLP tasks. Hallucination occurs when the output generated by an LLM either lacks logical consistency with the input or contradicts real-world facts ( Ji et al. 2023a ; Huang et al. 2023a ) . This phenomenon has been observed across a range of tasks, including machine translation ( Dale et al. 2023 ; Guerreiro et al. 2023 ) , summarization ( Zhao et al. 2020 ; Choubey et al. 2023 ) , and dialogue generation ( Ji et al. 2023b ) . It is particularly problematic in high-stakes domains that demand high reliability, such as law ( Magesh et al. 2024 ) , medicine ( Farquhar et al. 2024 ) , and science ( Dong et al. 2024b ) . To address hallucinations, recent research has advanced both detection and mitigation techniques ( Ji et al. 2023a ; Huang et al. 2023a ; Zhang et al. 2023 ) . Detection strategies range from word-level and sentence-level analysis ( Huang et al. 2023b ; Yang et al. 2023 ) to self-verification via sampling ( Manakul et al. 2023 ) and methods employing eigen-scores ( Chen et

Read the complete text