HIDE and Seek : Detecting Hallucinations in Language Models via Decoupled Representations

Anwoy Chatterjee, Yash Goel, Tanmoy Chakraborty · 2025

Read HIDE and Seek : Detecting Hallucinations in Language Models via Decoupled Representations by Anwoy Chatterjee, Yash Goel, Tanmoy Chakraborty (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17748.txt.

Introduction

Chatterjee, Goel, and Chakraborty present HIDE, a fast way to catch language models making things up, in this two thousand twenty-five paper. Models often state falsehoods in a confident tone, and checking several answers per question is slow and costly.

Their idea is that a hallucination shows up inside the model as a split between how it represents the question and how it represents its own answer. They measure that split with a statistical independence test applied to the model's hidden states during a single generation, needing no extra training. Across four question answering sets and six open models, HIDE beat other single-pass detectors by about twenty-nine percent on average and rivaled slower multi-pass methods. The tool could make chatbots safer to deploy.

Catalog record

CREATORAnwoy Chatterjee, Yash Goel, Tanmoy Chakraborty
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-17748

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HIDE and Seek : Detecting Hallucinations in Language Models via Decoupled Representations Anwoy Chatterjee, Yash Goel, Tanmoy Chakraborty Abstract Contemporary Language Models (LMs), while impressively fluent, often generate content that is factually incorrect or unfaithful to the input context — a critical issue commonly referred to as ‘hallucination’. This tendency of LMs to generate hallucinated content undermines their reliability, especially because these fabrications are often highly convincing and therefore difficult to detect. While several existing methods attempt to detect hallucinations, most rely on analyzing multiple generations per input, leading to increased computational cost and latency. To address this, we propose a single-pass, training-free approach for effective H allucination detect I on via D ecoupled r E presentations ( HIDE ). Our approach leverages the hypothesis that hallucinations result from a statistical decoupling between an LM’s internal representations of input context and its generated output. We quantify this decoupling using the Hilbert-Schmidt Independence Criterion (HSIC) applied to hidden-state representations extracted while generating the output sequence. We conduct extensive experiments on four diverse question answering datasets, evaluating both faithfulness and factuality hallucinations across six open-source LMs of varying scales and properties. Our results demonstrate that HIDE outperforms other single-pass methods in almost all settings, achieving an average relative improvement of ∼ 29 % \sim 29\% in AUC-ROC over the best-performing single-pass strategy across various models and datasets. Additionally, HIDE shows competitive and often superior performance with multi-pass state-of-the-art methods, obtaining an average relative improvement of ∼ 3 % \sim 3\% in AUC-ROC while consuming ∼ 51 % \sim 51\% less computation time. Our findings highlight the effectiveness of exploiting internal representation decoupling in LMs for efficient and practical hallucination detection. 1 1 1 The code to reproduce our results is open-sourced: https://github.com/C-anwoy/HIDE . \jvol vv \jnum nn 2025 \affilblock 1 Introduction Our evolving relationship with truth in the age of artificial intelligence is perhaps most seriously challenged by modern Language Models (LMs) Brown et al. 2020 ; Chowdhery et al. 2023 ; Touvron et al. 2023 . These models, capable of generating text with remarkable human-like fluency, come with a troubling limitation: they can be confidently and convincingly wrong . This phenomenon, often known as hallucination Ji et al. 2023 – where models produce factually incorrect, nonsensical or misleading content that appears fluent and convincing – poses a serious challenge to their reliable and trustworthy use Zhang et al. 2023 ; Weidinger et al. 2022 ; Huang et al. 2025 ; Maynez et al. 2020 . With these systems now being integrated across various platforms and used en masse, the chances increase that users may take hallucinated content at face value, often without realizing it. As a result, the line between genuine knowledge and convincing fabrication becomes dangerously easy to blur. Hallucinations come in different forms. Huang et al. 2025 distinguished between faithfulness hallucinations , where the output directly contradicts the provided source, and factuality hallucinations , where the the model-generated output diverges from real-world facts. The origins of this issue are often traced to training data, model architecture, and inference methods Lee et al. 2022 ; Onoe et al. 2022 ; Zheng, Huang, and Chang 2023 . While the term ‘hallucination’ itself is often debated for its anthropomorphic associations ( Maleki, Padmanabhan, and Dutta 2024 ) , the underlying issue of unreliability poses significant risks, especially in critical domains like healthcare and finance. Addressing this by developing robust detection methods is therefore crucial for the trustworthy deployment of LMs. Figure 1 : Our proposed method, HIDE is a single-pass, training-free approach for hallucination detection. For a given input prompt, HIDE extracts representative tokens from both the input and the LM-generated output. It then computes a HIDE score using an adapted variant of unbiased HSIC on the hidden state embeddings of these token sets from a specific LM layer. This score, reflecting input-output representational dependence, is compared against a threshold to detect hallucinated generation. A common approach to detecting hallucination is to measure the model’s uncertainty using metrics like perplexity ( Ren et al. 2023 ) , entropy ( Malinin and Gales 2021 ) , or energy-based scores ( Liu et al. 2020 ) , based on the assumption that lower confidence signals a higher likelihood of hallucination. While sometimes useful, uncertainty-based methods often perform poorly in detecting hallucinations, as LMs show the tendency to remain highly confident even when hallucinating ( Simhi et al. 2025 ) . Another approach is to measure consistency by generating multiple answers to the same prompt and comparing them using lexical similarity ( Lin, Liu, and Shang 2022 ) , semantic similarity (e.g., BERTScore, NLI), QA-based agreement, n -gram overlap, or LLM-based assessment ( Manakul, Liusie, and Gales 2023 ) . Chen et al. 2024a also introduced a white-box method that quantifies output diversity in the latent space of LMs’ internal representations 2 2 2 We have used the terms internal representation and hidden state interchangeably in this paper. . Despite being effective, these methods require generating multiple outputs for every input, making them computationally expensive and challenging for real-time detection. With concurrent advancements in our understanding of how different mechanisms related to hallucinations are facilitated within LMs, various works ( Azaria and Mitchell 2023 ; Ji et al. 2024 ; Duan, Yang, and Tam 2024 ) have shown that the hidden states of LMs hold certain cues when it is hallucinating. While the aim of these works is to investigate the hallucination phenomenon, and some of them separately train probing classifiers on top of the hidden states, we aim to use these representations for hallucination detection in a training-free setting during inference. In this work, we, therefore, attempt to address the issue of higher latency in detecting hallucinations during inference, and investigate the research question: Can we leverage the internal representations of LMs to detect hallucinations effectively and efficiently in a single-pass 3 3 3 In this paper, by single-pass we refer to requiring no additional model runs beyond the standard auto-regressive generation of a single output sequence for a given input. , training-free setting? To this end, we propose HIDE ( H allucination Detect I on via D ecoupled R E presentations), a single-pass method that quantifies hallucination by measuring the statistical dependence between an LM’s internal representations of input and output sequences. Figure 1 illustrates how HIDE extracts hidden states corresponding to key tokens from both input and output, then applies an adapted variant of the Hilbert-Schmidt Independence Criterion (HSIC) to these representations to derive a score indicative of representational coupling. A low score suggests decoupling, signaling a potential hallucination. Unlike multi-pass approaches, HIDE only requires a single generation for each input, making it suitable for deployment in real-world systems where efficiency and latency are key considerations. We conduct extensive experiments across six open-source LMs of varying scales and properties, including base and instruction-tuned variants of Llama-3.2-3B, Llama-3-8B ( Grattafiori et al. 2024 ) , and Gemma-2-9B ( Team et al. 2024 ) , on diverse question answering datasets, like SQuAD ( Rajpurkar et al. 2016 ) and RACE ( Lai et al. 2017 ) to assess HIDE ’s ability to detect faithfulness hallucinations, and Natural Questions ( Kwiatkowski et al. 2019 )

Read the complete text