HalluShift : Measuring Distribution Shifts towards Hallucination Detection in LLMs

Sharanya Dasgupta, Sujoy Nath, Arkaprabha Basu, Pourya Shamsolmoali, Swagatam Das · 2025

Read HalluShift : Measuring Distribution Shifts towards Hallucination Detection in LLMs by Sharanya Dasgupta, Sujoy Nath, Arkaprabha Basu, Pourya Shamsolmoali, Swagatam Das (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09482.txt.

Introduction

Sharanya Dasgupta and co-authors write a 2025 paper on catching false output from language models. Such models can state wrong facts in smooth, confident prose, a failure known as hallucination. The team argues the drift toward error starts inside the model as it writes, in small steps.

Their method, HalluShift, tracks how the model's inner states and word odds shift across a reply. A slide toward a wrong answer leaves a pattern that the method can measure. Tests across several benchmark sets beat prior detectors. The authors liken the slide to how human speech can stay logical in form while small parts quietly go wrong, and they share their code for others.

Catalog record

CREATORSharanya Dasgupta, Sujoy Nath, Arkaprabha Basu, Pourya Shamsolmoali, Swagatam Das
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-09482

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HalluShift : Measuring Distribution Shifts towards Hallucination Detection in LLMs Sharanya Dasgupta, Sujoy Nath, Arkaprabha Basu, Pourya Shamsolmoali, Swagatam Das Abstract Large Language Models (LLMs) have recently garnered widespread attention due to their adeptness at generating innovative responses to the given prompts across a multitude of domains. However, LLMs often suffer from the inherent limitation of hallucinations and generate incorrect information while maintaining well-structured and coherent responses. In this work, we hypothesize that hallucinations stem from the internal dynamics of LLMs. Our observations indicate that, during passage generation, LLMs tend to deviate from factual accuracy in subtle parts of responses, eventually shifting toward misinformation. This phenomenon bears a resemblance to human cognition, where individuals may hallucinate while maintaining logical coherence, embedding uncertainty within minor segments of their speech. To investigate this further, we introduce an innovative approach, Hallushift , designed to analyze the distribution shifts in the internal state space and token probabilities of the LLM-generated responses. Our method attains superior performance compared to existing baselines across various benchmark datasets. Our codebase is available at https://github.com/sharanya-dasgupta001/hallushift . Fig. 1: Comparison of AUC-ROC (%) across different QA datasets for hallucination detection using various methods, evaluated on OPT6.7B (a) and LLaMA-2-7B (b) models. I Introduction The recent era of Large Language Models has transformed various domains through the foundation of transformer architectures [ 1 ] , encoding vast amounts of knowledge within deep learning models. However, a crucial question arises: Are these models innovative enough to produce factually accurate answers while achieving their creative potential? This critical need has paved the way for research on prompt engineering, exploring how to generate more meaningful content through structured and segmented prompts. However, amid progress in the factual accuracy of LLM responses, a critical challenge has emerged: hallucination [ 2 ] . Generally, hallucination occurs when an LLM generates responses that exceed its knowledge boundaries while maintaining linguistic coherence and fluency [ 3 ] . Although hallucination in LLM can manifest in various forms, we specifically address factuality hallucinations, where generated content contains facts that can be grounded in real-world information but present contradictions (factual inconsistency) or contains facts that are unverifiable against established real-world knowledge (factual fabrication) [ 3 , 4 ] . To mitigate this challenge, few-shot prompting, with a few examples for task-specific learning, has been instrumental in improving in-context learning without requiring extensive retraining [ 5 ] . However, it still faces challenges in multistep problem-solving and introduces hallucinations. To address this, Chain-of-Thought (CoT) prompting [ 6 ] , enables LLMs to generate intermediate reasoning steps, thus improving performance on intricate tasks. Despite its effectiveness, CoT is limited in more complex scenarios, encouraging the development of zero-shot CoT approaches that allow models to reason without examples, using reasoning phrases like “Let’s think step by step” to guide logical thinking [ 7 ] , engaging stepwise generations that lead to less hallucinated answers. However, zero-shot CoT often struggles with tasks that require deeper, multistep reasoning. To overcome this, approaches such as Plan-and-Solve prompting [ 8 ] have been introduced, which involve generating a plan before solving the problem and enhancing performance on multistep tasks. Additionally, the Hint of Thought (HoT) prompting technique has further advanced zero-shot reasoning by the importance of explainable substeps [ 9 ] . Furthermore, the Tree of Thoughts prompting [ 10 ] pushes the boundaries of deliberate problem-solving by allowing LLMs to explore multiple reasoning paths. All research leads to mitigating hallucination by considering the LLM as a black box. Our focus is to build on the very root of the LLM that allows us to treat it as a transparent box, where we can dissect it to understand the behaviors that may lead to a solution to the hallucination problem. However, since 2020, Retrieval Augmented Generation (RAG) [ 11 ] , [ 12 ] , [ 13 ] , [ 14 ] , revealed that model training with vast knowledge bases can be bypassed through efficient extraction of meaningful embeddings from static vector databases. RAG significantly reduces hallucinations and allows LLMs to answer out-of-domain questions properly [ 15 ] . RAG mechanisms often prove computationally expensive, particularly for smaller research groups that require real-time access and manipulation of large vector databases. This limitation motivates our focus on developing methodologies accessible to a broader research community by prioritizing computational efficiency and exploring smaller LLM models, which, despite their increased inclination for hallucinations, provide a more practical basis for research. In this article, we propose a hallucination detection mechanism using two key aspects: distribution shift patterns of internal states and token probability features derived from model outputs. We speculate that hallucinated content exhibits distinctive statistical signatures in both the embedding space and prediction probability distributions. By analyzing these patterns, we can identify potential hallucinations without relying on external knowledge sources or complex verification mechanisms. We analyze variations in model dynamics between truthful and hallucinated outputs by examining the shift in the layer distribution within LLMs’ hidden activations. Our hypothesis suggests that the model’s sensitivity to non-factual information manifests in its internal representations, enabling hallucination detection within a single response. In spite of taking directly the hidden states to generate a hallucination score (that is, the likelihood of the generated response being hallucinated), we focus on how the internal states vary over layers during generation. We summarize our contributions as follows, • In contrast to previous approaches [ 16 ] , [ 17 ] , [ 18 ] , we eliminated the dependency on manual selection of the internal layer (i.e., including the final layer) and proposed an automated way to range-wise feature selection across the layers followed by a series of statistical transformations. This guarantees that internal states are not transmitted directly for membership selection and states that hallucinations mostly manifest across multiple layers rather than being confined to specific layers of the model. • We observed that distribution shift and similarity measures offer different views on the internal dynamics of LLM representations. However, probabilistic features capture the word selection patterns by assessing alternative choices available during generation, raising a question about potential interrelationships. Therefore, we present a comprehensive approach that unifies these different aspects, two derived from internal state patterns and one from external generation dynamics, to approximate our membership function. • We validate HalluShift through experiments across diverse benchmark datasets and multiple LLMs. Compared to state-of-the-art approaches, our method demonstrates significantly reduced computational cost. II Related Works The tendency of LLMs to generate diverse but factually incorrect answers is the primary concern in domains such as healthcare and customer acquisition, where factual accuracy is vital. Although Retrieval Augmented Generation (RAG) [ 11 ] and large models [ 19 , 20 ] with more parameters have shown promising performance in reducing hallucinations, these solutions remain questionable for resource-constrained environments and smaller models. Our approach focuses on two popular open-source LLM families: LLaMA and OPT. LLaMA [ 21 , 22 ] comprises foundation models ranging from 1B to 65B parameters, trained exclusively on public datasets, demonstrating remarkable performance. Alternatively, OPT [ 23 ] presents a series of open-source transformer-based language models spanning 125M to 175B parameters, designed to match the capabilities of GPT-3 [ 19 ] . Recent studies

Read the complete text