MultiHaluDet : Multilingual Hallucination Detection via LLM Hidden State Probing
Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi · 2026
Read MultiHaluDet : Multilingual Hallucination Detection via LLM Hidden State Probing by Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24919.txt.
Introduction
Riasad Alvi leads MultiHaluDet, a detector for hallucinations in large language models working across languages without language-specific tuning. Output-confidence tricks and single-layer probes often miss deep factual slips. The problem bites hardest outside English. The method probes the full hidden-state path of a frozen model, pulling cross-layer features through multi-scale and self-attention pooling. It feeds out-of-fold embeddings to a tuned ensemble of classifiers. It tops prior English hallucination results and carries its skill to French, Bangla, and Amharic.
Catalog record
| CREATOR | Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37867 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24919
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MultiHaluDet : Multilingual Hallucination Detection via LLM Hidden State Probing Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi Abstract Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment, a vulnerability heavily exacerbated in non-English and resource-constrained contexts. Existing detection approaches that rely on output confidence heuristics or single-layer internal representations frequently fail to capture deep, complex factual inconsistencies across diverse languages. To address this, we introduce MultiHaluDet , a novel four-stage framework that detects multilingual hallucinations by probing the full hidden state trajectories of frozen LLMs without requiring language-specific fine-tuning. Our method extracts sequential features across multiple layers and processes them via a hybrid architecture using multi-scale attention and self-attention pooling. By generating out-of-fold embeddings that feed into a learned ensemble meta-learner, MultiHaluDet captures both fine-grained and coarse-grained patterns of factual inconsistency. Extensive experiments demonstrate that our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks using Mistral-7B and LLaMA2-7B architectures. Crucially, we rigorously evaluate our framework’s cross-lingual generalization across high (French), medium (Bangla), and low-resource (Amharic) languages. MultiHaluDet demonstrates exceptional representational robustness, consistently outperforming baselines and successfully transferring hallucination detection capabilities across typologically diverse linguistic tiers. 1 Introduction Hallucinations in LLMs have emerged as a critical barrier to their reliable deployment across high-stakes domains ( 6 ; 17 ; 23 ) . These generations pose significant risks ( 6 ) , motivating diverse detection techniques. Evidence-based methods ( 4 ; 27 ) retrieve external information to verify factual consistency, but are computationally intensive. Evidence-free methods leverage inherent model characteristics: logit-based and consistency-based methods estimate uncertainty and output stability, while classification-based methods ( 18 ; 2 ) probe internal hidden states without external retrieval. While internal state probing shows promise ( 18 ; 2 ) , current approaches remain inadequate, particularly across diverse languages. Recent work reveals truthfulness information is concentrated in specific tokens ( 18 ) ; however, single-position methods struggle when non-factual tokens are distributed across sequences. Furthermore, attention-based approaches ( 5 ; 2 ) using simple ratios achieve limited discrimination. Probabilistic frameworks ( 7 ) require complex reasoning pipelines, while active validation ( 23 ) and tool-augmented systems ( 4 ; 27 ) introduce substantial latency. As noted in 6 , hallucinations manifest as semantic-level confabulations rather than token-level uncertainty. Existing methods largely fail to address these confabulations across non-English and resource-constrained languages. Transformer architectures process information through deep layers, embedding language-agnostic hallucination signals within hidden state trajectories. Deep sequence modeling with multi-scale feature aggregation offers a promising solution. We introduce MultiHaluDet , a supervised framework leveraging multi-scale attention and transformer encoders to model hidden state dynamics across the full depth of frozen LLMs. Unlike methods focused on individual tokens or static layers, our approach aggregates information across multiple scales through self-attention pooling. We evaluate across multiple architectures and language tiers, demonstrating strong cross-lingual performance. Our contributions are summarized as follows: • We introduce MultiHaluDet , a four-stage framework comprising dynamic feature extraction, multi-scale attention encoding, out-of-fold deep feature generation, and learned ensemble meta-learning, which jointly capture fine-grained and coarse-grained patterns from hidden-state trajectories. • We evaluate our framework on the HaluEval and TriviaQA benchmarks using Llama-2-7B and Mistral-7B-Instruct, demonstrating substantial performance gains and representational robustness over existing baselines. • We comprehensively evaluate the cross-lingual generalization of our framework by translating standard benchmarks into French (high-resource), Bangla (medium-resource), and Amharic (low-resource). Our results demonstrate that internal state probing can maintain strong detection signals across typologically diverse linguistic tiers under controlled translation-based evaluation, without language-specific fine-tuning. 2 Related Work Hallucination detection approaches fall into three categories: (i) evidence-based methods verifying outputs against external knowledge, (ii) self-detection methods leveraging internal model states, and (iii) consistency-based approaches assessing output stability. Evidence-based methods retrieve external information for verification. 11 converts LLM responses into knowledge graphs for atomic fact verification. 22 introduced multi-source retrieval with contradiction graph analysis. 26 employed HHEM for lightweight consistency assessment. While effective, these methods depend on retrieval quality, making them vulnerable to gaps and latency. Self-detection methods probe LLM internal representations without external resources. 21 introduced MIND, training classifiers on auto-generated pseudo-labels. 16 employed Bayesian optimization to identify optimal layer insertion points. 28 proposed MHAD, selecting specific neurons via linear probing. 13 approximated semantic uncertainty from hidden states. 3 proposed INSIDE, measuring consistency through eigenvalues of response covariances. 12 analyzed layer-wise usable information across transformer depths. 19 demonstrated strong detection using only four token probability features. Consistency-based methods assess output stability. 24 proposed MetaQA, leveraging metamorphic relations for semantic consistency verification. Despite these advances, challenges remain. Evidence-based methods suffer from retrieval dependency. Self-detection methods relying on single-position representations struggle when non-factual tokens appear at sequence beginnings. Consistency-based methods incur costs from multiple generation passes. 20 demonstrated that LLMs can hallucinate with high certainty despite possessing correct knowledge. 25 revealed the need to distinguish between intelligent and defective hallucinations. Our work aligns with internal state probing methods but addresses their key limitations. While 16 and 28 focus on specific layers or tokens, we employ dynamic layer sampling and multi-scale attention to aggregate information across the full depth trajectory. Unlike 13 and 3 , which probe specific positions, our architecture processes sequential layer features with transformer encoders and self-attention pooling, enabling adaptive depth selection. Our out-of-fold stacking with ensemble meta-learning provides more robust generalization than single-classifier approaches ( 21 ; 16 ) . Figure 1: Overview of the four-stage MultiHaluDet framework for multilingual hallucination detection. 3 Methodology We present MultiHaluDet (Figure 1 ), a four-stage framework for hallucination detection. 3.1 LLM-Based Feature Extraction 3.1.1 Prompt Construction and Forward Pass Let \mathcal{D}=\{(q_{i},a_{i},y_{i})\}_{i=1}^{N} denote a dataset of question–answer pairs, where q_{i} is a natural-language question, a_{i} is a candidate answer, and y_{i}\in\{0,1\} is a binary label indicating whether a_{i} is a hallucination ( y_{i}=1 ) or a faithful response ( y_{i}=0 ). Each sample (q_{i},a_{i}) is formatted as a structured natural-language prompt and tokenized with a fixed maximum sequence length. The prompt is passed through a frozen, quantized LLM in a single forward pass, yielding a sequence of hidden state tensors \{\mathbf{H}^{(l)}\}_{l=0}^{L} , where L is the number of transformer layers and \mathbf{H}^{(l)}\in\mathbb{R}^{T\times d} collects the d -dimensional representations of T tokens at layer l . The next-token logit vector \mathbf{z}\in\mathbb{R}^{V} at the final position is retained for global statistics. No gradient is computed during feature extraction; the LLM parameters remain fully frozen throughout all experiments. 3.1.2 Dynamic Layer Sampling To ensure architectural compatibility across LLMs of varying depth, we introduce a dynamic layer sampling strategy that maps any model’s L transformer layers to a fixed target count K , producing a uniform sequential representation regardless of model size. Three regimes are handled without model-specific configuration: • Exact match ( L=K ): identity mapping \mathcal{I}_{K}=\{1,\ldots,K\} . • Shallow model ( L<K ): all L layers are used and the deepest layer is repeated to pad the sequence to K . • Deep model ( L>K ): K indices are selected by uniform interpolation over the full depth: \mathcal{I}_{K}=\left\{\operatorname{clip}\!\left(\left\lfloor 1+(L-1)\cdot\frac{k}{K-1}\right\rceil,1,L\right)\right\}_{k=0}^{K-1} (1) where \lfloor\cdot\rceil denotes rounding to the nearest integer and \operatorname{clip}(\cdot,1,L) clamps indices to valid range. This formulation is differentiable with respect to K and requires no architecture-specific