Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction

Dan WilsonUniversity of Tennessee [email protected], Mohamed Akrout University of Tennessee [email protected] · 2026

Read Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction by Dan WilsonUniversity of Tennessee [email protected], Mohamed Akrout University of Tennessee [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05134.txt.

Introduction

Dan Wilson and Mohamed Akrout detect language model hallucinations cheaply by treating the model as a black-box dynamical system whose outputs trace its hidden motion. Existing detectors often need expensive repeated sampling or outside knowledge lookups. Their method instead embeds each response and watches the resulting vector sequence evolve over the answer.

Using Koopman operator theory, they fit transition rules for factual and hallucinated regimes separately. A score based on the difference in prediction errors then flags which regime a new response follows. Testing across three benchmarks gave state-of-the-art detection with lower resource use, and the whole check needs only a single sample pass over the model's answer.

Catalog record

CREATORDan WilsonUniversity of Tennessee [email protected], Mohamed Akrout University of Tennessee [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05134

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction Dan WilsonUniversity of Tennessee [email protected], Mohamed Akrout University of Tennessee [email protected] Abstract Large Language Models (LLMs) frequently generate plausible but non-factual content, a phenomenon known as hallucination. While existing detection methods typically rely on computationally expensive sampling-based consistency checks or external knowledge retrieval, we propose a new method that treats the LLM as a black-box dynamical system. By projecting LLM responses into a high-dimensional manifold via an embedding model, we characterize the resulting vector sequences as observable realizations of the model’s latent state-space dynamics. Leveraging Koopman operator theory, we fit the transition operators for both factual and hallucinated regimes and define a differential residual score based on their respective prediction errors. To accommodate varying user requirements and domain-specific sensitivities, we introduce a preference-aware calibration mechanism that optimizes the classification threshold based on a small set of demonstrations. This approach enables low-cost hallucination detection in a single-sample pass, avoiding the need for secondary sampling or external grounding. Extensive testing across three data benchmarks demonstrates that our method achieves state-of-the-art performance with reduced resource overhead. Figure 1: Our proposed differential residual score \Delta\mathcal{E} accurately captures the transition between factual and hallucinated sentences in segmented LLM responses from the WikiBio dataset. The score is computed by comparing the relative accuracy between predictions of the dynamics of token embeddings from Llama-3 with dynamical system models inferred from both hallucinated and non-hallucinated responses. 1 Introduction Large Language Models (LLMs) have achieved remarkable success across diverse natural language processing tasks ( 9 ; 1 ) . However, their tendency to generate fluent yet factually incorrect content, commonly known as hallucination, remains a critical barrier to reliable deployment in high-stakes domains such as healthcare, law, finance, and autonomous agents ( 19 ; 5 ; 42 ) . As LLMs become increasingly integrated into real-world applications, the ability to detect and mitigate hallucinations has emerged as a pressing research priority. Hallucination in LLMs is not merely a design flaw but a statistical necessity 19 ; 45 . Any calibrated language model must inevitably produce false or unsubstantiated claims, establishing hallucination as theoretically unavoidable 22 . This phenomenon emerges from the fundamental trade-off between finite training data and open-ended generation 8 . Empirical evidence shows that even state-of-the-art models cannot reliably distinguish belief from knowledge 21 ; 41 ; 4 ; 29 . Ultimately, the persistent reliance on external verification 5 ; 27 highlights a critical gap in LLM autonomy. To achieve scalable trust, it is critical to develop self-contained detection frameworks that identify hallucinations with minimal to no reliance on external knowledge retrieval. Hallucination detection methods are generally categorized by their level of model access. White-box approaches leverage internal representations like hidden states or attention maps ( 3 ; 40 ; 11 ) , while gray-box methods utilize token-level output probabilities 33 ; 6 . However, both remain impractical for many commercial LLMs accessed via restricted application programming interfaces (APIs). Black-box methods circumvent these access barriers by operating solely on generated text, typically through knowledge retrieval or sampling-based consistency checks, such as SelfCheckGPT ( 30 ) . Existing black-box approaches are suffer from significant computational overhead: retrieval-based methods depend on external knowledge latency, and sampling-based methods require multiple forward passes. This creates a dual barrier of high API costs and latency for real-time applications. To address these limitations, we treat the LMM as dynamical system (DS) and propose a novel DS prediction approach that considers the temporal evolution of generated token embeddings. Our method identifies hallucinations by classifying the evolution of the output token embeddings by comparing the relative accuracy of predictive models trained from both hallucinated and non-hallucinated responses allowing for the definition of a differential prediction score \Delta\mathcal{E} . Fig. 1 provides an example of this strategy showing the evolution of this score over the token embedding of an LLM response. Here \Delta\mathcal{E} rises and falls with the factuality of LLM’s output sentences in the embedding space. We build upon the differential prediction score to provide a hallucination detection technique that uses one-single pass detection without the need for output token probabilities or access to external information. To our knowledge, this is the first work to consider the output of an LLM as a black-box dynamical system with hallucinations and factual responses stemming from trajectories evolving on different manifolds embedded in the state space. Our primary contributions are as follows: 1) We devise a new method for hallucination detection that treats LLM generation as a dynamical system with a high-dimensional observable space. Results suggest that the temporal evolution of the observables stemming from factual and hallucinated responses evolve on distinct manifolds that can be distinguished from one another. 2) We introduce a computationally efficient hallucination detection algorithm using a differential error score between the prediction for trajectories evolving on the accurate and inaccurate manifolds. Unlike traditional black-box metrics that necessitate multiple samples or external grounding, our method identifies hallucinations using a one-sample pass. This is achieved by fitting linear dynamical systems on a small set of LLM responses, bypassing the need for computationally heavy retrieval modules. 3) We establish the state-of-the-art performance of our DS method through an extensive evaluation across three data benchmarks: FELM 49 , HaluEval 26 , and WikiBio 30 . We benchmark our approach against established methods that generally require significantly greater resources, including multi-sample consistency checks and/or external knowledge retrieval. Despite the absence of comparable single-pass retrieval-free black-box baselines in current literature, our DS approach consistently outperforms or matches these resource-intensive benchmarks. 4) We introduce a calibration mechanism that tunes the classification threshold via a set of user-annotated samples according to the user’s preferred strictness (e.g., tolerant or strict of minor hallucinations). This personalization allows the detector to adapt to individual preferences. 2 Related Work Hallucination detection methods differ in the degree of model access they require, and this difference their applicability across different deployment settings (see 18 for a comprehensive review). White-box and gray-box methods. White-box methods require full access to the internal representations of the model, including hidden-state activations, embedding layers, attention maps, and gradients 3 ; 11 ; 40 ; 17 . By directly inspecting these internal signals, white-box approaches can capture fine-grained indicators of hallucination that are invisible at the output level 39 . Despite their strong performance, white-box methods are typically feasible only when the model is hosted locally, as they require unrestricted access to the model architecture and its intermediate computations. Gray-box methods, however, relax the access requirements of white-box approaches by leveraging the output probability distribution over tokens, including next-token probabilities 35 and logit entropy 13 ; 24 which are commonly exposed by commercial LLM APIs without granting access to internal model states 33 ; 6 ; 7 ; 34 . Black-box methods. Black-box methods impose the fewest access requirements because they operate exclusively on the generated output tokens with no visibility into internal states or probability distributions. These approaches typically rely on sampling multiple responses and evaluating their consistency through lexical overlap, entailment-based comparison, or knowledge-graph representations of factual claims 30 ; 37 ; 47 ; 23 ; 14 . For example, Manakul et al. propose SelfCheckGPT , which detects hallucinations by measuring

Read the complete text