A Single Direction of Truth: An Observer Model’s Linear Residual Probe Exposes and Steers Contextual Hallucinations
Charles O’Neill, Slava Chalnev, Chi Chi Zhao, Max Kirkby, Mudith Jayasekara · 2025
Read A Single Direction of Truth: An Observer Model’s Linear Residual Probe Exposes and Steers Contextual Hallucinations by Charles O’Neill, Slava Chalnev, Chi Chi Zhao, Max Kirkby, Mudith Jayasekara (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23221.txt.
Introduction
Charles O'Neill and four co-authors find a single internal direction that flags made-up statements. Their paper appeared in two thousand twenty-five. Contextual hallucinations claim things the given context does not support. Their trick uses a second model as an observer. One forward pass plus a linear probe does the detection. The probe reads the observer's residual stream.
A single linear direction separates false from faithful text. It transfers across generators and beats baselines clearly. The signal peaks in middle layers and holds across model sizes. Tracing it points to sparse late-layer activity. Pushing along the direction changes how often generators hallucinate. Detection thus comes with a steering knob attached. Interpretability gains a concrete, usable signal.
Catalog record
| CREATOR | Charles O’Neill, Slava Chalnev, Chi Chi Zhao, Max Kirkby, Mudith Jayasekara |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-23221
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Single Direction of Truth: An Observer Model’s Linear Residual Probe Exposes and Steers Contextual Hallucinations Charles O’Neill, Slava Chalnev, Chi Chi Zhao, Max Kirkby, Mudith Jayasekara Abstract Contextual hallucinations –- statements unsupported by given context –- remain a significant challenge in AI. We demonstrate a practical interpretability insight: a generator-agnostic observer model detects hallucinations via a single forward pass and a linear probe on its residual stream. This probe isolates a single, transferable linear direction separating hallucinated from faithful text, outperforming baselines by 5–27 points and showing robust mid‑layer performance across Gemma‑2 models (2B → \to 27B). Gradient‑times‑activation localises this signal to sparse, late‑layer MLP activity. Critically, manipulating this direction causally steers generator hallucination rates, proving its actionability. Our results offer novel evidence of internal, low‑dimensional hallucination tracking linked to specific MLP sub‑circuits, exploitable for detection and mitigation. We release the 2000‑example ContraTales benchmark for realistic assessment of such solutions. 1 Introduction Large language models (LLMs) have made striking progress in open‑ended generation, yet they continue to produce statements that are plausible but unsupported by their given context – a failure mode broadly labelled hallucination ( Huang et al. 2024 ; Ji et al. 2023 ) . When these systems are deployed in medicine, finance, or law, even an isolated hallucination can trigger costly or harmful downstream actions, eroding user trust and slowing adoption. Detecting hallucinations after text has been produced is therefore a key practical requirement, both for automated self‑monitoring and for post‑hoc auditing workflows ( Valentin et al. 2024 ) . Unfortunately, most high‑performing detectors to date either (i) require privileged access to the generator model ’s parameters and logits – an unrealistic assumption for commercial APIs and safety‑critical settings – or (ii) fall back on brittle surface cues such as lexical overlap, unseen named entities, or low embedding similarity between source and output. These heuristics falter whenever hallucinations manifest as subtle logical contradictions rather than obvious factual novelties, a gap that is starkly revealed by synthetic logic benchmarks such as our ContraTales . This highlights the need for robust, interpretability-based approaches that offer deeper insights beyond surface cues for more reliable detection Recent work in mechanistic interpretability argues that transformers internally encode rich world knowledge along approximately linear directions in activation space ( Burns et al. 2023 ; Park et al. 2023 ) . Linear or sparse probes applied to hidden states have uncovered directions for factual truth, policy compliance, and other high‑level properties ( Alain & Bengio 2016 ; Marks et al. 2024 ) . Yet, existing studies either rely on signals inside the generator or do not test whether the discovered directions are (i) consistent across model sizes and domains, (ii) causal —that is, capable of steering generation when manipulated—and (iii) sufficiently salient to be decoded by small observer models, limiting their direct actionability in practical, generator-agnostic scenarios. We ask: does a transformer notice when a span of text contradicts its earlier context, and if so, is that realisation encoded along a single, linearly accessible axis that can be read out by another model? We cast this as a mechanistic hypothesis and test it directly. Concretely, we introduce the observer paradigm : a separate , frozen language model ingests an arbitrary document consisting of a source passage followed by a candidate continuation and, in a single forward pass, predicts whether the continuation is contextually supported. Our detector uses nothing more than a linear probe on the observer’s residual‑stream activations at the full stop of the final sentence. Across standard news‑domain summarisation sets ( CNN/DailyMail , XSum ) and an out‑of‑domain corpora (our introduced dataset ContraTales ) our linear probe approach achieves F 1 scores up to 0.99 on news and 0.84 on ContraTales , significantly beating lexical‑overlap, entity‑verification, semantic‑similarity, and attention‑pattern baselines by 5–27 points. Layer sweeps reveal a broad mid‑layer performance plateau that is shared across model sizes (Gemma-2 2B \to 27B) and datasets. Attribution analysis based on gradient‑times‑activation localises the signal for contextual inconsistency to a sparse pattern of late‑layer MLP activity, whose characteristics are notably stable across diverse datasets. Most importantly, causal interventions that ablate or inject the learned direction in a generator modulate hallucination rates, demonstrating that the axis captures a functionally meaningful representation. Contributions. 1. We provide the first generator‑agnostic detector that identifies contextual hallucinations with a single forward pass , requiring no knowledge of the text’s origin and no sampling overhead, demonstrating a practical application of interpretability for efficient AI monitoring. 2. We show that a single, highly linear residual‑stream direction robustly separates hallucinated from supported spans across domains and remains readable by comparatively small observer models, offering an actionable insight into model representations. 3. Through gradient‑times‑activation attribution, we map the representation of contextual inconsistency to a compact and consistent pattern of late‑layer MLP activity, and demonstrate its sparsity and stability across key datasets. 4. We establish causality : manipulating activity along this axis in a generator steerably reduces or amplifies hallucination prevalence. 5. To contribute to developing realistic benchmarking methods for actionable interpretability, we release ContraTales , a 2000‑example benchmark of purely logical contradictions tailored to stress‑test contextual hallucination detectors. 2 Background and Related Work Large language models (LLMs) frequently generate plausible yet factually incorrect content that contradicts their input or world knowledge, a phenomenon known as hallucination ( Huang et al. 2024 ; Ji et al. 2023 ) . Despite remarkable capabilities in text generation and comprehension, this tendency to hallucinate significantly undermines model reliability in high-stakes domains ( Ji et al. 2023 ) . We focus on intrinsic hallucinations , which directly contradict provided source content, as opposed to extrinsic hallucinations , which introduce unverifiable external information ( Ji et al. 2023 ; Huang et al. 2024 ) . 1 1 1 Readers new to mechanistic interpretability (MI) will find a concise overview in Rai et al. 2024 . 2.1 Hallucination Detection Methodologies Existing detection methods can be categorised by their access requirements to the generating model and by underlying techniques. Generator-internal approaches require access to the generator’s internals, encompassing token-probability analysis, calibrated confidence estimation, and internal-state probing ( Azaria & Mitchell 2023 ) . Recent MI-based work extends this line, with Yu et al. 2024 identifying drifting sub-modules for non-factual outputs, and Sun et al. 2024 tracing retrieval versus parametric knowledge to flag hallucinations in RAG systems. However, such methods are inapplicable to black-box APIs ( Valentin et al. 2024 ) . Post-hoc external methods operate on generated text alone, and include checking claims against knowledge bases, using entailment models, or enlisting an LLM judge ( Huang et al. 2024 ; Valentin et al. 2024 ) . Sampling-based consistency, such as SelfCheckGPT ( Manakul et al. 2023 ) , generates and compares multiple outputs for agreement but is computationally intensive. Embedding/representation-based methods exploit vector representations or internal states without needing generator weight access. Examples include semantic similarity checks of source and output embeddings ( Ji et al. 2023 ) , internal-state distribution analysis for hidden-state drift ( Farquhar et al. 2024 ) , and efficient linear classifiers like Semantic Entropy Probes (SEPs) ( Kossen et al. 2024 ) . Hallucination-resistant finetuning along “hallucination directions” has also been demonstrated ( Research 2025 ) . The