H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

Cheng Gao, Huimin Chen, Chaojun Xiao, Zhiyi Chen, Zhiyuan Liu, Maosong Sun · 2025

Read H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs by Cheng Gao, Huimin Chen, Chaojun Xiao, Zhiyi Chen, Zhiyuan Liu, Maosong Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01797.txt.

Introduction

Cheng Gao and co-authors hunt for neurons tied to hallucinations in language models. Models often state false facts with full confidence. Past work blamed data or training goals at a broad level. Neuron-level causes stayed largely unexplored until now.

The team finds a set of neurons linked to hallucination acts. They study the impact and origin of those neurons. If a small set drives many false claims, targeted fixes become feasible. Neuron-level insight could lead to monitors that flag risky outputs as they form. Hallucination neurons may cluster by fact type, which would let monitors watch the riskiest sets during high-stakes use like medicine.

Catalog record

CREATORCheng Gao, Huimin Chen, Chaojun Xiao, Zhiyi Chen, Zhiyuan Liu, Maosong Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01797

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs Cheng Gao, Huimin Chen, Chaojun Xiao, Zhiyi Chen, Zhiyuan Liu, Maosong Sun Abstract Large language models (LLMs) frequently generate hallucinations – plausible but factually incorrect outputs – undermining their reliability. While prior work has examined hallucinations from macroscopic perspectives such as training data and objectives, the underlying neuron-level mechanisms remain largely unexplored. In this paper, we conduct a systematic investigation into hallucination-associated neurons (H-Neurons) in LLMs from three perspectives: identification, behavioral impact, and origins. Regarding their identification, we demonstrate that a remarkably sparse subset of neurons (less than 0.1 % 0.1\% of total neurons) can reliably predict hallucination occurrences, with strong generalization across diverse scenarios. In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors. Concerning their origins, we trace these neurons back to the pre-trained base models and find that these neurons remain predictive for hallucination detection, indicating they emerge during pre-training. Our findings bridge macroscopic behavioral patterns with microscopic neural mechanisms, offering insights for developing more reliable LLMs. 1 Introduction In recent years, large language models (LLMs) have achieved groundbreaking advancements in natural language processing tasks, demonstrating impressive potential towards artificial general intelligence ( 3 ; 4 ; 37 ; 35 ) . However, these advancements come with a persistent reliability challenge that troubles researchers and users alike: hallucinations. Hallucinations occur when models produce outputs that seem plausible but are factually inaccurate or unsupported by evidence ( 32 ; 16 ) . For example, GPT-3.5 has been shown to hallucinate in approximately 40% of citation-based factuality evaluations, a figure that improves but remains high at 28.6% for GPT-4 ( 5 ) . Similarly, emerging reasoning-centric systems such as DeepSeek-R1, despite demonstrating strong performance on complex tasks, continue to exhibit pronounced hallucination modes ( 2 ) . Collectively, these observations indicate that hallucinations persist regardless of model architecture, highlighting a critical bottleneck in the reliability of state-of-the-art LLMs. To improve LLM reliability, researchers have invested considerable effort in uncovering the mechanisms and factors behind hallucinations, which can be broadly grouped into three categories. First, from a training data perspective, distribution imbalances and inherent biases within datasets make it difficult for models to accurately recall long-tail facts ( 40 ; 25 ) . Second, training objectives in both pretraining and post-training phases primarily incentivize confident predictions without promoting the expression of uncertainty for unfamiliar information, encouraging models to output incorrect guesses ( 18 ) . Specifically, the next-token prediction goal in pretraining prioritizes fluent continuations over factual accuracy, while instruction tuning or reinforcement learning often favors generating superficially helpful responses, sometimes at the expense of honest refusals to answer. Third, decoding algorithms introduce instability through randomness and error accumulation in autoregressive generation, allowing small deviations to snowball into hallucinations ( 45 ; 24 ; 20 ) . Current studies largely treat LLMs as black boxes, examining hallucination causes at a macroscopic level while neglecting microscopic insights into neuron-level mechanisms. Yet, such fine-grained analysis holds immense promise for explaining how hallucinations arise and for developing mitigation strategies. Just as biological research on cellular division informs treatments for diseases such as cancer ( 8 ; 31 ) , and neuroscience investigations into individual neuronal activity and synaptic interactions shape theories of cognition like learning ( 30 ) and memory ( 34 ; 29 ) , analyzing neurons – the fundamental computational units of LLMs – is essential for decoding hallucination. By scrutinizing neurons’ activation patterns in relation to hallucinations, we can gain deeper insights into model reliability. In terms of interpretability, neuron-level analysis can enable the prediction of when hallucinations are prone to emerge; for alignment and behavioral control, it provides actionable intervention points, such as activating or suppressing specific subsets of neurons to reliably modify model outputs. In this paper, we adopt a neuron-centric perspective to investigate the microscopic mechanisms of hallucinations in LLMs. Prior research has shown that internal hidden states can serve as effective features for detecting hallucinations ( 15 ) , and others using sparse autoencoders have provided case studies connecting hallucinations to specific neuron activations ( 27 ; 10 ) , hinting at a deeper link between neuronal behavior and hallucination generation. Building on this foundation, we identify a set of hallucination-associated neurons and term them as H-Neurons . We then systematically explore the existence, behavioral impacts, and origins of H-Neurons. We address the following three research questions: • Q1: Do H-Neurons exist? Can we identify specific neurons whose activations reliably distinguish between hallucinatory and faithful outputs? • Q2: How do these neurons influence model behavior? Specifically, what types of tasks exhibit a significant change on performance when these neurons’ activations are altered, thereby establishing a link between hallucination and another capability? • Q3: When do these neurons originate? Are they introduced during the post-training alignment phase or already present in the pre-trained phase? Specifically, drawing from setups in previous work ( 6 ; 43 ; 9 ) , we focus on neurons in the feedforward networks and examine hallucinations in knowledge-based question answering and make the following observations. Existence of H-Neurons Our investigation reveals that a remarkably sparse subset of neurons – comprising less than 0.1\% of the model’s total neurons – can accurately predict whether the model will produce hallucinated responses. We refer to these predictive neurons as H-Neurons . To identify these neurons, we develop a systematic methodology that contrasts activation patterns between faithful and hallucinated responses, then apply sparse logistic regression to uncover the most predictive neurons. Notably, the neurons identified through simple QA tasks demonstrate strong generalization capability: they maintain robust predictive accuracy across out-of-distribution scenarios, ranging from specialized cross-domain contexts to pure fabrications concerning non-existent entities, achieving reliable hallucination detection. Impact on Model Behavior Our analysis uncovers that H-Neurons are linked to over-compliance behaviors in LLMs. To establish this causal relationship, we conduct controlled interventions by systematically scaling the activation magnitudes of these neurons. The interventions reveal a distinctive behavioral pattern: amplifying H-Neurons’ activations systematically increases a spectrum of over-compliance behaviors – ranging from overcommitment to incorrect premises and heightened susceptibility to misleading contexts, to increased adherence to harmful instructions and stronger sycophantic tendencies. These findings suggest that H-Neurons do not simply encode factual errors, but rather represent a general tendency to prioritize conversational compliance over factual integrity. Origin of H-Neurons Our investigation reveals that H-Neurons originate during the pre-training phase, providing empirical evidence for the insights proposed by OpenAI researchers from the perspective of learning theory ( 18 ) . To trace their developmental timeline, we conduct cross-model transfer experiments: we apply the hallucination neurons identified in instruction-tuned models to their corresponding base models and evaluate their predictive efficacy. The results demonstrate that these neurons retain their predictive ability in base models, successfully detecting hallucinations even prior to fine-tuning. In summary, this paper provides a systematic neuron-level investigation into the microscopic mechanisms of hallucinations in LLMs. By bridging the gap between macroscopic behavioral patterns and fine-grained neural activations, we hope our work can deepen the understanding of how hallucinations arise at the computational level, and offer actionable insights for developing more reliable LLMs. 2 Identification of H-Neurons While prior work has

Read the complete text