Hallucinated Span Detection with Multi-View Attention Features

Yuya Ogasa, Yuki Arase · 2025

Read Hallucinated Span Detection with Multi-View Attention Features by Yuya Ogasa, Yuki Arase (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04335.txt.

Introduction

Yuya Ogasa and Yuki Arase spot the exact spans of text a language model made up. Their paper, from twenty twenty-five, tackles hallucination at the span level. Most past work flags a whole answer as false or true. Pinpointing the false stretch is more useful, and attention maps hold clues.

When a model invents, its attention often looks off in telling ways. The team builds features that view attention from three angles. They check if tokens are heeded or ignored, if focus is skewed, and how wide the context is. A transformer judge then labels each token in turn. On long tasks like summaries, their method beats strong rivals.

Catalog record

CREATORYuya Ogasa, Yuki Arase
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-04335

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hallucinated Span Detection with Multi-View Attention Features Yuya Ogasa, Yuki Arase Abstract This study addresses the problem of hallucinated span detection in the outputs of large language models. It has received less attention than output-level hallucination detection despite its practical importance. Prior work has shown that attentions often exhibit irregular patterns when hallucinations occur. Motivated by these findings, we extract features from the attention matrix that provide complementary views capturing (a) whether certain tokens are influential or ignored, (b) whether attention is biased toward specific subsets, and (c) whether a token is generated referring to a narrow or broad context, in the generation. These features are input to a Transformer-based classifier to conduct sequential labelling to identify hallucinated spans. Experimental results indicate that the proposed method outperforms strong baselines on hallucinated span detection with longer input contexts, such as data-to-text and summarisation tasks. 1 Introduction Large Language Models (LLMs) have significantly advanced natural language processing and demonstrated high performance across tasks ( Minaee et al., 2024 ) . However, hallucinations persisting in texts generated by LLMs have been identified as a serious issue, which undermines LLM safety ( Ji et al., 2024b ) . To tackle this challenge, hallucination detection has been actively studied ( Huang et al., 2025 ) . Model-level (e.g., ( Min et al., 2023 ) ) or response-level (e.g., ( Manakul et al., 2023 ) ) hallucination detection has been proposed. However, identification of the hallucinated span is less explored despite its practical importance. Hallucinated span detection enables understanding and manually editing the problematic portion of the output. It also provides clues to mitigate hallucinations in LLM development. To address this, we tackle hallucinated span detection. While there have been various types of hallucinations ( Wang et al., 2024 ) , this study targets hallucinations on contextualised generations that add baseless and contradictive information against the given input context. Motivated by the findings that irregular attention patterns are observed when hallucination occurs ( Chuang et al., 2024 ; Zaranis et al., 2024 ) , we extract features to characterise the distributions of attention weights. Specifically, the proposed method extracts an attention matrix from an LLM by inputting a set of prompt, context, and LLM output of concern. It then assembles features for each token from the attention matrix: average and diversity of incoming attention as well as diversity of outgoing attention, which complementarily capture the attention patterns of language models. The former two features indicate whether attention is distributed in a balanced manner for tokens in the output text. The last feature reveals if an output token was generated by broadly attending to other tokens. These features are then fed to a Transformer encoder with a conditional random field layer on top to conduct sequential labelling to determine whether a token is hallucinated or not. Experimental results on hallucinated span detection confirmed that the proposed method outperforms strong baselines on data-to-text and summarisation tasks, improving token-level F 1 score for 4.9 and 2.9 points, respectively. An in-depth analysis reveals that the proposed method is capable of handling longer input contexts. Our code is available at https://github.com/Ogamon958/mva_hal_det . Figure 1: Overview of the proposed method 2 Related Work This section discusses hallucination detection that utilises various internal states of LLMs. Attention-Based Hallucination Detection Lookback Lens ( Chuang et al., 2024 ) is the most relevant method to our study, which identifies hallucinations using only attention matrices. It computes the “Lookback” ratio of attention to assess whether generated tokens attend well to the input context. In contrast, our features primarily focus on the attention of output texts and capture more nuanced and structural attention patterns. ALTI+ ( Ferrando et al., 2022 ; Zaranis et al., 2024 ) tracks token interactions across layers. ALTI+ has been applied to hallucination detection in machine translation, highlighting cases where the model fails to properly utilise source text information. A drawback of ALTI+ is its computational cost. It computes a token-to-token contribution matrix for each layer and for each attention head. Therefore, memory consumption linearly increases depending on the length of context and output as well as LLM sizes. Indeed, Zaranis et al. (2024) excluded sequences longer than 400 tokens due to GPU memory constraints. Other Internal States for Hallucination Detection Hallucination detection has also explored various internal states of LLMs other than attention. Xiao and Wang (2021) and Zhang et al. (2023) identify hallucinations as tokens generated with anomalously low confidence based on the probability distribution in the final layer. Azaria and Mitchell (2023) and Ji et al. (2024a) use layer-wise Transformer block outputs to estimate hallucination risk. These studies assume that hallucination detection will be conducted on the same LLM generating output and can access such Transformer block outputs. In contrast, we empirically showed that the proposed method can also be applied to closed LLMs. Further, attention-based methods are distinctive from these studies in that they aim to model inter-token interactions. Figure 2: Feature extraction from attention matrix (these attention values are for illustrative purposes.) 3 Proposed Method The proposed method is illustrated in Figure 1 . It conducts sequential labelling, i.e., predicts binary labels that indicate whether a token in text, which has been generated by a certain LLM, is hallucinated or not. Specifically, the proposed method takes a set of prompt, input context, and output generated by an LLM of concern as input to another LLM and obtains the attention matrix of the output text span. It then extracts features from the attention matrix (Sections 3.1 and 3.2 ). These features are fed to a Transformer encoder model with the prediction head of a conditional random field (CRF) to conduct sequential labelling to identify hallucinated spans (Section 3.3 ). As the attention matrix provides crucial information for our method, we compare the raw attention and a variation based on the analysis of attention mechanism ( Kobayashi et al., 2020 ) (Section 3.4 ). We remark that only the hallucination detection model needs training, i.e., the LLM for attention matrix extraction is kept frozen, which makes our method computationally efficient. Our method applies to both scenarios where the LLM that generated outputs and the LLM for hallucinated span detection are the same or different. In practice, the latter setting is expected to be more common in an era where LLMs are widely used for writing tasks. In addition, we cannot access the internal state of proprietary LLMs. Our experiments assume the scenario where the LLM for generation and the LLM for detection are different. 3.1 Feature Design Previous studies revealed that irregular patterns of attention are incurred when hallucination occurs ( Chuang et al., 2024 ; Zaranis et al., 2024 ) . Based on these findings, we design features to complementarily capture irregular attentions. Specifically, we extract features providing complementary views of the attention matrix as shown in Figure 2 : (a) average attention a token receives ( Average Incoming Attention ), (b) diversity of attention a token receives ( Incoming Attention Entropy ), and (c) diversity of tokens that a token attends to ( Outgoing Attention Entropy ). Average Incoming Attention We compute the average attention weights that a token receives when generating others. This feature indicates whether certain

Read the complete text