Cross-Attention is Half Explanation in Speech-to-Text Models
Sara Papi, Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli · 2025
Read Cross-Attention is Half Explanation in Speech-to-Text Models by Sara Papi, Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18010.txt.
Introduction
Sara Papi and 4 co-authors ask how much of a speech model's focus scores truly explain its output. Their 2025 paper deals with speech-to-text models, where a cross-attention step links heard speech to written words. Folk have reused those scores for time stamps and word alignment, as if they showed true links.
The team checks those scores against maps of which input parts drive the output. Across tongues, tasks, and model sizes, the scores line up fair to strong with the maps. Yet they catch just about half of the input weight. Even at best, they miss a fair share of how the model reads speech.
Catalog record
| CREATOR | Sara Papi, Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-18010
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cross-Attention is Half Explanation in Speech-to-Text Models Sara Papi, Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli Abstract Cross-attention is a core mechanism in encoder-decoder architectures, widespread in many fields, including speech-to-text (S2T) processing. Its scores have been repurposed for various downstream applications–such as timestamp estimation and audio-text alignment–under the assumption that they reflect the dependencies between input speech representation and the generated text. While the explanatory nature of attention mechanisms has been widely debated in the broader NLP literature, this assumption remains largely unexplored within the speech domain. To address this gap, we assess the explanatory power of cross-attention in S2T models by comparing its scores to input saliency maps derived from feature attribution. Our analysis spans monolingual and multilingual, single-task and multi-task models at multiple scales, and shows that attention scores moderately to strongly align with saliency-based explanations, particularly when aggregated across heads and layers. However, it also shows that cross-attention captures only about 50% of the input relevance and, in the best case, only partially reflects how the decoder attends to the encoder’s representations–accounting for just 52-75% of the saliency. These findings uncover fundamental limitations in interpreting cross-attention as an explanatory proxy, suggesting that it offers an informative yet incomplete view of the factors driving predictions in S2T models. 1 Introduction Cross-attention ( Bahdanau et al. 2015 ) is the core mechanism of the encoder-decoder Transformer architecture ( Vaswani et al. 2017 ) , a model that has become foundational across numerous AI domains ( Galassi et al. 2021 ; Lin et al. 2022 ; Lee et al. 2023 ; Wang et al. 2024b ; Lu et al. 2024 ) , including natural language processing (NLP). Designed for modeling dependencies between the generated output sequence and the input representations, the cross-attention scores–derived from the attention mechanism–have been leveraged in various NLP tasks ( Hu 2020 ; Zhang & Kim 2023 ) , such as source-target textual alignment ( Garg et al. 2019 ; Chen et al. 2020 ) , co-reference resolution ( Voita et al. 2018 ) , and word sense disambiguation ( Tang et al. 2018 ) . In speech-to-text (S2T) modeling, cross-attention scores have been widely repurposed for diverse downstream applications such as audio-text alignment ( Zhao et al. 2020 ; Lee et al. 2020 ) , speaker identification ( Kim et al. 2019 ) , timestamp estimation ( Li et al. 2022 ; Louradour 2023 ; Zusag et al. 2024 ) , and guiding simultaneous automatic speech recognition (ASR) and speech translation (ST) ( Wang et al. 2024a ; Papi et al. 2023a ; Papi et al. 2023b ) . These applications rely on the implicit assumption that cross-attention reliably indicates what the model attends to in the input signal during output generation. However, despite its widespread use, this assumption has never been verified. A key concern is that cross-attention operates over the encoder’s output sequence–rather than directly on the raw audio–which may have been reorganized or mixed with contextual information. This phenomenon, known as context mixing ( Mohebbi et al. 2023b ) , can potentially obscure the alignment between cross-attention weights and the original input signal. Similar concerns have been extensively debated in the NLP community, where the reliability of attention mechanisms as explanations has been both challenged and defended, leading to conflicting perspectives and empirical evidence ( Serrano & Smith 2019 ; Jain & Wallace 2019 ; Wiegreffe & Pinter 2019 ; Bastings & Filippova 2020 ) . In contrast, this question remains largely underexplored in the speech domain. Existing work on explainability in S2T has primarily focused on self-attention ( Shim et al. 2022 ; Audhkhasi et al. 2022 ; A Shams et al. 2024 ) , or on empirically measuring the effects of context mixing ( Mohebbi et al. 2023a ) , without directly assessing the explanatory potential of cross-attention mechanisms. To address this gap, we present the first systematic analysis of cross-attention as a proxy for input-output dependencies in S2T models. Our study serves two main objectives: i) assessing the validity of using cross-attention as a surrogate for input-output alignment, and ii) evaluating whether it provides insights comparable to formal explainability methods such as feature attribution–while being more lightweight and less computationally expensive to obtain ( Samek et al. 2021 ; Madsen et al. 2022 ) . We compare cross-attention scores with input saliency maps derived from SPES ( Fucci et al. 2025 ) , the current state-of-the-art feature-attribution method in S2T, to determine the extent to which cross-attention captures which input features are relevant for models’ predictions. In addition, we compute saliency maps on encoder outputs and compare them with cross-attention scores to evaluate whether cross-attention fully explains how the decoder uses encoded representations, avoiding potential discrepancies of context mixing. Our analysis spans ASR and ST tasks across monolingual, multilingual, and multitask settings using state-of-the-art speech processing architectures ( Gulati et al. 2020 ) at multiple scales. With consistent trends across different settings, we find that cross-attention exhibits moderate to strong correlations with input saliency maps and aligns more closely with encoder output representations, suggesting an influence of context mixing. However, our results also indicate that the overall explanatory power of cross-attention is limited–accounting for only \sim 50% of input relevance and, at best, 52-75% of encoder output saliency. Our findings uncover fundamental limitations in interpreting cross-attention as an explanatory proxy, suggesting that it provides an informative yet incomplete view of the factors driving predictions in S2T models. 2 Related Works Explainability in Speech-to-Text. Explainable AI (XAI) has emerged to make model behavior more interpretable to humans, thereby supporting informed decision-making and responsible deployment ( Barredo Arrieta et al. 2020 ) . While XAI research has seen a rapid growth in the last years across multiple modalities, including vision and language ( Sharma et al. 2024 ) , progress in the speech domain has lagged. This gap arises from the inherent complexities of speech processing, including the multidimensional nature of speech signals across time and frequency, and the variability in output sequence length ( Wu et al. 2024 ) . Despite these challenges, growing concerns about trustworthiness are driving explainability efforts in speech classification ( Becker et al. 2024 ; Pastor et al. 2024 ) and S2T generation ( Mandel 2016 ; Kavaki & Mandel 2020 ; Trinh & Mandel 2020 ; Markert et al. 2021 ; Wu et al. 2023 ; Wu et al. 2024 ; Fucci et al. 2025 ) . Most of these works rely on perturbation-based methods that assess how input modifications affect model predictions ( Covert et al. 2021b ; Ivanovs et al. 2021 ) . Among these, Fucci et al. 2025 recently proposed a technique for autoregressive S2T models that identifies regions of the spectrogram that most influence predictions to generate saliency maps. However, XAI methods are generally computationally expensive–especially perturbation-based approaches applied to large models ( Luo & Specia 2024 ; Yin et al. 2025 ) –which motivates exploring whether cross-attention, already computed at inference time, could serve as a lightweight alternative in a landscape still lacking efficient explainability tools for speech-based models. Attention as Explanation. Attention mechanisms have been widely used to probe model behavior in text-based NLP, as attention