Attention Sinks Induce Gradient Sinks

Yihong Chen, Quanming Yao · 2026

Read Attention Sinks Induce Gradient Sinks by Yihong Chen, Quanming Yao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17771.txt.

Introduction

Chen, Lin, and Yao link attention sinks to a backward-pass twin they call gradient sinks. In pre-norm transformers, huge residual norms barely touch the forward pass. Under causal masking, sinks concentrate gradients on a few spots during training, they show. RMSNorm then softens that pressure in step with activation size. Their V-scale tweak damps sink gradients and shrinks the massive activations. Attention sinks survive the change, though.

Catalog record

CREATORYihong Chen, Quanming Yao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27533
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17771

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Attention Sinks Induce Gradient Sinks Yihong Chen, Quanming Yao Abstract Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing studies have largely focused on the forward pass, making it unclear whether their connection is direct or mediated by a training-time mechanism. We study this question from the perspective of backpropagation. Empirically and theoretically, we show that under causal mask, attention sinks can induce pronounced gradient concentration, which we term gradient sinks . Furthermore, in pre-norm architectures with RMSNorm, massive activations can be understood as an adaptive response to this localized gradient pressure during training. To test this hypothesis, we introduce V-scale, a modification that adjusts value-path backpropagated gradients. In pretrained V-scale models, attention sinks are preserved whereas massive activations are suppressed. These results support the interpretation that gradient sink is a key training-time mediator linking attention sinks and massive activations. 1 Introduction Two prominent and frequently co-occurring phenomena in Transformer-based large language models (LLMs) are attention sinks (AS), where a small number of tokens attract disproportionate attention mass 33 ; 9 , and massive activations (MA), where activations become unusually large on a small set of tokens and features 5 ; 28 ; 2 . They are predominantly observed on the first token especially in pre-norm LLMs. These phenomena have attracted lots of research interest. A line of work treats AS as exploitable structure for long-context inference and fine-tuning 33 ; 27 ; 16 ; 17 ; 7 ; 14 and structural work studies how to migrate or reshape these phenomena by attention or normalization variants 12 ; 20 ; 13 ; 37 ; 21 ; 6 ; 29 . Particularly, mechanistic studies seek to explain their functions and origins. AS is mainstreamly explained as a way to implement “no-ops” attention heads that contribute little to the current token 9 ; 4 ; 10 , driven by the sum-to-one constraint of the Softmax operator. Meanwhile, MA is often described to help create sink formation 28 ; 26 ; 22 . Together, these prior studies have established that AS and MA are functionally correlated structural artifacts that play important roles in LLMs, even as they impose severe numerical outliers. Yet one question remains unresolved: Why should optimization learn such massive token activations at all? This question is especially nontrivial for modern pre-norm Transformers. In this architecture, both attention and MLP sublayers operate on normalized inputs 35 ; 34 ; 30 . Locally, the forward computation depends much more strongly on representation direction than on raw token norm. This makes MA appear somewhat unnecessary. Most interventions in the literature modify trained models post hoc by changing MLP outputs or token activations and then observe joint changes in AS and MA. While informative, such interventions do not constitute sufficiently clean causal evidence for their relationship, because both magnitude and direction are perturbed at the same time. Recent works that examine training dynamics 8 ; 22 provide valuable additional perspective, but still do not directly isolate the backward mechanism linking sink structure to activation growth. Likewise, showing that AS and MA can be decoupled by architectural changes 29 does not by itself explain why they are learned to be coupled in standard architectures. This paper studies this problem from the perspective of backpropagation. Our starting point is that, under causal mask, sink tokens should not only accumulate attention mass in the forward computation, but also accumulate gradients during backpropagation. As later tokens place attention weights on the first token, their gradients aggregate there. We call this effect a gradient sink (GS). In pre-norm Transformers with RMSNorm, such localized gradient pressure can be partially offset because RMSNorm attenuates backpropagated gradients approximately in inverse proportion to activation norm. This suggests a different interpretation of massive activations: rather than being the direct mechanistic cause of attention sinks, they may emerge as an adaptive response to the gradient imbalance induced by attention sinks. We substantiate this view with complementary evidence. Empirically, we track gradient sinks during training and show that it is concentrated in attention blocks, while RMSNorm-mediated gradient compression scales almost inversely with activation norms. Theoretically, we show that under causal mask, sink structure provides a natural route to gradient aggregation. Finally, we validate the mechanism by intervening the backward path. Our proposed V-scale acts as a gradient valve that attenuates value-path backpropagation of sink tokens. Pretrained V-scale models retain strong AS yet exhibit weaker MA, which is consistent with GS serving as the training-time mediator between the two phenomena. 2 Preliminaries Here, we formalize the Transformer architecture, along with forward and backward observables. These definitions lay the groundwork for studying gradient concentration in subsequent sections. Pre-norm Transformers We study decoder-only Llama-like Transformers with pre-norm residual blocks, RMSNorm, RoPE positional encoding, and SwiGLU MLPs 30 ; 35 ; 25 ; 24 . Let H^{\ell}=(h_{1}^{\ell},\dots,h_{T}^{\ell})^{\top}\in\mathbb{R}^{T\times d_{\mathrm{model}}} denote the hidden states inputting to layer \ell . A pre-norm block updates the sequence as \displaystyle\widetilde{H}^{\ell} \displaystyle=\mathrm{RMSNorm}(H^{\ell}), \displaystyle R^{\mathrm{attn},\ell} \displaystyle=\mathrm{Attention}(\widetilde{H}^{\ell}), \displaystyle H^{\ell+\frac{1}{2}} \displaystyle=H^{\ell}+R^{\mathrm{attn},\ell}, \displaystyle\widetilde{H}^{\ell+\frac{1}{2}} \displaystyle=\mathrm{RMSNorm}(H^{\ell+\frac{1}{2}}), \displaystyle R^{\mathrm{mlp},\ell} \displaystyle=\mathrm{MLP}(\widetilde{H}^{\ell+\frac{1}{2}}), \displaystyle H^{\ell+1} \displaystyle=H^{\ell+\frac{1}{2}}+R^{\mathrm{mlp},\ell}. The Attention block can be either multi-head attention (MHA) 31 or a variant such as grouped-query attention (GQA) 1 . For simplicity, we fix one layer \ell and one attention head h , and suppress the layer/head superscripts unless needed. With RoPE, the query, key, and value states at token position t are q_{t}=R_{t}W_{Q}\widetilde{h}_{t} , k_{t}=R_{t}W_{K}\widetilde{h}_{t} , and v_{t}=W_{V}\widetilde{h}_{t} , where R_{t}\in\mathbb{R}^{d_{\mathrm{head}}\times d_{\mathrm{head}}} is the RoPE rotation matrix. Then the causal attention logits and weights are computed by z_{tj}=\frac{\langle q_{t},k_{j}\rangle}{\sqrt{d}}+m_{tj} and a_{tj}=\mathrm{softmax}(z_{t,:})_{j} , where m_{tj}=0 for j\leq t otherwise -\infty . Finally we have the value aggregation y_{t}=\sum\nolimits_{j\leq t}a_{tj}v_{j} and head output o_{t}=W_{O}y_{t} . Forward metrics for attention sinks and activations For a candidate sink position s (typically the first token), we follow 9 ; 22 and use thresholded criteria to count sink-like behavior: \text{Sink}_{s}^{\epsilon,\ell}=\frac{1}{H}\sum\nolimits_{h=1}^{H}\mathbb{I}(\alpha_{s}^{\ell,h}>\epsilon),\quad\alpha_{s}^{\ell,h}=\frac{1}{T_{\epsilon}}\sum\nolimits_{t=s}^{s+T_{\epsilon}-1}a_{ts}^{\ell,h}, (1) where H is the number of attention heads in each layer. Following prior work 9 we set the threshold \epsilon=0.3 and T_{\epsilon}=64 . In this paper, for theory and training dynamics measurements, we also introduce the following simpler column statistics: \!\!\overline{M}_{s}^{\ell,h}\!=\!\frac{M_{s}^{\ell,h}}{T-s+1}\!=\!\frac{1}{T-s+1}\sum\nolimits_{t=s}^{T}a_{ts}^{\ell,h},\quad\overline{S}_{s}^{\ell,h}\!=\!\frac{S_{s}^{\ell,h}}{T-s+1}\!=\!\frac{1}{T-s+1}\sum\nolimits_{t=s}^{T}(a_{ts}^{\ell,h})^{2}. (2) The quantity \overline{M}_{s} is the average attention mass routed through token s by all causally allowed future queries and \overline{S}_{s} is a second-moment version. In experiments, we focus on the first token ( s=1 ) and further summarize ( 2 ) across heads and layers. For activations, many prior studies identify MA as single activation magnitudes that are large where both token and feature axes are considered 5 ; 28 . In this work, we concern token-wise behaviors, so we identify MA as large token-wise activation norms at an explicitly chosen computational site 10 , including the input of Transformer layers \|h_{t,\mathrm{in}}^{\ell}\|=\|h_{t}^{\ell}\|_{2} , the output of Transformer layers \|h_{t,\mathrm{out}}^{\ell}\|=\|h_{t}^{\ell+1}\|_{2} , and the hidden states after Attention \|h_{t,\mathrm{half}}^{\ell}\|=\|h_{t}^{\ell+{1}/{2}}\|_{2} . We also measure the output of the MLP and the value state. Backward observables We introduce token-wise gradient observables and local gradient-reshaping ratios that expose where gradient pressure is created and how it is subsequently transformed inside pre-norm residual branches. Let \mathcal{L} denote the training loss. Recalling that h_{t}^{\ell+{1}/{2}}=h_{t}^{\ell}+r_{t}^{\mathrm{attn},\ell} for Attention blocks, we have \nabla_{r_{t}^{\mathrm{attn},\ell}}\mathcal{L}=\nabla_{h_{t}^{\ell+{1}/{2}}}\mathcal{L} . Moreover, the difference \nabla_{h_{t}^{\ell}}\mathcal{L}-\nabla_{h_{t}^{\ell+{1}/{2}}}\mathcal{L} isolates the additional gradient contribution propagated through the

Read the complete text