Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models
Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu Tencent AI Lab · 2024
Read Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models by Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu Tencent AI Lab (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16545.txt.
Introduction
Zhisong Zhang and co-authors published a research paper in 2024 on why language models lose skill when a long context is encoded in parallel chunks instead of one stream. Chunking is faster and respects the text's structure, but models were never trained that way, so quality drops. The team traced much of the drop to one measure: attention entropy.
When text is split, attention spreads too evenly, and that unusual flatness tracks the errors. Two simple fixes, adding attention sinks and selective attention, pull the flatness back down. Across tasks, the fixes closed much of the gap with normal encoding. The study points to entropy as a dial designers should watch.
Catalog record
| CREATOR | Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu Tencent AI Lab |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-16545
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu Tencent AI Lab Abstract Large language models have shown remarkable performance across a wide range of language tasks, owing to their exceptional capabilities in context modeling. The most commonly used method of context modeling is full self-attention, as seen in standard decoder-only Transformers. Although powerful, this method can be inefficient for long sequences and may overlook inherent input structures. To address these problems, an alternative approach is parallel context encoding, which splits the context into sub-pieces and encodes them parallelly. Because parallel patterns are not encountered during training, naively applying parallel encoding leads to performance degradation. However, the underlying reasons and potential mitigations are unclear. In this work, we provide a detailed analysis of this issue and identify that unusually high attention entropy can be a key factor. Furthermore, we adopt two straightforward methods to reduce attention entropy by incorporating attention sinks and selective mechanisms. Experiments on various tasks reveal that these methods effectively lower irregular attention entropy and narrow performance gaps. We hope this study can illuminate ways to enhance context modeling mechanisms. 1 Introduction Large language models (LLMs) have demonstrated exceptional capabilities across various language tasks ( Achiam et al. 2023 ; Dubey et al. 2024 ) . A key factor contributing to this success is their remarkable ability of context modeling. This capability forms the basics of instruction following ( Ouyang et al. 2022 ; Bai et al. 2022 ) and in-context learning ( Brown et al. 2020 ; Dong et al. 2024 , ICL;) , enabling LLMs to comprehend contexts effectively. Consequently, LLMs can solve tasks directly when provided with appropriate prompts ( Liu et al. 2023 ) . Figure 1: An overview. The upper part illustrates the full attention and parallel context encoding schemes (superscripts denote position encoding), while the lower part shows parallel encoding leads to irregularly high attention entropy for the query tokens. We explore two methods to reduce entropy (§ 4 ): “Sink” means adding attention sinks, and “SEL” means selective attention. To model contexts, most LLMs adopt a similar architectural design: an auto-regressive decoder-only Transformer with full self-attention ( Vaswani et al. 2017 ; Radford et al. 2019 ) . This architecture does not assume contextual independence, allowing each token to attend to all previous tokens. While powerful and flexible, this design is not without concerns. First, full attention requires computational complexity that scales quadratically with the input sequence length. This poses challenges for long sequence processing and necessitates more efficient alternatives ( Tay et al. 2022 ) . Additionally, in many applications, contexts or prompts exhibit natural parallel structures , consisting of independent sub-pieces, such as documents in retrieval-augmented generation ( Lewis et al. 2020 , RAG;) and demonstrations in ICL. It is intuitive to leverage these structures more effectively. To enhance the efficiency of context encoding and leverage the input structures, a natural strategy is to divide the context into sub-pieces, encode each one in parallel, and then concatenate them for final use. Figure 1 illustrates the differences between full attention encoding and parallel context encoding. Compared to full encoding, the parallel approach can reduce the computational complexity since each sub-piece does not interact with others during the context encoding phase, and the parallel input structures are directly utilized for context splitting. However, mainstream LMs typically rely on full attention and haven’t been trained with parallel contexts, posing the question of whether parallel context encoding is compatible with full-attention-based pre-trained LMs . While specialized fine-tuning can ensure compatibility, it can also be computationally costly ( Yen et al. 2024a ; Sun et al. 2024 ; Lu et al. 2024 ) . Recent studies have explored settings that do not require additional fine-tuning but these are limited to specific scenarios, such as restricted numbers of context windows ( Ratner et al. 2023 ) or specific tasks like ICL ( Hao et al. 2022 ) or RAG ( Merth et al. 2024 ) . In contrast, through detailed evaluations over various tasks, we provide more comprehensive analyses of this question, showing the connections between irregular attention entropy and the final performance. We conduct experiments on a variety of language tasks, including language modeling (LM), ICL, RAG and synthetic tasks. Through fair and direct comparisons between full-attention and parallel encoding schemes, we demonstrate that naively applying parallel encoding results in significant performance declines. By analyzing the attention patterns of both schemes, we find that parallel encoding leads to higher attention entropy on the final query tokens (Figure 1 shows a typical example). Furthermore, we discover strong correlations between attention entropy and model performance, suggesting that attention entropy can be an indicator of irregular performance. To address this, we adopt two straightforward methods to reduce attention entropy: attention sinks ( Xiao et al. 2024 ) , which adds a shared prefix to the context that each sub-piece can attend to, and selective attention , which incorporates a hard selection mechanism into the attention operation. Experimental results show that both methods can reduce the irregular attention entropy and mitigate the performance gaps, verifying our hypothesis. Additionally, we provide a detailed analysis of how different selective attention choices affect performance across various tasks. We hope that our analysis could offer insights into exploring alternative context-modeling mechanisms beyond the full attention scheme. 2 Experimental Settings 2.1 Parallel Context Encoding LM (PPL \downarrow ) ICL (Acc \uparrow ) RAG (SubEM \uparrow ) Synthetic (SubEM \uparrow ) 4K 8K 16K 4K 8K 16K 4K 8K 16K 4K 8K 16K Full 5.54 5.35 5.19 55.20 66.00 72.20 61.25 60.25 60.25 99.88 99.50 97.25 P=2 5.83 5.66 5.47 50.80 63.60 70.20 61.50 59.50 57.50 93.81 94.81 95.25 P=4 6.29 6.16 6.04 36.40 57.20 67.80 59.25 50.75 52.50 79.19 81.56 82.44 P=8 6.91 6.92 6.96 29.20 44.40 60.20 53.50 48.75 44.50 25.94 41.00 41.44 P=16 7.69 7.97 8.54 21.00 34.00 46.40 49.00 41.75 40.00 3.38 2.19 2.00 P=32 8.54 9.24 10.87 10.80 17.40 33.60 45.00 39.25 35.25 0.31 0.00 0.00 P=64 9.35 10.46 13.18 5.00 10.80 19.80 45.00 33.00 26.75 0.00 0.00 0.00 Table 1: Comparisons between full-attention and naive parallel encoding with Llama-3.1-8B (results are macro-averaged over all sub-tasks). Here, “P” indicates the parallel degree (the number of context sub-pieces). For each task, we also vary the total sequence lengths (considering 4K, 8K and 16K). In a vanilla Transformer-based LM, to encode a context sequence of N tokens (assuming the use of a decoder-only model with causal masks), each token needs to attend to all preceding tokens in the context. Consequently, we need to calculate the attention scores for \frac{1}{2}\cdot{}N(N+1) token pairs. With parallel context encoding, we split the context into P sub-pieces. 1 1 1 We refer to the number of sub-pieces as parallel degree , which is one of the main variables examined in this study. In this scheme, each piece is encoded separately, and tokens within one piece do not attend to tokens in other pieces. Assuming that we evenly split the context for simplicity, the token-pair calculation requirement is P\cdot{}\frac{1}{2}\cdot{}\frac{N}{P}(\frac{N}{P}+1)=\frac{1}{2P}\cdot{}N(N+P) , which is