Identifying and Evaluating Inactive Heads in Pretrained LLMs
Pedro Sandoval-Segura, Xijun Wang, Ashwinee Panda, Micah Goldblum, Ronen Basri, Tom Goldstein, David Jacobs · 2025
Read Identifying and Evaluating Inactive Heads in Pretrained LLMs by Pedro Sandoval-Segura, Xijun Wang, Ashwinee Panda, Micah Goldblum, Ronen Basri, Tom Goldstein, David Jacobs (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03889.txt.
Description
Pedro Sandoval-Segura and co-authors hunted for dead weight inside large language models: attention heads that contribute almost nothing. Attention heads are meant to focus on relevant words, but some park most of their focus on the first token, a behavior called an attention sink. The team tested twelve scoring functions that flag inactivity in different ways, then removed the flagged heads to see whether the models noticed.
More than twelve percent of heads proved inactive on average. Removing them in the right contexts kept test accuracy within one percent of the original model. Scores based on a head's output size found slack heads that attention weights alone missed, while sink-based scores missed over seven percent of the inactive set. Fine-tuning, they also found, barely changes attention habits.
Catalog record
| CREATOR | Pedro Sandoval-Segura, Xijun Wang, Ashwinee Panda, Micah Goldblum, Ronen Basri, Tom Goldstein, David Jacobs |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03889