HeadRouter : Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
Peize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University · 2026
Read HeadRouter : Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models by Peize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23717.txt.
Description
He and nine colleagues present HeadRouter, a training-free token pruning method for large audio language models that respects the fact that attention heads differ across audio tasks. Instead of averaging head scores, it weights heads by task importance to keep the tokens that matter most. On two audio benchmarks it sets a new compression mark, even beating the unpruned model while keeping only 70 percent of audio tokens.
Catalog record
| CREATOR | Peize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41324 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23717