HeadRouter : Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models

Peize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University · 2026

Read HeadRouter : Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models by Peize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23717.txt.

Description

He and nine colleagues present HeadRouter, a training-free token pruning method for large audio language models that respects the fact that attention heads differ across audio tasks. Instead of averaging head scores, it weights heads by task importance to keep the tokens that matter most. On two audio benchmarks it sets a new compression mark, even beating the unpruned model while keeping only 70 percent of audio tokens.

Catalog record

CREATORPeize He, Yaodi Luo, Xiaoqian Liu, Xuyang Liu, Jiahang Deng, Yaosong Du, Li Bangyu, Xiyan Gui, Yuxuan Chen, Linfeng Zhang, [1.4ex] EPIC Lab, Shanghai Jiao Tong University, [0.2ex] Sichuan University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41324
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23717

Browse the full library · About the Tower of Babel