Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Confer
Dayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao · 2025
Read Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Confer by Dayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17705.txt.
Description
Dayan Pan, with Zhaoyang Fu and four co-authors, adapts language models to many tasks at once in a 2025 arXiv paper. Full fine tuning can erase old knowledge, a failure called catastrophic forgetting. Lighter tuning methods save resources but often lag on complex multi task mixes. The authors target the attention modules, the parts that weigh which words matter most.
Their mechanism, called CAM, adjusts attention outputs dynamically as tasks change. It boosts task specific features while guarding general knowledge. A hybrid framework pairs one shared full size module with several small specialist modules. A routing strategy blends their knowledge as needed. Across question answering, code generation, and logic tasks, scores rose by close to four percent on average. That beat existing adaptation approaches.
Catalog record
| CREATOR | Dayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17705