Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Confer

Dayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao · 2025

Read Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Confer by Dayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17705.txt.

Description

Dayan Pan, with Zhaoyang Fu and four co-authors, adapts language models to many tasks at once in a 2025 arXiv paper. Full fine tuning can erase old knowledge, a failure called catastrophic forgetting. Lighter tuning methods save resources but often lag on complex multi task mixes. The authors target the attention modules, the parts that weigh which words matter most.

Their mechanism, called CAM, adjusts attention outputs dynamically as tasks change. It boosts task specific features while guarding general knowledge. A hybrid framework pairs one shared full size module with several small specialist modules. A routing strategy blends their knowledge as needed. Across question answering, code generation, and logic tasks, scores rose by close to four percent on average. That beat existing adaptation approaches.

Catalog record

CREATORDayan Pan, Zhaoyang Fu, Jingyuan Wang, Xiao Han, Yue Zhu, Xiangyu Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17705

Browse the full library · About the Tower of Babel