Injecting Imbalance Sensitivity for Multi-Task Learning
Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong · 2025
Read Injecting Imbalance Sensitivity for Multi-Task Learning by Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08006.txt.
Description
Zhou, Liu, Zhao, and Gong argue that multi-task learning research has worried about the wrong problem. When one model learns several tasks at once, gradient methods have focused on conflicts, where tasks pull the shared network in opposite directions. The authors show empirically that imbalance matters more: a dominant task can swamp the others even without direct conflict. Their fix, IMGrad, adds imbalance sensitivity to a standard gradient method by constraining the sizes of the projected task gradients.
They test the method on mainstream multi-task benchmarks in both supervised learning and reinforcement learning. Performance stayed competitive across the board. The authors present the work as evidence that balancing task influence, not only resolving conflict, should guide how shared models are trained.
Catalog record
| CREATOR | Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08006