Injecting Imbalance Sensitivity for Multi-Task Learning

Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong · 2025

Read Injecting Imbalance Sensitivity for Multi-Task Learning by Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08006.txt.

Description

Zhou, Liu, Zhao, and Gong argue that multi-task learning research has worried about the wrong problem. When one model learns several tasks at once, gradient methods have focused on conflicts, where tasks pull the shared network in opposite directions. The authors show empirically that imbalance matters more: a dominant task can swamp the others even without direct conflict. Their fix, IMGrad, adds imbalance sensitivity to a standard gradient method by constraining the sizes of the projected task gradients.

They test the method on mainstream multi-task benchmarks in both supervised learning and reinforcement learning. Performance stayed competitive across the board. The authors present the work as evidence that balancing task influence, not only resolving conflict, should guide how shared models are trained.

Catalog record

CREATORZhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-08006

Browse the full library · About the Tower of Babel