HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learning

Xuerui Zhang, Xuehao Wang, Zhan Zhuang, Linglan Zhao, Ziyue Li, Xinmin Zhang, Zhihuan Song, Yu Zhang · 2026

Read HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learning by Xuerui Zhang, Xuehao Wang, Zhan Zhuang, Linglan Zhao, Ziyue Li, Xinmin Zhang, Zhihuan Song, Yu Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26192.txt.

Description

Zhang and co-authors formalize lifelong learning across tasks with different output shapes. Most lifelong work assumes a stream of like tasks, such as classification only. Their setting, lifelong heterogeneous learning, mixes task types over time. They test it on dense prediction tasks. Their method, HAD, keeps old knowledge by self-distillation in each new phase. One loss balances the prediction spread across tasks. Another focuses on sharp edge pixels from the Sobel operator. The method needs no stored examples and beats prior methods, the authors report.

Catalog record

CREATORXuerui Zhang, Xuehao Wang, Zhan Zhuang, Linglan Zhao, Ziyue Li, Xinmin Zhang, Zhihuan Song, Yu Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59505
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26192

Browse the full library · About the Tower of Babel